Re: [PATCH]i386: fix overflow in vmap on an x86 system which has more than 4GB memory.

[Hugh Dickins <hugh@veritas.com>]

This is a 2.4 fix (not needed in 2.6): let's CC maintainer Willy Tarreau.

On Fri, 15 Sep 2006, Anatoli Antonovitch wrote:

> Description
> (max_mapnr << PAGE_SHIFT) would overflow on an x86 system which has more
> than 4GB memory, and hence cause vmap to fail every time.

Good point, thanks for the patch.  Sorry I'm so slow to get to it.

> 
> Signed-off-by: Michael Chen <micche@ati.com>
> 
> Patch
> diff -Nur linux-2.4.21-40.EL/mm/vmalloc.c
> linux-2.4.21-40.EL.diff/mm/vmalloc.c
> --- linux-2.4.21-40.EL/mm/vmalloc.c     2006-02-02 21:13:20.000000000
> -0600
> +++ linux-2.4.21-40.EL.diff/mm/vmalloc.c        2006-09-04

And still needs fixing in latest mainline 2.4.

> 11:29:33.000000000 -0500
> @@ -298,8 +298,8 @@
>         struct vm_struct *area;
>         unsigned long size = count << PAGE_SHIFT;
>  
> -       if (!size || size > (max_mapnr << PAGE_SHIFT))
> -               return NULL;
> +    if (!count || count > max_mapnr)
> +        return NULL;

I'm afraid the tabs got messed up in both the old and new lines.
Also, count is a signed int (whereas size and max_mapnr are both
unsigned longs), so best reject "count <= 0" rather than just "!count".

>         area = get_vm_area(size, flags);
>         if (!area) {
>                 return NULL;

Here's a replacement patch for Willy.  Anatoli, you didn't sign
off the patch yourself: so I'm assuming Michael is the originator.


From: Michael Chen <micche@ati.com>

(max_mapnr << PAGE_SHIFT) would overflow on a system which has
4GB memory or more, and so could cause vmap to fail every time.

Signed-off-by: Michael Chen <micche@ati.com>
Signed-off-by: Hugh Dickins <hugh@veritas.com>
---

 mm/vmalloc.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- 2.4.34-pre3/mm/vmalloc.c	2004-04-14 14:05:41.000000000 +0100
+++ linux/mm/vmalloc.c	2006-09-23 17:52:59.000000000 +0100
@@ -293,7 +293,7 @@ void * vmap(struct page **pages, int cou
 	struct vm_struct *area;
 	unsigned long size = count << PAGE_SHIFT;
 
-	if (!size || size > (max_mapnr << PAGE_SHIFT))
+	if (count <= 0 || count > max_mapnr)
 		return NULL;
 	area = get_vm_area(size, flags);
 	if (!area) {
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/