[PATCH]i386: fix overflow in vmap on an x86 system which has more than 4GB memory.

Description
(max_mapnr << PAGE_SHIFT) would overflow on an x86 system which has more
than 4GB memory, and hence cause vmap to fail every time.


Signed-off-by: Michael Chen <[email protected]>

Patch
diff -Nur linux-2.4.21-40.EL/mm/vmalloc.c
linux-2.4.21-40.EL.diff/mm/vmalloc.c
--- linux-2.4.21-40.EL/mm/vmalloc.c     2006-02-02 21:13:20.000000000
-0600
+++ linux-2.4.21-40.EL.diff/mm/vmalloc.c        2006-09-04
11:29:33.000000000 -0500
@@ -298,8 +298,8 @@
        struct vm_struct *area;
        unsigned long size = count << PAGE_SHIFT;
 
-       if (!size || size > (max_mapnr << PAGE_SHIFT))
-               return NULL;
+    if (!count || count > max_mapnr)
+        return NULL;
        area = get_vm_area(size, flags);
        if (!area) {
                return NULL;


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

---

• Follow-Ups:
  • Re: [PATCH]i386: fix overflow in vmap on an x86 system which has more than 4GB memory.
    • From: Hugh Dickins <[email protected]>

• Prev by Date: Re: [-mm patch 2/3] AVR32 MTD: Unlock flash if necessary (try 2)
• Next by Date: Re: cachefiles on latest -mm fails to build on powerpc
• Previous by thread: [PATCH] MTD: Fix bug in fixup_convert_atmel_pri
• Next by thread: Re: [PATCH]i386: fix overflow in vmap on an x86 system which has more than 4GB memory.
• Index(es):
  • Date
  • Thread

[Index of Archives]     [Kernel Newbies]     [Netfilter]     [Bugtraq]     [Photo]     [Stuff]     [Gimp]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Video 4 Linux]     [Linux for the blind]     [Linux Resources]