[PATCH]mm: Fixing bug in brk

["Ram Gupta" <ram.gupta5@gmail.com>]

This patch fixes a bug in sys_brk . The code checks for newbrk with
oldbrk which are page aligned before  making a check for the memory
limit set of data segment. If the memory limit is not page aligned in
that case it bypasses the test for the limit if the memory allocation
is still for the same page. Please apply

Signed-off-by: Ram Gupta ( ram.gupta5@gmail.com)
----------------------------
--- linux-2.6.16/mm/mmap.c.orig 2006-04-03 10:20:18.000000000 +0000
+++ linux-2.6.16/mm/mmap.c      2006-04-03 09:40:00.000000000 +0000
@@ -220,6 +220,16 @@ asmlinkage unsigned long sys_brk(unsigne

        if (brk < mm->end_code)
                goto out;
+
+       /* Check against rlimit here. If this check is not done here & done
+        * later after the test of oldbrk with newbrk then it can escape the
+        * test & let the data segment grow beyond its set limit  in case
+        * when the limit is not page aligned -Ram Gupta */
+
+       rlim = current->signal->rlim[RLIMIT_DATA].rlim_cur;
+       if (rlim < RLIM_INFINITY && brk - mm->start_data > rlim)
+               goto out;
+
        newbrk = PAGE_ALIGN(brk);
        oldbrk = PAGE_ALIGN(mm->brk);
        if (oldbrk == newbrk)
@@ -232,11 +242,6 @@ asmlinkage unsigned long sys_brk(unsigne
                goto out;
        }

-       /* Check against rlimit.. */
-       rlim = current->signal->rlim[RLIMIT_DATA].rlim_cur;
-       if (rlim < RLIM_INFINITY && brk - mm->start_data > rlim)
-               goto out;
-
        /* Check against existing mmap mappings. */
        if (find_vma_intersection(mm, oldbrk, newbrk+PAGE_SIZE))
                goto out;

Thanks
Ram Gupta
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/