ptrace() associated seg faults using GDB, PowerPC versus x86

["Thomas E. Besemer" <tbesemer@tbcorp.com>]

Using 2.6.16.20, on both x86 and PowerPC, I am finding that
GDB will cause a seg fault on PowerPC, but not x86, using
the same test fixture.

The problem is associated with calling a subroutine in a
stopped process.  The attached C file provides a very
reproducible fault:

 - In x86, I can attach to the running process, and call
   'runStub()' without problems.

 - In PowerPC, I can attach.  I call 'runStub()', but it
   does not return until the sleep() times out, and then
   there is a seg fault.

I do not believe this is a GDB issue; GDB was used to help
diagnose the original problem in my application - using
ptrace() to invoke subroutines on a stopped process.  I
was met with continued seg faults on PowerPC, but not in
x86 world.  I have spent a lot of time looking at this, and
am extremely comfortable that the problem lay in either the
ptrace() implementation, or possibly the signal handling
functions.  More likely ptrace().

I have looked at the ~arch specific routines for ptrace, and
signal handling.  I believe that these machine specific
files behave very differently, due to architecture specific
issues.

I am interested in finding a person who understands how
ptrace() works on the PowerPC, possibly the maintainer of
it, to do some back and forth to resolve/understand these
issues.  I will be happy to find/fix, with some guidance and
understanding.

Attachment: t1.c
Description: Binary data