Re: Inconsistent timing results of multithreaded program on an SMP machine.

[Marcel Zalmanovici <MARCEL@il.ibm.com>]

                      Con Kolivas                                                                                                  
                      <kernel@kolivas.o        To:       Marcel Zalmanovici/Haifa/IBM@IBMIL                                        
                      rg>                      cc:       linux-kernel@vger.kernel.org                                              
                                               Subject:  Re: Inconsistent timing results of multithreaded program on an SMP        
                      20/11/2005 12:28          machine.                                                                           
                                                                                                                                   
                                                                                                                                   
                                                                                                                                   










On Sun, 20 Nov 2005 21:18, Marcel Zalmanovici wrote:
> Hi Con,
>
> Thanks for the quick reply.

No problems.

Please reply below what you are replying to so we can keep track of email
threads.

>                       <kernel@kolivas.o        To:       Marcel
> Zalmanovici/Haifa/IBM@IBMIL rg>                      cc:
> linux-kernel@vger.kernel.org Subject:  Re: Inconsistent timing results of
> multithreaded program on an SMP 20/11/2005 11:54          machine.
>
> On Sun, 20 Nov 2005 20:27, Marcel Zalmanovici wrote:
> > Hi,
> >
> > I am trying, as part of my thesis, to make some improvement to the
linux
> > scheduler.
> > Therefore I've written a small multithreaded application and tested how
> > long on average it takes for it to complete.
> >
> > The results were very surprising. I expected to see the completion time
> > vary 1 to 2 seconds at most.
> > Instead what I've got was an oscillation where the maximum time was
twice
> > and more than the minimum!! For a short test results ranged ~7sec to
~16
> > sec, and the same happened to longer tests where the min was ~1min and
>
> the
>
> > max ~2:30min.
> >
> > Does anyone have any clue as to what might happen ?
> > Is there anything I can do to get stable results ?
> >
> > Here is a small test case program:
> >
> > (See attached file: sched_test.c)
> >
> > The test was always done on a pretty much empty machine. I've tried
both
> > kernel 2.6.4-52 and 2.6.13.4 but the results were the same.
> >
> > I'm working on a Xeon Intel machine, dual processor, hyperthreaded.
>
>                                                        ^^^^^^^^^^^^^^
>
> I suspect what you're seeing is the random nature of threads to bind
either
> to
> a hyperthread sibling or a real core. If all your threads land on only
one
> physical cpu and both hyperthread siblings it will run much slower than
if
> half land on one physical cpu and half on the other physical cpu. To
> confirm
> this, try setting cpu affinity just for one logical core of each phyiscal
> cpu
> (like affinity for cpu 1 and 3 only for example) or disabling hyperthread
> in
> the bios.

Ok I've had a look at the actual program now ;) Are you timing the time it
takes to completion of everything?

Yes. I'm timing the whole program.

This part of your program:
             for (i= 0; i<8; i++)
                         pthread_join(tid[i], NULL);

Cares about the order the threads finish. Do you think this might be
affecting
your results?

I highly doubt that. The worst case scenario would have thread 0 finish
last right ? but then, all other queries would simply return without
waiting.

Cheers,
Con


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/