On Sat, Apr 09, 2005 at 05:52:32PM -0400, Trond Myklebust wrote:
> lau den 09.04.2005 Klokka 23:35 (+0200) skreiv Jakob Oestergaard:
>
> > 2.6.11.6: (dual PIII 1GHz, 2G RAM, Intel e1000)
> >
> > File Block Num Seq Read Rand Read Seq Write Rand Write
> > Dir Size Size Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
> > ------- ------ ------- --- ----------- ----------- ----------- -----------
> > . 2000 4096 1 38.34 18.8% 19.61 6.77% 22.53 23.4% 6.947 15.6%
> > . 2000 4096 2 52.82 29.0% 24.42 9.37% 24.20 27.0% 7.755 16.7%
> > . 2000 4096 4 62.48 34.8% 33.65 17.0% 24.73 27.6% 8.027 15.4%
> >
> >
> > 44MiB/sec for 2.4 versus 22MiB/sec for 2.6 - any suggestions as to how
> > this could be improved?
>
> What happened to the retransmission rates when you changed to TCP?
tcp with timeo=600 causes retransmits (as seen with nfsstat) to drop to
zero.
>
> Note that on TCP (besides bumping the value for timeo) I would also
> recommend using a full 32k r/wsize instead of 4k (if the network is of
> decent quality, I'd recommend 32k for UDP too).
32k seems to be default for both UDP and TCP.
The network should be of decent quality - e1000 on client, tg3 on
server, both with short cables into a gigabit switch with plenty of
backplane headroom.
> The other tweak you can apply for TCP is to bump the value
> for /proc/sys/sunrpc/tcp_slot_table_entries. That will allow you to have
> several more RPC requests in flight (although that will also tie up more
> threads on the server).
Changing only to TCP gives me:
File Block Num Seq Read Rand Read Seq Write Rand Write
Dir Size Size Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
------- ------ ------- --- ----------- ----------- ----------- -----------
. 2000 4096 1 47.04 65.2% 50.57 26.2% 24.24 29.7% 6.896 28.7%
. 2000 4096 2 55.77 66.1% 61.72 31.9% 24.13 33.0% 7.646 26.6%
. 2000 4096 4 61.94 68.9% 70.52 42.5% 25.65 35.6% 8.042 26.7%
Pretty much the same as before - with writes being suspiciously slow
(compared to good ole' 2.4.25)
With tcp_slot_table_entries bumped to 64 on the client (128 knfsd
threads on the server, same as in all tests), I see:
File Block Num Seq Read Rand Read Seq Write Rand Write
Dir Size Size Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
------- ------ ------- --- ----------- ----------- ----------- -----------
. 2000 4096 1 60.50 67.6% 30.12 14.4% 22.54 30.1% 7.075 27.8%
. 2000 4096 2 59.87 69.0% 34.34 19.0% 24.09 35.2% 7.805 30.0%
. 2000 4096 4 62.27 69.8% 44.87 29.9% 23.07 34.3% 8.239 30.9%
So, reads start off better, it seems, but writes are still half speed of
2.4.25.
I should say that it is common to see a single rpciod thread hogging
100% CPU for 20-30 seconds - that looks suspicious to me, other than
that, I can't really point my finger at anything in this setup.
Any suggestions Trond? I'd be happy to run some tests for you if you
have any idea how we can speed up those writes (or reads for that
matter, although I am fairly happy with those).
--
/ jakob
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
