Hi,
We reevaluated shared pagetables with recent patches from Dave. As with
our previous evaluation, a database transaction-processing workload was
used. This time our evaluation focused on a 4-way x86-64 configuration
with 8 GB of memory.
In the case that the bufferpools were in small pages, shared pagetables
provided a 27% improvement in transaction throughput. The performance
increase is attributable to multiple factors. First, pagetable memory
consumption was reduced from 1.65 GB to 51 MB, freeing up 20% of the
system's memory. This memory was devoted to enlarging the database
bufferpools, which allowed more database data to be cached in memory.
The effect of this was to reduce the number of disk I/O's per
transaction by 23%, which contributed to a similar reduction in the
context switch rate. A second major component of the performance
improvement is reduced TLB and cache miss rates, due to the smaller
pagetable footprint. To try to isolate this benefit, we performed an
experiment where pagetables were shared, but the database bufferpools
were not enlarged. In this configuration, shared pagetables provided a
9% increase in database transaction throughput. Analysis of processor
performance counters revealed the following benefits from pagetable sharing:
- ITLB and DTLB page walks were reduced by 27% and 26%, respectively.
- L1 and L2 cache misses were reduced by 5%. This is due to fewer
pagetable entries crowding the caches.
- Front-side bus traffic was reduced approximately 10%.
When the bufferpools were in hugepages, shared pagetables provided a 3%
increase in database transaction throughput. Some of the underlying
benefits of pagetable sharing were as follows:
- Pagetable memory consumption was reduced from 53 MB to 37 MB.
- ITLB and DTLB page walks were reduced by 28% and 10%, respectively.
- L1 and L2 cache misses were reduced by 2% and 6.5%, respectively.
- Front-side bus traffic was reduced by approximately 4%.
The database transaction throughput achieved using small pages with
shared pagetables (with bufferpools enlarged) was within 3% of the
transaction throughput achieved using hugepages without shared
pagetables. Thus shared pagetables provided nearly all the benefit of
hugepages, without the requirement of having to deal with limitations of
hugepages. We believe this would be a significant benefit to customers
running these types of workloads.
We also measured the benefit of shared pagetables on our larger setups.
On our 4-way x86-64 setup with 64 GB memory, using small pages for the
bufferpools, shared pagetables provided a 33% increase in transaction
throughput. Using hugepages for the bufferpools, shared pagetables
provided a 3% increase. Performance with small pages and shared
pagetables was within 4% of the performance using hugepages without
shared pagetables.
On our ppc64 setups we used both Oracle and DB2 to evaluate the benefit
of shared pagetables. When database bufferpools were in small pages,
shared pagetables provided an increase in database transaction
throughput in the range of 60-65%, while in the hugepage case the
improvement was up to 2.4%.
We thank Kshitij Doshi and Ken Chen from Intel for their assistance in
analyzing the x86-64 data.
Cheers,
Brian
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
