Hugepages in Linux

I was recently looking into a performance issue on one of the databases. It was Oracle 10.2.0.4 on linux (RHEL 5). Having had the prior experience of working with Oracle on Linux, i could quickly make out that the issue was related to not using Huge pages for the SGA. I thought i would blog about the issue and while looking for similar articles on the web, found one Riyaz Shamshuddin. Since it was quite identical to our issue, i decided to link it here – Performance tuning: HugePages and Linux

Some notes from our site
1) We had a RAM of around 16GB and our SGA was around 4GB. Our pagetables without Hugepages were around 6.5GB!

/proc/meminfo before hugepages

MemTotal:     16305888 kB
MemFree:        823340 kB
.............
PageTables:    6631560 kB

/proc/meminfo after confuguring hugepages

MemTotal:     16305888 kB
MemFree:         84268 kB
..........
PageTables:     276628 kB
HugePages_Total:  2050
HugePages_Free:    356
HugePages_Rsvd:    355
Hugepagesize:     2048 kB

2) Modifying vm.nr_hugepages in /etc/sysctl.conf and reloading it using sysctl -p did not work in our case – probably because the system could not get contiguous physical memory. We had to reboot the system.

3) Due to a calculation mistake i had given one “0” less for the ulimit parameter “memlock” in /etc/securities/limits.conf. So the first time we started the instance, the SGA did not sit in the Huge pages. Restarting the instance after modifying /etc/securities/limits.conf with the proper value resolved the issue.

For those who have access there is also a very good article on Metalink about Hugepages on Linux- Article ID 361323.1