I was recently looking into a performance issue on one of the databases. It was Oracle 10.2.0.4 on linux (RHEL 5). Having had the prior experience of working with Oracle on Linux, i could quickly make out that the issue was related to not using Huge pages for the SGA. I thought i would blog about the issue and while looking for similar articles on the web, found one Riyaz Shamshuddin. Since it was quite identical to our issue, i decided to link it here – Performance tuning: HugePages and Linux
Some notes from our site
1) We had a RAM of around 16GB and our SGA was around 4GB. Our pagetables without Hugepages were around 6.5GB!
/proc/meminfo before hugepages MemTotal: 16305888 kB MemFree: 823340 kB ............. PageTables: 6631560 kB
/proc/meminfo after confuguring hugepages
MemTotal: 16305888 kB MemFree: 84268 kB .......... PageTables: 276628 kB HugePages_Total: 2050 HugePages_Free: 356 HugePages_Rsvd: 355 Hugepagesize: 2048 kB
2) Modifying vm.nr_hugepages in /etc/sysctl.conf and reloading it using sysctl -p did not work in our case – probably because the system could not get contiguous physical memory. We had to reboot the system.
3) Due to a calculation mistake i had given one “0” less for the ulimit parameter “memlock” in /etc/securities/limits.conf. So the first time we started the instance, the SGA did not sit in the Huge pages. Restarting the instance after modifying /etc/securities/limits.conf with the proper value resolved the issue.
For those who have access there is also a very good article on Metalink about Hugepages on Linux- Article ID 361323.1