What is HugePage?
HugePages is a feature of the Linux kernel which allows
larger pages to manage memory as the alternative to the small 4KB pagesize. For
a detailed introduction, see Document 361323.1
Why Do You Need HugePages?
HugePages is crucial for faster Oracle database performance on
Linux if you have a large RAM and SGA. If your combined database SGAs is large
(like more than 8GB, can even be important for smaller), you will need
HugePages configured. Note that the size of the SGA matters. Advantages of
HugePages are:
Larger Page Size and Less # of Pages: Default page size is 4K
whereas the HugeTLB size is 2048K. That means the system would need to handle
512 times less pages.
Reduced Page Table Walking: Since a HugePage covers greater
contiguous virtual address range than a regular sized page, a probability of
getting a TLB hit per TLB entry with HugePages are higher than with regular
pages. This reduces the number of times page tables are walked to obtain physical
address from a virtual address.
Less Overhead for Memory Operations: On virtual memory systems
(any modern OS) each memory operation is actually two abstract memory
operations. With HugePages, since there are less number of pages to work on,
the possible bottleneck on page table access is clearly avoided.
Less Memory Usage: From the Oracle Database perspective, with
HugePages, the Linux kernel will use less memory to create pagetables to
maintain virtual to physical mappings for SGA address range, in comparison to
regular size pages. This makes more memory to be available for process-private
computations or PGA usage.
No Swapping: We must avoid swapping to happen on Linux OS at all
Document 1295478.1. HugePages are not swappable (whereas regular pages are).
Therefore there is no page replacement mechanism overhead. HugePages are
universally regarded as pinned.
No 'kswapd' Operations: kswapd will get very busy if there is a
very large area to be paged (i.e. 13 million page table entries for 50GB
memory) and will use an incredible amount of CPU resource. When HugePages are
used, kswapd is not involved in managing them. See also Document 361670.1
How to Configure
HugePages
The configuration steps below will guide you to do a persistent
system configuration where you would need to do a complete reboot of the
system. Please plan your operations accordingly:
Step 1: Have the memlock user limit set in
/etc/security/limits.conf file. Set the value (in KB) slightly smaller than
installed RAM. e.g. If you have 64GB RAM installed, you may set:
* soft memlock 60397977
* hard memlock 60397977
There is no harm in setting this value large than your SGA
requirements.
The parameters will be set by default on:
Oracle Linux with oracle-validated package (See Document 437743.1)
installed.
Oracle Exadata DB compute nodes
Step 2: Re-logon to the Oracle product owner account (e.g.
'oracle') and check the memlock limit
$ ulimit -l
60397977
Step 3: If you have Oracle Database 11g or later, the default
database created uses the Automatic Memory Management (AMM) feature which is
incompatible with HugePages. Disable AMM before proceeding. To disable, set the
initialization parameters MEMORY_TARGET and MEMORY_MAX_TARGET to 0 (zero).
Please see Document 749851.1 for further information in case you encounter the
error below:
ORA-00845: MEMORY_TARGET not supported on this system
Step 4: Make sure that all your database instances are up
(including ASM instances) as they would run on production. Use the script
hugepages_settings.sh in Document 401749.1 to calculate the recommended value
for the vm.nr_hugepages kernel parameter. e.g.:
$ ./hugepages_settings.sh
...
Recommended setting: vm.nr_hugepages = 1496
Note: You can also calculate a proper value for the parameter
yourself but that is not advised if you do not have extensive experience with
HugePages and concepts.
Step 5: Edit the file /etc/sysctl.conf and set the vm.nr_hugepages
parameter there:
...
vm.nr_hugepages = 1496
...
This will make the parameter to be set properly with each reboot.
Step 6: Stop all the database instances and reboot the server
Check and Validate the Configuration
After the system is rebooted, make sure that your database
instances (including the ASM instances) are started. Automatic startup via OS
configuration or CRS, or manual startup (whichever method you use) should have
been performed. Check the HugePages state from /proc/meminfo. e.g.:
# grep HugePages /proc/meminfo
HugePages_Total: 1496
HugePages_Free: 485
HugePages_Rsvd: 446
HugePages_Surp: 0
