99% memory consumption on an Idle RAC nodes

We have been observed that the memory consumption on all RAC nodes(8 node setup) on HPUX Superdom (Itaninum II)  servers are consuming almost 99% memory round the clock.  There are no databases running, just Oracle clusterware was installed on these nodes. We would be soon using this setup for our production, thats our plan.

We initially thought there could be a memory leak which is causing the high memory consumption. Upon opening a TAR for this issue, Oracle support suggested that the memory consumption by the clusterware is pretty normal and there is no high memory consumption by any of Oracle CW processes. Shutting down the cluster in order to test the memory consumption doesn't help either as the memory consumption still was the same.

Then, we thought of investigating from OS point of view and we  found out that the OS kernel parameter 'dbc_max_pct' was set to the default value, i.e. 50%.  The general recommendation on most of the environments is 10%. Thank god, its a dyanmic parameter which can be modified without the shutdown. Upon resetting the value to 10%, the memory consumption drastically down from 99% to 35% on all the nodes.

I have also found a very good Metalink Note (68105.1 : Commonly Misconfigured HP-UX Kernel Parameters) which explains the most commonly misconfigured HP-UX kernal parameters. Dears, you are running any databases on HPUX, I strongly recommend you to have a look at this note and make changes accordingly.

Happy Reading,

Jaffar