Oracle 11gR2 addNode.sh fail with SEVERE:Cannot perform add node proceduure

Since December 2011, we have been struggling with an issue whilst adding two new nodes to our existing Oracle 11gR2 cluster that is running over HPUX platform. Just for the records, this environment had been recently upgraded from 10gR2 to 11gR2 and successfully applied the PSU ( in the recent past to over come from a significant number of Oracle bugs that we confronted.

We simple followed a very straight forward addNode.sh procedure to add two new nodes which unfortunately failed with the following error:

SEVERE:Cannot perform add node procedure as the value of CLUSTER_NEW_VIRTUAL_HOSTNAMES or CLUSTER_NEW_NODES or both could not be obtained from the command line or response file(s). Silent install cannot continue.

We gave up eventually after having several unsuccessfully attempts to fix the issue and finally opened a TAR with Oracle support to seek their assistance over the issue. Unfortunately, all workarounds suggested by the Oracle Support team doesn't yield any fruitful results, therefore, they have requested their development team to register a new bug for the issue. A new BUG 13555953 has been registered for the problem and still the development team is working over the issue. The problem with the development team is that they are unable to reproduce or simulate the bug at their premises.

However, last week we thought of giving a try adding a node at a time, rather than going with two nodes together. Bingo, the approach worked big time and we were able to add the node with small scale of hiccups.
The problem we got was resolved by applying the solution mentioned in the MOS [ID 1085893.1].

Though we have managed to add two nodes individually, the problem whilst attempting to add two nodes together is still persists. The Oracle development team is still working over the issue and they are yet give us the the root cause and a fix for the issue. We are having a migration project in the pipe line where we would be migrating our existing servers from HPUX SD1 to SD2. If the only workaround is to add a node at a given time, then, it gonna consume a significant time for us to complete the migration project.

I am very optimistic about getting a fix for the bug from Oracle and I shall keep posted the development on my blog.

Happy reading,


No comments: