10.10.2018

Few Exadata MOS Docs to review

If you have MOS login credentials and managing Exadata database machines, below is the list of few MOS Doc which is worth reading:

  • 888828.1, "Exadata Database Machine and Exadata Storage Server Supported Versions"
  • 1070954.1, "Oracle Exadata Database Machine Exachk or HealthCheck"
  • 1353073.2, "Exadata Diagnostic Collection Guide"
  • 1500257.1, " Exadata Write-Back Flash Cache - FAQ"
  • 1553103.1, "dbnodeupdate.sh and dbserver.patch.zip: Updating Exadata Database Server Software using the DBNodeUpdate Utility and patchmgr"
  • 1589868.1, "Procedure to check for corrupted root file system on Exadata Storage Servers and Linux database servers"

10.08.2018

(EX42) Flash disk failure may lead to ASM metadata corruption when using write-back flash cache

While reviewing the latest Exachk report on X5-2 machine, the following critical alrams were observed:



And details shows below description:


And the MOS Note : 1270094.1 explains the following:


According to MOS Doc: 2356460.1, the said behavior is due to a bug (27372426) which applies on Exa version 12.2.1.1.0 to 12.2.1.1.5 or 18.1.0.0.0 to 18.1.3.0.0.

Impact:

If you are running GI 11.2.0.4 or 12.1 with the above said Exa version, and  with FlashCache configured as Writeback mode, the following ORA error may encounter, during: ASM rebalancing operation, disk group mount, & disk group consistency checks, ASM review asm alert.log:

ORA-00600: internal error code, arguments: [kfdAuDealloc2]

WARNING: cache read a corrupt block: group=1(DATA) fn=381 indblk=27 disk=110 (DATA_CD_04_DM01CEL01)
ORA-15196: invalid ASM block header [kfc.c:26411] [endian_kfbh]

ORA-00600: internal error code, arguments: [kfrValAcd30]

ORA-00600: internal error code, arguments: [kfdAuPivotVec2], [kfCheckDG]

ERROR: file +DATADG1.3341.962251267: F3341 PX38530 => D55 A765853 => F1677
PX1647463: fnum mismatch
ERROR: file +DATADG1.3341.962251267: F3341 PX38531 => D15 A205431 => F3341
PX56068: xnum mismatch



Workaround:
To fix the bug, Following action plan needs to be applied:

1) Update the storage server to >=12.2.1.1.6 or >=18.1.4.0.0
2) Apply patch 27510959 and scan ASM metadata


Note :

The issues doesn't impact on GI 12.2 or whenever you have higher version of Exa software mentioned in this bug;
The bug also doesn't affect if the FlashCache mode is WriteThrough;

References:

Exadata Critical Issues (Doc ID 1270094.1)