ORA-30013: undo tablespace currently in use故障處理思路

當日早上，某系統數倉數據庫告警，數據庫版本為12c，操作系統為RHEL7.2

2018-08-23T06:43:17.297341+08:00
PDB$SEED(2):Opatch validation is skipped for PDB PDB$SEED (con_id=0)
PDB$SEED(2):
PDB$SEED(2):WARNING: Pluggable Database PDB$SEED with pdb id - 2 is
PDB$SEED(2): altered with errors or warnings. Please look into
PDB$SEED(2): PDB_PLUG_IN_VIOLATIONS view for more details.
PDB$SEED(2):
2018-08-23T06:43:25.423893+08:00
PDB$SEED(2):Opening pdb with no Resource Manager plan active
2018-08-23T06:44:17.773603+08:00
DCDB(3):Autotune of undo retention is turned off.
2018-08-23T06:44:17.870219+08:00
DCDB(3):attach called for domid 3 (domuid: 0x786a7683, options: 0x4, pid: 191312)
DCDB(3):queued attach broadcast request 0x12372e9f68
2018-08-23T06:44:18.028918+08:00
DWDBPDB(4):Autotune of undo retention is turned off.
2018-08-23T06:44:18.103157+08:00
DWDBPDB(4):attach called for domid 4 (domuid: 0xd95b03fa, options: 0x4, pid: 191314)
DWDBPDB(4):queued attach broadcast request 0x12372e9f10
2018-08-23T06:44:18.394250+08:00
DCDB(3):Endian type of dictionary set to little
2018-08-23T06:44:18.623678+08:00
DWDBPDB(4):Endian type of dictionary set to little
2018-08-23T06:44:19.122431+08:00
DCDB(3):Undo initialization errored: err:30013 serial:0 start:858517331 end:858517597 diff:266 ms (0.3 seconds)
Pdb DCDB hit error 30013 during open read write (1) and will be closed.

嘗試了一些手段，但是庫無法啟動，我們都知道，RAC系統正常情況是DB1使用untbs1，DB2使用untbs2。登陸系統查看也都正常。
16:56:09 > show parameter undo

NAME TYPE VALUE

_undo_autotune boolean FALSE
temp_undo_enabled boolean FALSE
undo_management string AUTO
undo_retention integer 7200
undo_tablespace string UNDOTBS1

16:56:09 > show parameter undo

NAME TYPE VALUE

_undo_autotune boolean FALSE
temp_undo_enabled boolean FALSE
undo_management string AUTO
undo_retention integer 7200
undo_tablespace string UNDOTBS2

無奈提交SR尋找后線支持，Oracle工程師反饋查詢：
select a.SID,a.NAME,a.VALUE$,b.PDB_NAME
from
sys.pdb_spfile$ a,cdb_pdbs b
where a.PDB_UID=b.CON_UID;
15:21:33 > col PDB_NAME for a20
15:21:42 > /

SID NAME VALUE$ PDB_NAME

• db_securefile 'PREFERRED' PDB$SEED
• parallel_force_local TRUE DWDBPDB
• db_securefile 'PREFERRED' DWDBPDB
• undo_tablespace 'UNDOTBS1' DWDBPDB
  DWDB1 undo_tablespace 'UNDOTBS1' DWDBPDB
  DWDB2 undo_tablespace 'UNDOTBS2' DWDBPDB
  dwdb1 undo_tablespace 'UNDOTBS1' DWDBPDB
  dwdb2 undo_tablespace 'UNDOTBS2' DWDBPDB
• undo_tablespace 'UNDOTBS2' DCDB
• db_securefile 'PREFERRED' DCDB

10 rows selected.

SR繼續反饋Please use following commands:

ALTER SESSION SET CONTAINER =DCDB;
ALTER SYSTEM SET undo_tablespace=UNDOTBS1 scope=spfile sid='dwdb1';
ALTER SYSTEM SET undo_tablespace=UNDOTBS2 scope=spfile sid='dwdb2';
alter pluggable database DCDB open;

Alert日志：
2018-08-23T16:18:39.584321+08:00
LOGMINER: End mining logfile for session 1 thread 1 sequence 212717, +DATA/DWDB/ONLINELOG/redo12a.log
2018-08-23T16:18:39.659829+08:00
LOGMINER: Begin mining logfile for session 1 thread 1 sequence 212718, +DATA/DWDB/ONLINELOG/redo13a.log
2018-08-23T16:18:46.032160+08:00
DCDB(3):ALTER SYSTEM SET undo_tablespace='UNDOTBS1' SCOPE=SPFILE SID='dwdb1' PDB='DCDB';
2018-08-23T16:18:52.234874+08:00
DCDB(3):ALTER SYSTEM SET undo_tablespace='UNDOTBS2' SCOPE=SPFILE SID='dwdb2' PDB='DCDB';
2018-08-23T16:19:11.306657+08:00
DCDB(3):alter pluggable database DCDB open

我們在查詢一下：
16:56:08 > select a.SID,a.NAME,a.VALUE$,b.PDB_NAME
16:56:09 2 from
16:56:09 3 sys.pdb_spfile$ a,cdb_pdbs b
16:56:09 4 where a.PDB_UID=b.CON_UID;

SID NAME VALUE$ PDB_NAME

• db_securefile 'PREFERRED' PDB$SEED
• parallel_force_local TRUE DWDBPDB
• db_securefile 'PREFERRED' DWDBPDB
• undo_tablespace 'UNDOTBS1' DWDBPDB
  DWDB1 undo_tablespace 'UNDOTBS1' DWDBPDB
  DWDB2 undo_tablespace 'UNDOTBS2' DWDBPDB
  dwdb1 undo_tablespace 'UNDOTBS1' DWDBPDB
  dwdb2 undo_tablespace 'UNDOTBS2' DWDBPDB
• undo_tablespace 'UNDOTBS2' DCDB
  dcdb1 undo_tablespace 'UNDOTBS1' DCDB
• db_securefile 'PREFERRED' DCDB
  dcdb2 undo_tablespace 'UNDOTBS2' DCDB
  dwdb1 undo_tablespace 'UNDOTBS1' DCDB
  dwdb2 undo_tablespace 'UNDOTBS2' DCDB

14 rows selected.

這里有個坑，需要我們指定對SID為正確的PDB，這里有個烏龍，Oracle SR后臺工程師讓此之前有錯誤建議：

ALTER SESSION SET CONTAINER =DCDB;
ALTER SYSTEM SET undo_tablespace=UNDOTBS1 scope=spfile sid='dcdb1';
ALTER SYSTEM SET undo_tablespace=UNDOTBS2 scope=spfile sid='dcdb2';
alter pluggable database DCDB open;

The difference is that:

2018-08-23T15:53:13.142757+08:00
DCDB(3):ALTER SYSTEM SET undo_tablespace='UNDOTBS1' SCOPE=SPFILE SID='dcdb1' PDB='DCDB'; ---錯誤
《==DCDB(3): this command is executed in pdb DCDB

2018-08-23T11:07:37.996006+08:00
ALTER SYSTEM SET undo_tablespace='UNDOTBS1' SCOPE=BOTH SID='dwdb1'; --正確

<=======this command is executed in cdb.

好記性不如爛筆頭，特此記錄一下12c的那些坑~