您好,登錄后才能下訂單哦!
通過案例學Oracle之--一次AIX rac誤操作引起的“血案”
系統環境:
操作系統: AIX 5300-09
集群軟件: CRS 10.2.0.1
數據庫: Oracle 10.2.0.1
本案例是用于基于VG Concurrent 的共享存儲,通過HACMP 實現卷組的并發
案例分析:
一、錯誤現象:
1、Oracle 用戶無法訪問設備文件
2、CRS server啟動失敗
[oracle@aix211 ~]$ls -l /dev
/dev/__vg10: No permission /dev/audit: No permission /dev/cd0: No permission /dev/clone: No permission /dev/console: No permission /dev/error: No permission
查看設備文件屬性,發現被改為oracle:dba
[oracle@aix211 ~]$ls -ld /dev
drw-rw---- 6 oracle dba 3584 Sep 16 11:38 /dev
重新更改設備文件屬性
[root@aix211 /]#chown root.system /dev [root@aix211 /]#ls -ld /dev drw-rw---- 6 root system 3584 Sep 16 11:38 /dev [root@aix211 /]#chmod 775 /dev
Oracle用戶可以正常訪問設備文件
[root@aix211 /]#su - oracle [oracle@aix211 ~]$ls -l /dev total 24 crw-rw---- 1 root system 10, 0 Aug 29 2013 IPL_rootvg srwxrwxrwx 1 root system 0 Sep 16 10:22 SRC brw-rw---- 1 oracle dba 88, 9 Sep 11 12:15 control1_1 brw-rw---- 1 oracle dba 88, 10 Sep 11 12:15 control2_2 brw-rw---- 1 oracle dba 88, 11 Sep 11 12:16 control3_3 crw-rw---- 1 root system 88, 0 Sep 11 12:08 datavg
但是CRS server仍然不能正常啟動!
二、重新配置CRS:
1、清理ocr和vote disk磁盤信息(兩個節點)
[root@aix211 /]#dd if=/dev/zero of=/dev/rrac_ocr bs=8192 count=2560 2560+0 records in 2560+0 records out [root@aix211 /]#dd if=/dev/zero of=/dev/rrac_vote bs=8192 count=2560 2560+0 records in 2560+0 records out [root@aix211 /]#ls -l /dev|grep ocr brw-rw---- 1 oracle dba 88, 1 Sep 11 12:15 rac_ocr crw-r----- 1 root oinstall 88, 1 Sep 16 11:05 rrac_ocr [root@aix211 /]#chown oracle:dba /dev/rrac_ocr
2、重新運行root.sh腳本,配置CRS(兩個節點)
node1:
[root@aix211 install]#./rootdelete.sh
Shutting down Oracle Cluster Ready Services (CRS): Sep 16 11:48:57.011 | ERR | failed to connect to daemon, errno(2) Stopping resources. Error while stopping resources. Possible cause: CRSD is down. Stopping CSSD. Unable to communicate with the CSS daemon. Shutdown has begun. The daemons should exit soon. Checking to see if Oracle CRS stack is down... Oracle CRS stack is not running. Oracle CRS stack is down now. Removing script for Oracle Cluster Ready services Updating ocr file for downgrade Cleaning up SCR settings in '/etc/oracle/scls_scr'
[root@aix211 install]#/u01/crs_1/root.sh
WARNING: directory '/u01' is not owned by root Checking to see if Oracle CRS stack is already configured Checking to see if any 9i GSD is up Setting the permissions on OCR backup directory Setting up NS directories Oracle Cluster Registry configuration upgraded successfully WARNING: directory '/u01' is not owned by root clscfg: EXISTING configuration version 3 detected. clscfg: version 3 is 10G Release 2. Successfully accumulated necessary OCR keys. Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897. node <nodenumber>: <nodename> <private interconnect name> <hostname> node 1: aix211 aix211-priv aix211 node 2: aix212 aix212-priv aix212 clscfg: Arguments check out successfully. NO KEYS WERE WRITTEN. Supply -force parameter to override. -force is destructive and will destroy any previous cluster configuration. Oracle Cluster Registry for cluster has already been initialized Startup will be queued to init within 30 seconds. Adding daemons to inittab Adding daemons to inittab Expecting the CRS daemons to be up within 600 seconds. CSS is active on these nodes. aix211 CSS is inactive on these nodes. aix212 Local node checking complete. Run root.sh on remaining nodes to start CRS daemons.
node2:
[root@aix212 install]#./rootdelete.sh
Shutting down Oracle Cluster Ready Services (CRS): Sep 16 11:48:57.011 | ERR | failed to connect to daemon, errno(2) Stopping resources. Error while stopping resources. Possible cause: CRSD is down. Stopping CSSD. Unable to communicate with the CSS daemon. Shutdown has begun. The daemons should exit soon. Checking to see if Oracle CRS stack is down... Oracle CRS stack is not running. Oracle CRS stack is down now. Removing script for Oracle Cluster Ready services Updating ocr file for downgrade Cleaning up SCR settings in '/etc/oracle/scls_scr'
[root@aix212@ /]#/u01/crs_1/root.sh
WARNING: directory '/u01' is not owned by root Checking to see if Oracle CRS stack is already configured Setting the permissions on OCR backup directory Setting up NS directories Oracle Cluster Registry configuration upgraded successfully WARNING: directory '/u01' is not owned by root clscfg: EXISTING configuration version 3 detected. clscfg: version 3 is 10G Release 2. Successfully accumulated necessary OCR keys. Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897. node <nodenumber>: <nodename> <private interconnect name> <hostname> node 1: aix211 aix211-priv aix211 node 2: aix212 aix212-priv aix212 clscfg: Arguments check out successfully. NO KEYS WERE WRITTEN. Supply -force parameter to override. -force is destructive and will destroy any previous cluster configuration. Oracle Cluster Registry for cluster has already been initialized Startup will be queued to init within 30 seconds. Adding daemons to inittab Adding daemons to inittab Expecting the CRS daemons to be up within 600 seconds. CSS is active on these nodes. aix211 aix212 CSS is active on all nodes. Waiting for the Oracle CRSD and EVMD to start Oracle CRS stack installed and running under init(1M) Running vipca(silent) for configuring nodeapps The given interface(s), "en0" is not public. Public interfaces should be used to configure virtual IPs.
在node2上運行vipca,配置vip
@至此,CRS重新配置成功!
[root@aix212@ /]#crsctl check crs CSS appears healthy CRS appears healthy EVM appears healthy [root@aix212@ /]#crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.aix211.gsd application ONLINE ONLINE aix211 ora.aix211.ons application ONLINE ONLINE aix211 ora.aix211.vip application ONLINE ONLINE aix211 ora.aix212.gsd application ONLINE ONLINE aix212 ora.aix212.ons application ONLINE ONLINE aix212 ora.aix212.vip application ONLINE ONLINE aix212
三、重新注冊Listener和Database
1、注冊listener
通過netca工具,重新reconfigure就可以完成listener的注冊!
[root@aix212@ /]#crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora....11.lsnr application ONLINE ONLINE aix211 ora.aix211.gsd application ONLINE ONLINE aix211 ora.aix211.ons application ONLINE ONLINE aix211 ora.aix211.vip application ONLINE ONLINE aix211 ora....12.lsnr application ONLINE ONLINE aix212 ora.aix212.gsd application ONLINE ONLINE aix212 ora.aix212.ons application ONLINE ONLINE aix212 ora.aix212.vip application ONLINE ONLINE aix212
2、注冊Database和Instance
注冊Database:
[root@aix212@ /]#srvctl add database -h
Usage: srvctl add database -d <name> -o <oracle_home> [-m <domain_name>] [-p <spfile>] [-A <name|ip>/netmask] [-r {PRIMARY | PHYSICAL_STANDBY | LOGICAL_STANDBY}] [-s <start_options>] [-n <db_name>] [-y {AUTOMATIC | MANUAL}] -d <name> Unique name for the database -o <oracle_home> ORACLE_HOME for cluster database -m <domain> Domain for cluster database -p <spfile> Server parameter file for cluster database -A <addr_str> Database cluster alias -n <db_name> Database name (DB_NAME), if different from the unique name given by the -d option -r <role> Role of the database (primary, physical_standby, logical_standby) -s <start_options> Startup options for the database -y <dbpolicy> Management policy for the database (automatic, manual) -h Print usage
[root@aix212@ /]#su - oracle
[oracle@aix212@ ~]$srvctl add database -d prod -o $ORACLE_HOME
[oracle@aix212@ ~]$crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora....11.lsnr application ONLINE ONLINE aix211 ora.aix211.gsd application ONLINE ONLINE aix211 ora.aix211.ons application ONLINE ONLINE aix211 ora.aix211.vip application ONLINE ONLINE aix211 ora....12.lsnr application ONLINE ONLINE aix212 ora.aix212.gsd application ONLINE ONLINE aix212 ora.aix212.ons application ONLINE ONLINE aix212 ora.aix212.vip application ONLINE ONLINE aix212 ora.prod.db application OFFLINE OFFLINE
注冊Instance:
[oracle@aix212@ ~]$srvctl add instance -h Usage: srvctl add instance -d <name> -i <inst_name> -n <node_name> -d <name> Unique name for the database -i <inst> Instance name -n <node> Node name -h Print usage [oracle@aix212@ ~]$srvctl add instance -d prod -i prod1 -n aix211 [oracle@aix212@ ~]$srvctl add instance -d prod -i prod2 -n aix212 [oracle@aix212@ ~]$crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora....11.lsnr application ONLINE ONLINE aix211 ora.aix211.gsd application ONLINE ONLINE aix211 ora.aix211.ons application ONLINE ONLINE aix211 ora.aix211.vip application ONLINE ONLINE aix211 ora....12.lsnr application ONLINE ONLINE aix212 ora.aix212.gsd application ONLINE ONLINE aix212 ora.aix212.ons application ONLINE ONLINE aix212 ora.aix212.vip application ONLINE ONLINE aix212 ora.prod.db application OFFLINE OFFLINE ora....d1.inst application OFFLINE OFFLINE ora....d2.inst application OFFLINE OFFLINE
通過crs工具啟動Database:
[oracle@aix212@ ~]$srvctl start database -d prod
PRKP-1001 : Error starting instance prod1 on node aix211
CRS-0184: Cannot communicate with the CRS daemon.
PRKP-1001 : Error starting instance prod2 on node aix212
CRS-0184: Cannot communicate with the CRS daemon.
啟動Instance失敗,通過sqlplus手工啟動:
[oracle@aix212@ ~]$sqlplus '/as sysdba' SQL*Plus: Release 10.2.0.1.0 - Production on Tue Sep 16 12:08:10 2014 Copyright (c) 1982, 2005, Oracle. All rights reserved. Connected to an idle instance. SQL> startup ORACLE instance started. Total System Global Area 1258291200 bytes Fixed Size 2020552 bytes Variable Size 352324408 bytes Database Buffers 889192448 bytes Redo Buffers 14753792 bytes Database mounted. Database opened.
[oracle@aix211 aix211]$sqlplus '/as sysdba' SQL*Plus: Release 10.2.0.1.0 - Production on Tue Sep 16 12:09:37 2014 Copyright (c) 1982, 2005, Oracle. All rights reserved. Connected to an idle instance. SQL> startup ORACLE instance started. Total System Global Area 1258291200 bytes Fixed Size 2020552 bytes Variable Size 335547192 bytes Database Buffers 905969664 bytes Redo Buffers 14753792 bytes Database mounted. Database opened.
查看crs啟動resource信息:
[oracle@aix211 aix211]$crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora....11.lsnr application ONLINE ONLINE aix211 ora.aix211.gsd application ONLINE ONLINE aix211 ora.aix211.ons application ONLINE ONLINE aix211 ora.aix211.vip application ONLINE ONLINE aix211 ora....12.lsnr application ONLINE ONLINE aix212 ora.aix212.gsd application ONLINE ONLINE aix212 ora.aix212.ons application ONLINE ONLINE aix212 ora.aix212.vip application ONLINE ONLINE aix212 ora.prod.db application ONLINE ONLINE aix211 ora....d1.inst application ONLINE ONLINE aix211 ora....d2.inst application ONLINE ONLINE aix212
再通過crs工具重新啟動Instance:
[oracle@aix211 aix211]$srvctl stop instance -d prod -i prod1
[oracle@aix211 aix211]$crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora....11.lsnr application ONLINE ONLINE aix211 ora.aix211.gsd application ONLINE ONLINE aix211 ora.aix211.ons application ONLINE ONLINE aix211 ora.aix211.vip application ONLINE ONLINE aix211 ora....12.lsnr application ONLINE ONLINE aix212 ora.aix212.gsd application ONLINE ONLINE aix212 ora.aix212.ons application ONLINE ONLINE aix212 ora.aix212.vip application ONLINE ONLINE aix212 ora.prod.db application ONLINE ONLINE aix211 ora....d1.inst application OFFLINE OFFLINE ora....d2.inst application ONLINE ONLINE aix212
[oracle@aix211 aix211]$srvctl start instance -d prod -i prod1
[oracle@aix211 aix211]$crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora....11.lsnr application ONLINE ONLINE aix211 ora.aix211.gsd application ONLINE ONLINE aix211 ora.aix211.ons application ONLINE ONLINE aix211 ora.aix211.vip application ONLINE ONLINE aix211 ora....12.lsnr application ONLINE ONLINE aix212 ora.aix212.gsd application ONLINE ONLINE aix212 ora.aix212.ons application ONLINE ONLINE aix212 ora.aix212.vip application ONLINE ONLINE aix212 ora.prod.db application ONLINE ONLINE aix211 ora....d1.inst application ONLINE ONLINE aix211 ora....d2.inst application ONLINE ONLINE aix212
@至此,通過crs工具可以正常啟動和關閉Database,由于誤操作而引起的血案,搶救成功!
免責聲明:本站發布的內容(圖片、視頻和文字)以原創、轉載和分享為主,文章觀點不代表本網站立場,如果涉及侵權請聯系站長郵箱:is@yisu.com進行舉報,并提供相關證據,一經查實,將立刻刪除涉嫌侵權內容。