Oracle race多路基软件变更后聚合盘路径改变后的处理
2016/10/5 10:32:31$ ssh 192.168.1.155
Connecting to 192.168.1.155:22...
Connection established.
To escape to local shell, press 'Ctrl+Alt+]'.
Last login: Wed Oct5 18:30:56 2016 from 192.168.1.1
#
#
#
#
# cat /etc/hosts
127.0.0.1 localhost
192.168.1.155 node1
192.168.1.156 node2
192.168.1.157 node1-vip
192.168.1.158 node2-vip
192.168.1.159 node-scan
10.10.5.155 node1-priv
10.10.5.156 node2-priv
# su - oragrid
$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.ARCH.dg ora....up.type ONLINE ONLINE node1
ora.DATA.dg ora....up.type ONLINE ONLINE node1
ora....ER.lsnr ora....er.type ONLINE ONLINE node1
ora....N1.lsnr ora....er.type ONLINE ONLINE node2
ora.OCRVOTE.dg ora....up.type ONLINE ONLINE node1
ora.asm ora.asm.type ONLINE ONLINE node1
ora.cvu ora.cvu.type ONLINE ONLINE node1
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE node1
ora....SM1.asm application ONLINE ONLINE node1
ora....E1.lsnr application ONLINE ONLINE node1
ora.node1.gsdapplication OFFLINE OFFLINE
ora.node1.onsapplication ONLINE ONLINE node1
ora.node1.vipora....t1.type ONLINE ONLINE node1
ora....SM2.asm application ONLINE ONLINE node2
ora....E2.lsnr application ONLINE ONLINE node2
ora.node2.gsdapplication OFFLINE OFFLINE
ora.node2.onsapplication ONLINE ONLINE node2
ora.node2.vipora....t1.type ONLINE ONLINE node2
ora.oc4j ora.oc4j.typeONLINE ONLINE node1
ora.ons ora.ons.type ONLINE ONLINE node1
ora.orcc.db ora....se.type ONLINE ONLINE node1
ora....ry.acfs ora....fs.type ONLINE ONLINE node1
ora.scan1.vipora....ip.type ONLINE ONLINE node2
$ crsctl check cluster -all
**************************************************************
node1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
node2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
$ multipath -ll
need to be root
$ exit
logout
# multipath -ll
arch (14f504e46494c4500764e577843332d547243472d37417072) dm-4 OPNFILER,VIRTUAL-DISK
size=8.3G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 37:0:0:0 sdg 8:96active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
`- 38:0:0:0 sdh 8:112 active ready running
vote_ocr (14f504e46494c45004932325a59732d757266762d4f486c59) dm-5 OPNFILER,VIRTUAL-DISK
size=5.6G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 34:0:0:0 sdd 8:48active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
`- 33:0:0:0 sdc 8:32active ready running
data (14f504e46494c45007469355367682d513879662d6d344836) dm-3 OPNFILER,VIRTUAL-DISK
size=11G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 36:0:0:0 sdf 8:80active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
`- 35:0:0:0 sde 8:64active ready running
#
#
#
# fdisk -l
Disk /dev/sda: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0002d1a4
Device Boot Start End Blocks IdSystem
/dev/sda1 * 1 256 2048000 83Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 256 1275 8192000 82Linux swap / Solaris
/dev/sda3 1275 6528 42187776 8eLinux LVM
Disk /dev/sdb: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xf289c154
Device Boot Start End Blocks IdSystem
/dev/sdb1 1 2610 20964793+8eLinux LVM
Disk /dev/mapper/vg01-LogVol00: 20.3 GB, 20329791488 bytes
255 heads, 63 sectors/track, 2471 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/vg01-Log_ora: 15.7 GB, 15728640000 bytes
255 heads, 63 sectors/track, 1912 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/vg01-LogVol02: 15.7 GB, 15728640000 bytes
255 heads, 63 sectors/track, 1912 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdc: 6006 MB, 6006243328 bytes
185 heads, 62 sectors/track, 1022 cylinders
Units = cylinders of 11470 * 512 = 5872640 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdd: 6006 MB, 6006243328 bytes
185 heads, 62 sectors/track, 1022 cylinders
Units = cylinders of 11470 * 512 = 5872640 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sde: 11.8 GB, 11844714496 bytes
64 heads, 32 sectors/track, 11296 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdf: 11.8 GB, 11844714496 bytes
64 heads, 32 sectors/track, 11296 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdg: 8959 MB, 8959033344 bytes
64 heads, 32 sectors/track, 8544 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdh: 8959 MB, 8959033344 bytes
64 heads, 32 sectors/track, 8544 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/data: 11.8 GB, 11844714496 bytes
255 heads, 63 sectors/track, 1440 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/arch: 8959 MB, 8959033344 bytes
255 heads, 63 sectors/track, 1089 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/vote_ocr: 6006 MB, 6006243328 bytes
255 heads, 63 sectors/track, 730 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
# lsscsi
disk VMware,VMware Virtual S 1.0 /dev/sda
disk VMware,VMware Virtual S 1.0 /dev/sdb
cd/dvdNECVMWar VMware SATA CD01 1.00/dev/sr0
disk OPNFILER VIRTUAL-DISK 0 /dev/sdc
disk OPNFILER VIRTUAL-DISK 0 /dev/sdd
disk OPNFILER VIRTUAL-DISK 0 /dev/sde
disk OPNFILER VIRTUAL-DISK 0 /dev/sdf
disk OPNFILER VIRTUAL-DISK 0 /dev/sdg
disk OPNFILER VIRTUAL-DISK 0 /dev/sdh
#
#
# su - oragrid
$ asmcmd
ASMCMD> lsdg
State Type RebalSectorBlock AUTotal_MBFree_MBReq_mir_free_MBUsable_file_MBOffline_disksVoting_filesName
MOUNTEDEXTERNN 512 40961048576 8544 8135 0 8135 0 NARCH/
MOUNTEDEXTERNN 512 40961048576 11296 8543 0 8543 0 NDATA/
MOUNTEDEXTERNN 512 40961048576 5728 5332 0 5332 0 YOCRVOTE/
ASMCMD> lsdsk
Path
/dev/mapper/arch
/dev/mapper/data
/dev/mapper/vote_ocr
ASMCMD>
ASMCMD>
ASMCMD>
ASMCMD>
ASMCMD> exit
$ more /etc/multipath.conf
defaults {
user_friendly_names yes
}
multipaths {
multipath {
wwid 14f504e46494c45004932325a59732d757266762d4f486c59
alias vote_ocr
}
multipath {
wwid 14f504e46494c45007469355367682d513879662d6d344836
alias data
}
multipath {
wwid 14f504e46494c4500764e577843332d547243472d37417072
alias arch
}
}
$
$
$
$
$ cd $ORACLE_HOME/dbs
$ ls
ab_+ASM1.dathc_+ASM1.datinit.oraorapw+ASM
$ ls -l
?????16
-rw-rw---- 1 oragrid oinstall 1053 10?? 5 18:27 ab_+ASM1.dat
-rw-rw---- 1 oragrid oinstall 1544 10?? 5 18:27 hc_+ASM1.dat
-rw-r--r-- 1 oragrid oinstall 2851 5?? 15 2009 init.ora
-rw-r----- 1 oragrid oinstall 1536 7?? 17 00:17 orapw+ASM
$
$
$
$
$
$
$ ls -l
?????16
-rw-rw---- 1 oragrid oinstall 1053 10?? 5 18:27 ab_+ASM1.dat
-rw-rw---- 1 oragrid oinstall 1544 10?? 5 18:27 hc_+ASM1.dat
-rw-r--r-- 1 oragrid oinstall 2851 5?? 15 2009 init.ora
-rw-r----- 1 oragrid oinstall 1536 7?? 17 00:17 orapw+ASM
$ ls -l
??? 16
-rw-rw---- 1 oragrid oinstall 1053 10?5 18:27 ab_+ASM1.dat
-rw-rw---- 1 oragrid oinstall 1544 10?5 18:27 hc_+ASM1.dat
-rw-r--r-- 1 oragrid oinstall 2851 5?15 2009 init.ora
-rw-r----- 1 oragrid oinstall 1536 7?17 00:17 orapw+ASM
$
$
$
$ cat /etc/multipath.conf
defaults {
user_friendly_names yes
}
multipaths {
multipath {
wwid 14f504e46494c45004932325a59732d757266762d4f486c59
alias vote_ocr
}
multipath {
wwid 14f504e46494c45007469355367682d513879662d6d344836
alias data
}
multipath {
wwid 14f504e46494c4500764e577843332d547243472d37417072
alias arch
}
}
$ cat /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c45004932325a59732d757266762d4f486c59",OWNER="oragrid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c45007469355367682d513879662d6d344836",OWNER="oragrid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c4500764e577843332d547243472d37417072",OWNER="oragrid",GROUP="asmadmin",MODE="0660"
$
$
$
$
$
$ chkconfig multipathd off
?????????????
$ exit
logout
# chkconfig multipathd off
# ssh node2
root@node2's password:
Last login: Sat Jul 16 23:47:59 2016 from 192.168.1.1
# chkconfig multipathd off
# service multipathd stop
????multipathd ??????: [??]
# exit
logout
Connection to node2 closed.
# service multipathd stop
????multipathd ??????: [??]
#
#
#
#
# cd /etc/udev/rules.d/
# mv 99-oracle-asmdevices.rules 99-oracle-asmdevices.rules.bak
#
#
#
# ssh node2
root@node2's password:
Last login: Wed Oct5 18:36:50 2016 from node1
# cd /etc/udev/rules.d/
# mv 99-oracle-asmdevices.rules 99-oracle-asmdevices.rules.bak
# exit
logout
Connection to node2 closed.
#
#
# pwd
/etc/udev/rules.d
# vi 99-oracle-asmdevices.rules
# fdisk -l | grep sd
Disk /dev/sda: 53.7 GB, 53687091200 bytes
/dev/sda1 * 1 256 2048000 83Linux
/dev/sda2 256 1275 8192000 82Linux swap / Solaris
/dev/sda3 1275 6528 42187776 8eLinux LVM
Disk /dev/sdb: 21.5 GB, 21474836480 bytes
/dev/sdb1 1 2610 20964793+8eLinux LVM
Disk /dev/sdc: 6006 MB, 6006243328 bytes
Disk /dev/sdd: 6006 MB, 6006243328 bytes
Disk /dev/sde: 11.8 GB, 11844714496 bytes
Disk /dev/sdf: 11.8 GB, 11844714496 bytes
Disk /dev/sdg: 8959 MB, 8959033344 bytes
Disk /dev/sdh: 8959 MB, 8959033344 bytes
# for i in c d e f g h ;
> do
> echo "KERNEL==\"sd*\", BUS==\"scsi\", PROGRAM==\"/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/\$name\", RESULT==\"`/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$i`\", NAME=\"asm-disk$i\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\""
> done
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c45004932325a59732d757266762d4f486c59", NAME="asm-diskc", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c45004932325a59732d757266762d4f486c59", NAME="asm-diskd", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c45007469355367682d513879662d6d344836", NAME="asm-diske", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c45007469355367682d513879662d6d344836", NAME="asm-diskf", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c4500764e577843332d547243472d37417072", NAME="asm-diskg", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c4500764e577843332d547243472d37417072", NAME="asm-diskh", OWNER="grid", GROUP="asmadmin", MODE="0660"
#
#
# fdisk -l | grep sd
# vi 99-oracle-asmdevices.rules
# cat99-oracle-asmdevices.rules
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c45004932325a59732d757266762d4f486c59", NAME="asm-diskc", OWNER="oragrid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c45007469355367682d513879662d6d344836", NAME="asm-diske", OWNER="oragrid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c4500764e577843332d547243472d37417072", NAME="asm-diskg", OWNER="oragrid", GROUP="asmadmin", MODE="0660"
#
#
#
#
# start_udev
???? udev: [??]
# ls -l /dev/asm-*
brw-rw---- 1 oragrid asmadmin 8,48 10?5 18:44 /dev/asm-diskc
brw-rw---- 1 oragrid asmadmin 8,64 10?5 18:44 /dev/asm-diske
brw-rw---- 1 oragrid asmadmin 8, 112 10?5 18:44 /dev/asm-diskg
# scp 99-oracle-asmdevices.rules node2:/etc/udev/rules.d/
root@node2's password:
99-oracle-asmdevices.rules 100%705 0.7KB/s 00:00
# ssh node2
root@node2's password:
Last login: Wed Oct5 18:38:07 2016 from node1
# start_udev
???? udev: [??]
# ls -l /dev/asm-*
brw-rw---- 1 oragrid asmadmin 8,48 10?5 18:44 /dev/asm-diskc
brw-rw---- 1 oragrid asmadmin 8,80 10?5 18:44 /dev/asm-diske
brw-rw---- 1 oragrid asmadmin 8, 112 10?5 18:44 /dev/asm-diskg
#
#
#
#
#
# su - oragrid
$ cd $ORACLE_HOME/log
$ ls
crsdiagnode2
$ cd node2
$ ls
acfsacfslogacfsreplacfsreplrootacfssecadminagentalertnode2.logclientcrflogdcrfmondcrsdcssdctssdcvudiskmonevmdgipcdgnsdgpnpdmdnsdohasdracgsrvm
$ tail -300 alertnode2.log
2016-07-17 00:11:12.818:
CRS-2101:The OLR was formatted using version 3.
2016-07-17 00:11:45.527:
CRS-2112:The OLR service started on node node2.
2016-07-17 00:11:45.536:
CRS-1301:Oracle High Availability Service started on node node2.
CRS-10001:17-Jul-16 00:11 ACFS-9200: Supported
CRS-10001:17-Jul-16 00:12 ACFS-9300: ADVM/ACFS distribution files found.
CRS-10001:17-Jul-16 00:12 ACFS-9307: Installing requested ADVM/ACFS software.
CRS-10001:17-Jul-16 00:12 ACFS-9308: Loading installed ADVM/ACFS drivers.
CRS-10001:17-Jul-16 00:12 ACFS-9321: Creating udev for ADVM/ACFS.
CRS-10001:17-Jul-16 00:12 ACFS-9323: Creating module dependencies - this may take some time.
CRS-10001:17-Jul-16 00:13 ACFS-9154: Loading 'oracleoks.ko' driver.
CRS-10001:17-Jul-16 00:13 ACFS-9154: Loading 'oracleadvm.ko' driver.
CRS-10001:17-Jul-16 00:13 ACFS-9154: Loading 'oracleacfs.ko' driver.
CRS-10001:17-Jul-16 00:13 ACFS-9327: Verifying ADVM/ACFS devices.
CRS-10001:17-Jul-16 00:13 ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
CRS-10001:17-Jul-16 00:13 ACFS-9156: Detecting control device '/dev/ofsctl'.
CRS-10001:17-Jul-16 00:13 ACFS-9309: ADVM/ACFS installation correctness verified.
2016-07-17 00:13:24.628:
CRS-2328:GPNPD started on node node2.
2016-07-17 00:13:26.868:
CRS-1713:CSSD daemon is started in exclusive mode
2016-07-17 00:13:28.745:
CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2016-07-17 00:13:28.746:
CRS-2769:Unable to failover resource 'ora.diskmon'.
2016-07-17 00:14:03.533:
CRS-1707:Lease acquisition for node node2 number 2 completed
CRS-1636:The CSS daemon was started in exclusive mode but found an active CSS daemon on node node1 and is terminating; details at (:CSSNM00006:) in /grid/grid_home/log/node2/cssd/ocssd.log
2016-07-17 00:14:03.674:
CRS-2765:Resource 'ora.cssdmonitor' has failed on server 'node2'.
2016-07-17 00:14:05.403:
CRS-2329:GPNPD on node node2 shutdown.
2016-07-17 00:14:06.763:
CRS-5602:mDNS service stopping by request.
2016-07-17 00:14:22.904:
CRS-2328:GPNPD started on node node2.
2016-07-17 00:14:25.512:
CRS-1713:CSSD daemon is started in clustered mode
2016-07-17 00:14:27.378:
CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2016-07-17 00:14:27.379:
CRS-2769:Unable to failover resource 'ora.diskmon'.
2016-07-17 00:14:37.096:
CRS-1707:Lease acquisition for node node2 number 2 completed
2016-07-17 00:14:38.356:
CRS-1605:CSSD voting file is online: /dev/mapper/vote_ocr; details in /grid/grid_home/log/node2/cssd/ocssd.log.
2016-07-17 00:14:40.537:
CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 node2 .
2016-07-17 00:14:42.952:
CRS-2407:The new Cluster Time Synchronization Service reference node is host node1.
2016-07-17 00:14:42.951:
CRS-2401:The Cluster Time Synchronization Service started on host node2.
2016-07-17 00:14:48.211:
CRS-2408:The clock on host node2 has been updated by the Cluster Time Synchronization Service to be synchronous with the mean cluster time.
CRS-10001:17-Jul-16 00:14 ACFS-9391: Checking for existing ADVM/ACFS installation.
CRS-10001:17-Jul-16 00:14 ACFS-9392: Validating ADVM/ACFS installation files for operating system.
CRS-10001:17-Jul-16 00:14 ACFS-9393: Verifying ASM Administrator setup.
CRS-10001:17-Jul-16 00:14 ACFS-9308: Loading installed ADVM/ACFS drivers.
CRS-10001:17-Jul-16 00:14 ACFS-9327: Verifying ADVM/ACFS devices.
CRS-10001:17-Jul-16 00:14 ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
CRS-10001:17-Jul-16 00:14 ACFS-9156: Detecting control device '/dev/ofsctl'.
CRS-10001:17-Jul-16 00:14 ACFS-9322: completed
2016-07-17 00:15:00.705:
CRS-5018:(:CLSN00037:) Removed unused HAIP route:169.254.0.0 / 255.255.0.0 / 0.0.0.0 / bond0
2016-07-17 00:15:15.805:
CRS-1012:The OCR service started on node node2.
2016-07-17 00:15:15.815:
CRS-1401:EVMD started on node node2.
2016-07-17 00:15:17.201:
CRS-1201:CRSD started on node node2.
2016-07-17 00:58:43.472:
CRS-2408:The clock on host node2 has been updated by the Cluster Time Synchronization Service to be synchronous with the mean cluster time.
2016-07-17 01:34:43.983:
CRS-2408:The clock on host node2 has been updated by the Cluster Time Synchronization Service to be synchronous with the mean cluster time.
2016-07-17 01:35:53.526:
CRS-5822:Agent '/grid/grid_home/bin/orarootagent_root' disconnected from server. Details at (:CRSAGF00117:) {0:3:125} in /grid/grid_home/log/node2/agent/crsd/orarootagent_root/orarootagent_root.log.
2016-07-17 01:35:54.444:
CRS-2405:The Cluster Time Synchronization Service on host node2 is shutdown by user
2016-07-17 01:36:04.751:
CRS-1603:CSSD on node node2 shutdown by user.
2016-07-17 01:36:04.857:
CRS-2767:Resource state recovery not attempted for 'ora.cssdmonitor' as its target state is OFFLINE
2016-07-17 01:36:04.857:
CRS-2769:Unable to failover resource 'ora.cssdmonitor'.
2016-07-17 01:36:04.954:
CRS-1660:The CSS daemon shutdown has completed
2016-07-17 01:36:06.154:
CRS-2767:Resource state recovery not attempted for 'ora.cssd' as its target state is OFFLINE
2016-07-17 01:36:06.154:
CRS-2769:Unable to failover resource 'ora.cssd'.
2016-07-17 01:37:08.545:
CRS-5602:mDNS service stopping by request.
2016-07-17 01:37:11.266:
CRS-2329:GPNPD on node node2 shutdown.
2016-10-05 18:26:23.110:
CRS-2112:The OLR service started on node node2.
2016-10-05 18:26:23.147:
CRS-1301:Oracle High Availability Service started on node node2.
2016-10-05 18:26:23.170:
CRS-8017:location: /etc/oracle/lastgasp has 2 reboot advisory log files, 0 were announced and 0 errors occurred
2016-10-05 18:26:27.485:
CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2016-10-05 18:26:31.893:
CRS-2328:GPNPD started on node node2.
2016-10-05 18:26:35.350:
CRS-1713:CSSD daemon is started in clustered mode
2016-10-05 18:26:36.967:
CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2016-10-05 18:26:36.968:
CRS-2769:Unable to failover resource 'ora.diskmon'.
2016-10-05 18:26:45.353:
CRS-1707:Lease acquisition for node node2 number 2 completed
2016-10-05 18:26:46.618:
CRS-1605:CSSD voting file is online: /dev/mapper/vote_ocr; details in /grid/grid_home/log/node2/cssd/ocssd.log.
2016-10-05 18:26:56.042:
CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 node2 .
2016-10-05 18:26:58.581:
CRS-2401:The Cluster Time Synchronization Service started on host node2.
2016-10-05 18:26:58.581:
CRS-2407:The new Cluster Time Synchronization Service reference node is host node1.
2016-10-05 18:27:03.284:
CRS-2408:The clock on host node2 has been updated by the Cluster Time Synchronization Service to be synchronous with the mean cluster time.
CRS-10001:05-Oct-16 18:27 ACFS-9391: Checking for existing ADVM/ACFS installation.
CRS-10001:05-Oct-16 18:27 ACFS-9392: Validating ADVM/ACFS installation files for operating system.
CRS-10001:05-Oct-16 18:27 ACFS-9393: Verifying ASM Administrator setup.
CRS-10001:05-Oct-16 18:27 ACFS-9308: Loading installed ADVM/ACFS drivers.
CRS-10001:05-Oct-16 18:27 ACFS-9154: Loading 'oracleoks.ko' driver.
CRS-10001:05-Oct-16 18:27 ACFS-9154: Loading 'oracleadvm.ko' driver.
CRS-10001:05-Oct-16 18:27 ACFS-9154: Loading 'oracleacfs.ko' driver.
CRS-10001:05-Oct-16 18:27 ACFS-9327: Verifying ADVM/ACFS devices.
CRS-10001:05-Oct-16 18:27 ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
CRS-10001:05-Oct-16 18:27 ACFS-9156: Detecting control device '/dev/ofsctl'.
CRS-10001:05-Oct-16 18:27 ACFS-9322: completed
2016-10-05 18:27:36.366:
CRS-1012:The OCR service started on node node2.
2016-10-05 18:27:36.428:
CRS-1401:EVMD started on node node2.
2016-10-05 18:27:38.397:
CRS-1201:CRSD started on node node2.
$ exit
logout
#
#
# exit
logout
Connection to node2 closed.
# su - oragrid
$ cd $ORACLE_HOME/log
$ ls
crsdiagnode1
$ cd node1
$ ls
acfsacfslogacfsreplacfsreplrootacfssecadminagentalertnode1.logclientcrflogdcrfmondcrsdcssdctssdcvudiskmonevmdgipcdgnsdgpnpdmdnsdohasdracgsrvm
$ tail -f alertnode1.log
2016-10-05 18:27:44.520:
CRS-2772:Server 'node1' has been assigned to pool 'Generic'.
2016-10-05 18:27:44.531:
CRS-2772:Server 'node1' has been assigned to pool 'ora.orcc'.
2016-10-05 18:27:45.125:
CRS-4743:File /grid/grid_home/oc4j/j2ee/home/OC4J_DBWLM_config/system-jazn-data.xml was updated from OCR(Size: 13365(New), 13378(Old) bytes)
2016-10-05 18:28:01.879:
CRS-2772:Server 'node2' has been assigned to pool 'Generic'.
2016-10-05 18:28:01.880:
CRS-2772:Server 'node2' has been assigned to pool 'ora.orcc'.
^C
$ tail -200f alertnode1.log
CRS-2328:GPNPD started on node node1.
2016-07-17 00:05:07.025:
CRS-1713:CSSD daemon is started in exclusive mode
2016-07-17 00:05:08.917:
CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2016-07-17 00:05:08.917:
CRS-2769:Unable to failover resource 'ora.diskmon'.
2016-07-17 00:05:13.559:
CRS-1709:Lease acquisition failed for node node1 because no voting file has been configured; Details at (:CSSNM00031:) in /grid/grid_home/log/node1/cssd/ocssd.log
2016-07-17 00:05:22.325:
CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 .
2016-07-17 00:05:24.846:
CRS-2401:The Cluster Time Synchronization Service started on host node1.
2016-07-17 00:05:24.846:
CRS-2407:The new Cluster Time Synchronization Service reference node is host node1.
CRS-10001:17-Jul-16 00:05 ACFS-9203: true
CRS-10001:17-Jul-16 00:05 ACFS-9391: Checking for existing ADVM/ACFS installation.
CRS-10001:17-Jul-16 00:05 ACFS-9392: Validating ADVM/ACFS installation files for operating system.
CRS-10001:17-Jul-16 00:05 ACFS-9393: Verifying ASM Administrator setup.
CRS-10001:17-Jul-16 00:05 ACFS-9308: Loading installed ADVM/ACFS drivers.
CRS-10001:17-Jul-16 00:05 ACFS-9327: Verifying ADVM/ACFS devices.
CRS-10001:17-Jul-16 00:05 ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
CRS-10001:17-Jul-16 00:05 ACFS-9156: Detecting control device '/dev/ofsctl'.
CRS-10001:17-Jul-16 00:05 ACFS-9322: completed
2016-07-17 00:05:46.497:
CRS-5018:(:CLSN00037:) Removed unused HAIP route:169.254.0.0 / 255.255.0.0 / 0.0.0.0 / bond0
2016-07-17 00:05:50.451:
CRS-1001:The OCR was formatted using version 3.
2016-07-17 00:05:52.046:
CRS-1012:The OCR service started on node node1.
2016-07-17 00:05:53.777:
CRS-1605:CSSD voting file is online: /dev/mapper/vote_ocr; details in /grid/grid_home/log/node1/cssd/ocssd.log.
2016-07-17 00:05:53.777:
CRS-1626:A Configuration change request completed successfully
2016-07-17 00:05:53.782:
CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 .
2016-07-17 00:06:06.810:
CRS-2405:The Cluster Time Synchronization Service on host node1 is shutdown by user
2016-07-17 00:06:18.913:
CRS-1603:CSSD on node node1 shutdown by user.
2016-07-17 00:06:19.119:
CRS-1660:The CSS daemon shutdown has completed
2016-07-17 00:06:19.394:
CRS-2767:Resource state recovery not attempted for 'ora.cssdmonitor' as its target state is OFFLINE
2016-07-17 00:06:19.394:
CRS-2769:Unable to failover resource 'ora.cssdmonitor'.
2016-07-17 00:06:20.317:
CRS-2767:Resource state recovery not attempted for 'ora.cssd' as its target state is OFFLINE
2016-07-17 00:06:20.317:
CRS-2769:Unable to failover resource 'ora.cssd'.
2016-07-17 00:06:21.723:
CRS-2329:GPNPD on node node1 shutdown.
2016-07-17 00:06:23.220:
CRS-5602:mDNS service stopping by request.
2016-07-17 00:06:31.290:
CRS-2328:GPNPD started on node node1.
2016-07-17 00:06:33.901:
CRS-1713:CSSD daemon is started in clustered mode
2016-07-17 00:06:35.760:
CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2016-07-17 00:06:35.760:
CRS-2769:Unable to failover resource 'ora.diskmon'.
2016-07-17 00:07:06.673:
CRS-1707:Lease acquisition for node node1 number 1 completed
2016-07-17 00:07:07.934:
CRS-1605:CSSD voting file is online: /dev/mapper/vote_ocr; details in /grid/grid_home/log/node1/cssd/ocssd.log.
2016-07-17 00:07:17.060:
CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 .
2016-07-17 00:07:19.336:
CRS-2407:The new Cluster Time Synchronization Service reference node is host node1.
2016-07-17 00:07:19.335:
CRS-2401:The Cluster Time Synchronization Service started on host node1.
2016-07-17 00:07:41.530:
CRS-1012:The OCR service started on node node1.
2016-07-17 00:07:42.028:
CRS-1401:EVMD started on node node1.
2016-07-17 00:07:43.106:
CRS-1201:CRSD started on node node1.
2016-07-17 00:07:43.454:
CRS-2772:Server 'node1' has been assigned to pool 'Free'.
CRS-10001:17-Jul-16 00:08 ACFS-9200: Supported
2016-07-17 00:08:26.456:
CRS-4742:OCR updated with contents of /grid/grid_home/oc4j/j2ee/home/OC4J_DBWLM_config/system-jazn-data.xml (New = 13378, Old = 0 bytes)
2016-07-17 00:08:26.545:
CRS-4743:File /grid/grid_home/oc4j/j2ee/home/OC4J_DBWLM_config/system-jazn-data.xml was updated from OCR(Size: 13377(New), 13378(Old) bytes)
2016-07-17 00:08:26.584:
CRS-4743:File /grid/grid_home/oc4j/j2ee/home/OC4J_DBWLM_config/system-jazn-data.xml was updated from OCR(Size: 13377(New), 13378(Old) bytes)
2016-07-17 00:08:26.652:
CRS-4742:OCR updated with contents of /grid/grid_home/oc4j/j2ee/home/OC4J_DBWLM_config/system-jazn-data.xml (New = 13378, Old = 13378 bytes)
2016-07-17 00:08:37.884:
CRS-4743:File /grid/grid_home/oc4j/j2ee/home/OC4J_DBWLM_config/system-jazn-data.xml was updated from OCR(Size: 13377(New), 13378(Old) bytes)
CRS-10001:17-Jul-16 00:08 ACFS-9200: Supported
2016-07-17 00:14:40.763:
CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 node2 .
2016-07-17 00:15:20.258:
CRS-2772:Server 'node2' has been assigned to pool 'Free'.
CRS-10001:17-Jul-16 00:17 ACFS-9203: true
CRS-10001:17-Jul-16 00:17 ACFS-9200: Supported
CRS-10001:17-Jul-16 00:17 ACFS-9200: Supported
CRS-10001:17-Jul-16 00:46 ACFS-9203: true
CRS-10001:17-Jul-16 00:46 ACFS-9203: true
2016-07-17 00:54:02.914:
CRS-2773:Server 'node2' has been removed from pool 'Free'.
2016-07-17 00:54:02.914:
CRS-2772:Server 'node2' has been assigned to pool 'Generic'.
2016-07-17 00:54:02.915:
CRS-2773:Server 'node1' has been removed from pool 'Free'.
2016-07-17 00:54:02.915:
CRS-2772:Server 'node1' has been assigned to pool 'Generic'.
2016-07-17 00:54:02.915:
CRS-2772:Server 'node1' has been assigned to pool 'ora.orcc'.
2016-07-17 00:54:02.916:
CRS-2772:Server 'node2' has been assigned to pool 'ora.orcc'.
2016-07-17 01:35:55.522:
CRS-2405:The Cluster Time Synchronization Service on host node1 is shutdown by user
2016-07-17 01:35:55.531:
CRS-5822:Agent '/grid/grid_home/bin/orarootagent_root' disconnected from server. Details at (:CRSAGF00117:) {0:3:32} in /grid/grid_home/log/node1/agent/crsd/orarootagent_root/orarootagent_root.log.
2016-07-17 01:36:04.852:
CRS-1625:Node node2, number 2, was manually shut down
2016-07-17 01:36:04.854:
CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 .
2016-07-17 01:36:06.742:
CRS-1603:CSSD on node node1 shutdown by user.
2016-07-17 01:36:06.847:
CRS-2767:Resource state recovery not attempted for 'ora.cssdmonitor' as its target state is OFFLINE
2016-07-17 01:36:06.847:
CRS-2769:Unable to failover resource 'ora.cssdmonitor'.
2016-07-17 01:36:06.945:
CRS-1660:The CSS daemon shutdown has completed
2016-07-17 01:36:08.135:
CRS-2767:Resource state recovery not attempted for 'ora.cssd' as its target state is OFFLINE
2016-07-17 01:36:08.135:
CRS-2769:Unable to failover resource 'ora.cssd'.
2016-07-17 01:36:10.675:
CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/grid_home/log/node1/client/crsctl_orardbms.log.
2016-07-17 01:36:20.399:
CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/grid_home/log/node1/client/crsctl_orardbms.log.
2016-07-17 01:36:30.495:
CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/grid_home/log/node1/client/crsctl_orardbms.log.
2016-07-17 01:36:40.782:
CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/grid_home/log/node1/client/crsctl_orardbms.log.
2016-07-17 01:36:50.208:
CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/grid_home/log/node1/client/crsctl_orardbms.log.
2016-07-17 01:37:03.264:
CRS-5602:mDNS service stopping by request.
2016-07-17 01:37:06.085:
CRS-2329:GPNPD on node node1 shutdown.
2016-10-05 18:26:19.904:
CRS-2112:The OLR service started on node node1.
2016-10-05 18:26:20.192:
CRS-1301:Oracle High Availability Service started on node node1.
2016-10-05 18:26:20.295:
CRS-8017:location: /etc/oracle/lastgasp has 2 reboot advisory log files, 0 were announced and 0 errors occurred
2016-10-05 18:26:28.827:
CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2016-10-05 18:26:33.095:
CRS-2328:GPNPD started on node node1.
2016-10-05 18:26:35.882:
CRS-1713:CSSD daemon is started in clustered mode
2016-10-05 18:26:37.461:
CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2016-10-05 18:26:37.494:
CRS-2769:Unable to failover resource 'ora.diskmon'.
2016-10-05 18:26:45.859:
CRS-1707:Lease acquisition for node node1 number 1 completed
2016-10-05 18:26:47.144:
CRS-1605:CSSD voting file is online: /dev/mapper/vote_ocr; details in /grid/grid_home/log/node1/cssd/ocssd.log.
2016-10-05 18:26:55.712:
CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 node2 .
2016-10-05 18:26:58.047:
CRS-2407:The new Cluster Time Synchronization Service reference node is host node1.
2016-10-05 18:26:58.047:
CRS-2401:The Cluster Time Synchronization Service started on host node1.
CRS-10001:05-Oct-16 18:27 ACFS-9391: Checking for existing ADVM/ACFS installation.
CRS-10001:05-Oct-16 18:27 ACFS-9392: Validating ADVM/ACFS installation files for operating system.
CRS-10001:05-Oct-16 18:27 ACFS-9393: Verifying ASM Administrator setup.
CRS-10001:05-Oct-16 18:27 ACFS-9308: Loading installed ADVM/ACFS drivers.
CRS-10001:05-Oct-16 18:27 ACFS-9154: Loading 'oracleoks.ko' driver.
CRS-10001:05-Oct-16 18:27 ACFS-9154: Loading 'oracleadvm.ko' driver.
CRS-10001:05-Oct-16 18:27 ACFS-9154: Loading 'oracleacfs.ko' driver.
CRS-10001:05-Oct-16 18:27 ACFS-9327: Verifying ADVM/ACFS devices.
CRS-10001:05-Oct-16 18:27 ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
CRS-10001:05-Oct-16 18:27 ACFS-9156: Detecting control device '/dev/ofsctl'.
CRS-10001:05-Oct-16 18:27 ACFS-9322: completed
2016-10-05 18:27:31.947:
CRS-1012:The OCR service started on node node1.
2016-10-05 18:27:31.978:
CRS-1401:EVMD started on node node1.
2016-10-05 18:27:36.399:
CRS-1201:CRSD started on node node1.
2016-10-05 18:27:44.520:
CRS-2772:Server 'node1' has been assigned to pool 'Generic'.
2016-10-05 18:27:44.531:
CRS-2772:Server 'node1' has been assigned to pool 'ora.orcc'.
2016-10-05 18:27:45.125:
CRS-4743:File /grid/grid_home/oc4j/j2ee/home/OC4J_DBWLM_config/system-jazn-data.xml was updated from OCR(Size: 13365(New), 13378(Old) bytes)
2016-10-05 18:28:01.879:
CRS-2772:Server 'node2' has been assigned to pool 'Generic'.
2016-10-05 18:28:01.880:
CRS-2772:Server 'node2' has been assigned to pool 'ora.orcc'.
^C
$ ls -ltr
??? 108
drwxr-x--- 2 oragrid oinstall4096 7?17 00:00 srvm
drwxr-x--- 2 root oinstall4096 7?17 00:00 gnsd
drwxr-x--- 2 oragrid oinstall4096 7?17 00:00 diskmon
drwxr-x--- 4 oragrid oinstall4096 7?17 00:00 cvu
drwxr-xr-x 2 root oinstall4096 7?17 00:00 acfssec
drwxr-x--- 2 oragrid oinstall4096 7?17 00:00 acfsrepl
drwxr-x--- 2 oragrid oinstall4096 7?17 00:00 acfslog
drwxrwxr-t 4 root oinstall4096 7?17 00:00 agent
drwxrwxr-t 5 oragrid oinstall4096 7?17 00:00 racg
drwxr-x--- 2 oragrid oinstall4096 7?17 00:00 admin
drwxr-x--- 2 root oinstall4096 7?17 00:00 acfsreplroot
drwxr-x--- 2 root oinstall4096 7?17 00:03 ohasd
drwxr-xr-x 3 root root 4096 7?17 00:05 acfs
drwxr-x--- 2 oragrid oinstall4096 7?17 00:05 mdnsd
drwxr-x--- 2 oragrid oinstall4096 7?17 00:05 gipcd
drwxr-x--- 2 oragrid oinstall4096 7?17 00:05 cssd
drwxr-x--- 2 root oinstall4096 7?17 00:05 ctssd
drwxr-x--- 2 root oinstall4096 7?17 00:05 crsd
drwxr-x--- 2 root oinstall4096 7?17 00:07 crfmond
drwxr-x--- 2 root oinstall4096 7?17 00:07 crflogd
drwxr-x--- 2 oragrid oinstall4096 7?17 00:07 evmd
drwxr-x--- 2 oragrid oinstall4096 10?5 18:26 gpnpd
-rw-rw-r-- 1 oragrid oinstall 13847 10?5 18:28 alertnode1.log
drwxrwxrwt 2 oragrid oinstall4096 10?5 18:32 client
$ cd cssd
$ ls
cssdOUT.logocssd.log
$ tail -100f ocssd.log
2016-10-05 18:45:49.707: [ CSSD]clssnmSendingThread: sending status msg to all nodes
2016-10-05 18:45:49.707: [ CSSD]clssnmSendingThread: sent 5 status msgs to all nodes
2016-10-05 18:45:53.710: [ CSSD]clssnmSendingThread: sending status msg to all nodes
2016-10-05 18:45:53.710: [ CSSD]clssnmSendingThread: sent 4 status msgs to all nodes
2016-10-05 18:45:58.714: [ CSSD]clssnmSendingThread: sending status msg to all nodes
2016-10-05 18:45:58.714: [ CSSD]clssnmSendingThread: sent 5 status msgs to all nodes
2016-10-05 18:46:03.721: [ CSSD]clssnmSendingThread: sending status msg to all nodes
2016-10-05 18:46:03.721: [ CSSD]clssnmSendingThread: sent 5 status msgs to all nodes
2016-10-05 18:46:07.724: [ CSSD]clssnmSendingThread: sending status msg to all nodes
2016-10-05 18:46:07.724: [ CSSD]clssnmSendingThread: sent 4 status msgs to all nodes
2016-10-05 18:46:12.727: [ CSSD]clssnmSendingThread: sending status msg to all nodes
2016-10-05 18:46:12.727: [ CSSD]clssnmSendingThread: sent 5 status msgs to all nodes
2016-10-05 18:46:17.731: [ CSSD]clssnmSendingThread: sending status msg to all nodes
2016-10-05 18:46:17.731: [ CSSD]clssnmSendingThread: sent 5 status msgs to all nodes
^C
$
$
$
$
$ su
??:
# cd $ORACLE_HOME/bin
# ./crsctl check cluster -all
**************************************************************
node1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
node2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
# ./crsctl stop cluster -all
CRS-2673: Attempting to stop 'ora.crsd' on 'node2'
CRS-2673: Attempting to stop 'ora.crsd' on 'node1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'node1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'node1'
CRS-2673: Attempting to stop 'ora.cvu' on 'node1'
CRS-2673: Attempting to stop 'ora.OCRVOTE.dg' on 'node1'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'node1'
CRS-2673: Attempting to stop 'ora.orcc.db' on 'node1'
CRS-2673: Attempting to stop 'ora.oc4j' on 'node1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'node2'
CRS-2673: Attempting to stop 'ora.OCRVOTE.dg' on 'node2'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'node2'
CRS-2673: Attempting to stop 'ora.orcc.db' on 'node2'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'node2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'node2'
CRS-2677: Stop of 'ora.cvu' on 'node1' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.node1.vip' on 'node1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.node2.vip' on 'node2'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'node2'
CRS-2677: Stop of 'ora.registry.acfs' on 'node1' succeeded
CRS-2677: Stop of 'ora.registry.acfs' on 'node2' succeeded
CRS-2677: Stop of 'ora.orcc.db' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.ARCH.dg' on 'node1'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'node1'
CRS-2677: Stop of 'ora.node1.vip' on 'node1' succeeded
CRS-2677: Stop of 'ora.ARCH.dg' on 'node1' succeeded
CRS-2677: Stop of 'ora.node2.vip' on 'node2' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'node1' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'node2' succeeded
CRS-2677: Stop of 'ora.orcc.db' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.ARCH.dg' on 'node2'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'node2'
CRS-2677: Stop of 'ora.DATA.dg' on 'node2' succeeded
CRS-2677: Stop of 'ora.ARCH.dg' on 'node2' succeeded
CRS-2677: Stop of 'ora.oc4j' on 'node1' succeeded
CRS-2677: Stop of 'ora.OCRVOTE.dg' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'node1'
CRS-2677: Stop of 'ora.OCRVOTE.dg' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'node2'
CRS-2677: Stop of 'ora.asm' on 'node1' succeeded
CRS-2677: Stop of 'ora.asm' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'node2'
CRS-2677: Stop of 'ora.ons' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'node2'
CRS-2673: Attempting to stop 'ora.ons' on 'node1'
CRS-2677: Stop of 'ora.net1.network' on 'node2' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'node2' has completed
CRS-2677: Stop of 'ora.crsd' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'node2'
CRS-2673: Attempting to stop 'ora.evmd' on 'node2'
CRS-2673: Attempting to stop 'ora.asm' on 'node2'
CRS-2677: Stop of 'ora.ons' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'node1'
CRS-2677: Stop of 'ora.net1.network' on 'node1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'node1' has completed
CRS-2677: Stop of 'ora.crsd' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'node1'
CRS-2673: Attempting to stop 'ora.evmd' on 'node1'
CRS-2673: Attempting to stop 'ora.asm' on 'node1'
CRS-2677: Stop of 'ora.evmd' on 'node2' succeeded
CRS-2677: Stop of 'ora.evmd' on 'node1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'node2' succeeded
CRS-2677: Stop of 'ora.asm' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'node2'
CRS-2677: Stop of 'ora.ctssd' on 'node1' succeeded
CRS-2677: Stop of 'ora.asm' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'node1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'node2'
CRS-2677: Stop of 'ora.cssd' on 'node2' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'node1'
CRS-2677: Stop of 'ora.cssd' on 'node1' succeeded
# ps -ef | grep d.bin
root 2704 10 18:26 ? 00:00:11 /grid/grid_home/bin/ohasd.bin reboot
oragrid 3030 10 18:26 ? 00:00:00 /grid/grid_home/bin/mdnsd.bin
oragrid 3040 10 18:26 ? 00:00:01 /grid/grid_home/bin/gpnpd.bin
oragrid 3053 10 18:26 ? 00:00:07 /grid/grid_home/bin/gipcd.bin
root 3065 12 18:26 ? 00:00:35 /grid/grid_home/bin/osysmond.bin
root 7390 69310 18:47 pts/0 00:00:00 grep d.bin
#
#
#
#
# crsctl start cluster -all
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'node1'
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'node2'
CRS-2676: Start of 'ora.cssdmonitor' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'node2'
CRS-2676: Start of 'ora.cssdmonitor' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.diskmon' on 'node2'
CRS-2672: Attempting to start 'ora.cssd' on 'node1'
CRS-2672: Attempting to start 'ora.diskmon' on 'node1'
CRS-2676: Start of 'ora.diskmon' on 'node1' succeeded
CRS-2676: Start of 'ora.diskmon' on 'node2' succeeded
^C
# cd $ORACLE_HOME/log/node1
# ls
acfsacfslogacfsreplacfsreplrootacfssecadminagentalertnode1.logclientcrflogdcrfmondcrsdcssdctssdcvudiskmonevmdgipcdgnsdgpnpdmdnsdohasdracgsrvm
# tail -100f alertnode1.log
CRS-1660:The CSS daemon shutdown has completed
2016-07-17 01:36:08.135:
CRS-2767:Resource state recovery not attempted for 'ora.cssd' as its target state is OFFLINE
2016-07-17 01:36:08.135:
CRS-2769:Unable to failover resource 'ora.cssd'.
2016-07-17 01:36:10.675:
CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/grid_home/log/node1/client/crsctl_orardbms.log.
2016-07-17 01:36:20.399:
CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/grid_home/log/node1/client/crsctl_orardbms.log.
2016-07-17 01:36:30.495:
CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/grid_home/log/node1/client/crsctl_orardbms.log.
2016-07-17 01:36:40.782:
CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/grid_home/log/node1/client/crsctl_orardbms.log.
2016-07-17 01:36:50.208:
CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /grid/grid_home/log/node1/client/crsctl_orardbms.log.
2016-07-17 01:37:03.264:
CRS-5602:mDNS service stopping by request.
2016-07-17 01:37:06.085:
CRS-2329:GPNPD on node node1 shutdown.
2016-10-05 18:26:19.904:
CRS-2112:The OLR service started on node node1.
2016-10-05 18:26:20.192:
CRS-1301:Oracle High Availability Service started on node node1.
2016-10-05 18:26:20.295:
CRS-8017:location: /etc/oracle/lastgasp has 2 reboot advisory log files, 0 were announced and 0 errors occurred
2016-10-05 18:26:28.827:
CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2016-10-05 18:26:33.095:
CRS-2328:GPNPD started on node node1.
2016-10-05 18:26:35.882:
CRS-1713:CSSD daemon is started in clustered mode
2016-10-05 18:26:37.461:
CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2016-10-05 18:26:37.494:
CRS-2769:Unable to failover resource 'ora.diskmon'.
2016-10-05 18:26:45.859:
CRS-1707:Lease acquisition for node node1 number 1 completed
2016-10-05 18:26:47.144:
CRS-1605:CSSD voting file is online: /dev/mapper/vote_ocr; details in /grid/grid_home/log/node1/cssd/ocssd.log.
2016-10-05 18:26:55.712:
CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 node2 .
2016-10-05 18:26:58.047:
CRS-2407:The new Cluster Time Synchronization Service reference node is host node1.
2016-10-05 18:26:58.047:
CRS-2401:The Cluster Time Synchronization Service started on host node1.
CRS-10001:05-Oct-16 18:27 ACFS-9391: Checking for existing ADVM/ACFS installation.
CRS-10001:05-Oct-16 18:27 ACFS-9392: Validating ADVM/ACFS installation files for operating system.
CRS-10001:05-Oct-16 18:27 ACFS-9393: Verifying ASM Administrator setup.
CRS-10001:05-Oct-16 18:27 ACFS-9308: Loading installed ADVM/ACFS drivers.
CRS-10001:05-Oct-16 18:27 ACFS-9154: Loading 'oracleoks.ko' driver.
CRS-10001:05-Oct-16 18:27 ACFS-9154: Loading 'oracleadvm.ko' driver.
CRS-10001:05-Oct-16 18:27 ACFS-9154: Loading 'oracleacfs.ko' driver.
CRS-10001:05-Oct-16 18:27 ACFS-9327: Verifying ADVM/ACFS devices.
CRS-10001:05-Oct-16 18:27 ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
CRS-10001:05-Oct-16 18:27 ACFS-9156: Detecting control device '/dev/ofsctl'.
CRS-10001:05-Oct-16 18:27 ACFS-9322: completed
2016-10-05 18:27:31.947:
CRS-1012:The OCR service started on node node1.
2016-10-05 18:27:31.978:
CRS-1401:EVMD started on node node1.
2016-10-05 18:27:36.399:
CRS-1201:CRSD started on node node1.
2016-10-05 18:27:44.520:
CRS-2772:Server 'node1' has been assigned to pool 'Generic'.
2016-10-05 18:27:44.531:
CRS-2772:Server 'node1' has been assigned to pool 'ora.orcc'.
2016-10-05 18:27:45.125:
CRS-4743:File /grid/grid_home/oc4j/j2ee/home/OC4J_DBWLM_config/system-jazn-data.xml was updated from OCR(Size: 13365(New), 13378(Old) bytes)
2016-10-05 18:28:01.879:
CRS-2772:Server 'node2' has been assigned to pool 'Generic'.
2016-10-05 18:28:01.880:
CRS-2772:Server 'node2' has been assigned to pool 'ora.orcc'.
2016-10-05 18:46:58.408:
ent_root/orarootagent_root.log.
2016-10-05 18:46:59.354:
CRS-2405:The Cluster Time Synchronization Service on host node1 is shutdown by user
2016-10-05 18:47:08.480:
CRS-1625:Node node2, number 2, was manually shut down
2016-10-05 18:47:08.486:
CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 .
2016-10-05 18:47:10.382:
CRS-1603:CSSD on node node1 shutdown by user.
2016-10-05 18:47:10.499:
CRS-2767:Resource state recovery not attempted for 'ora.cssdmonitor' as its target state is OFFLINE
2016-10-05 18:47:10.499:
CRS-2769:Unable to failover resource 'ora.cssdmonitor'.
2016-10-05 18:47:55.311:
CRS-1713:CSSD daemon is started in clustered mode
2016-10-05 18:47:55.399:
CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /grid/grid_home/log/node1/cssd/ocssd.log
2016-10-05 18:47:57.155:
CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2016-10-05 18:47:57.156:
CRS-2769:Unable to failover resource 'ora.diskmon'.
2016-10-05 18:48:10.416:
CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /grid/grid_home/log/node1/cssd/ocssd.log
2016-10-05 18:48:25.429:
CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /grid/grid_home/log/node1/cssd/ocssd.log
2016-10-05 18:48:40.452:
CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /grid/grid_home/log/node1/cssd/ocssd.log
2016-10-05 18:48:55.467:
CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /grid/grid_home/log/node1/cssd/ocssd.log
^C
#
#
#
# tail -200f /grid/grid_home/log/node1/cssd/ocssd.log
2016-10-05 18:49:23.020: [ CSSD]clssscSelect: cookie accept request 0x7f3d48084620
2016-10-05 18:49:23.020: [ CSSD]clssscevtypSHRCON: getting client with cmproc 0x7f3d48084620
2016-10-05 18:49:23.020: [ CSSD]clssgmRegisterClient: proc(4/0x7f3d48084620), client(86/0x7f3d480596c0)
2016-10-05 18:49:23.020: [ CSSD]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(0x7f3d48084620) client(0x7f3d480596c0)
2016-10-05 18:49:23.020: [ CSSD]clssgmDiscEndpcl: gipcDestroy 0x16fe
2016-10-05 18:49:23.371: [ CSSD]clssscSelect: cookie accept request 0xc7f8f0
2016-10-05 18:49:23.371: [ CSSD]clssgmAllocProc: (0x7f3d480717d0) allocated
2016-10-05 18:49:23.372: [ CSSD]clssgmClientConnectMsg: properties of cmProc 0x7f3d480717d0 - 1,2,3,4,5
2016-10-05 18:49:23.372: [ CSSD]clssgmClientConnectMsg: Connect from con(0x1730) proc(0x7f3d480717d0) pid(3053) version 11:2:1:4, properties: 1,2,3,4,5
2016-10-05 18:49:23.372: [ CSSD]clssgmClientConnectMsg: msg flags 0x0000
2016-10-05 18:49:23.857: [ CSSD]clssscSelect: cookie accept request 0x7f3d4806ed00
2016-10-05 18:49:23.857: [ CSSD]clssscevtypSHRCON: getting client with cmproc 0x7f3d4806ed00
2016-10-05 18:49:23.857: [ CSSD]clssgmRegisterClient: proc(3/0x7f3d4806ed00), client(87/0x7f3d48096b90)
2016-10-05 18:49:23.857: [ CSSD]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(0x7f3d4806ed00) client(0x7f3d48096b90)
2016-10-05 18:49:23.857: [ CSSD]clssgmDiscEndpcl: gipcDestroy 0x1756
2016-10-05 18:49:24.022: [ CSSD]clssscSelect: cookie accept request 0x7f3d48084620
2016-10-05 18:49:24.023: [ CSSD]clssscevtypSHRCON: getting client with cmproc 0x7f3d48084620
2016-10-05 18:49:24.023: [ CSSD]clssgmRegisterClient: proc(4/0x7f3d48084620), client(87/0x7f3d48096b90)
2016-10-05 18:49:24.023: [ CSSD]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(0x7f3d48084620) client(0x7f3d48096b90)
2016-10-05 18:49:24.023: [ CSSD]clssgmDiscEndpcl: gipcDestroy 0x176c
2016-10-05 18:49:24.860: [ CSSD]clssscSelect: cookie accept request 0x7f3d4806ed00
2016-10-05 18:49:24.860: [ CSSD]clssscevtypSHRCON: getting client with cmproc 0x7f3d4806ed00
2016-10-05 18:49:24.860: [ CSSD]clssgmRegisterClient: proc(3/0x7f3d4806ed00), client(88/0x7f3d48096b90)
2016-10-05 18:49:24.860: [ CSSD]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(0x7f3d4806ed00) client(0x7f3d48096b90)
2016-10-05 18:49:24.860: [ CSSD]clssgmDiscEndpcl: gipcDestroy 0x1782
2016-10-05 18:49:25.025: [ CSSD]clssscSelect: cookie accept request 0x7f3d48084620
2016-10-05 18:49:25.025: [ CSSD]clssscevtypSHRCON: getting client with cmproc 0x7f3d48084620
2016-10-05 18:49:25.026: [ CSSD]clssgmRegisterClient: proc(4/0x7f3d48084620), client(88/0x7f3d480916b0)
2016-10-05 18:49:25.026: [ CSSD]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(0x7f3d48084620) client(0x7f3d480916b0)
2016-10-05 18:49:25.026: [ CSSD]clssgmDiscEndpcl: gipcDestroy 0x1798
2016-10-05 18:49:25.486: [ GPNP]clsgpnp_profileCallUrlInt: get-profile call to url "ipc://GPNPD_node1" disco ""
2016-10-05 18:49:25.499: [ GPNP]clsgpnp_profileCallUrlInt: Result: (0) CLSGPNP_OK. Successful get-profile CALL to remote "ipc://GPNPD_node1" disco ""
2016-10-05 18:49:25.500: [ CSSD]clssnmReadDiscoveryProfile: voting file discovery string(/dev/mapper/*)
2016-10-05 18:49:25.500: [ CSSD]clssnmvDDiscThread: using discovery string /dev/mapper/* for initial discovery
2016-10-05 18:49:25.500: [ SKGFD]Discovery with str:/dev/mapper/*:
2016-10-05 18:49:25.500: [ SKGFD]UFS discovery with :/dev/mapper/*:
2016-10-05 18:49:25.500: [ SKGFD]Execute glob on the string /dev/mapper/*
2016-10-05 18:49:25.500: [ SKGFD]running stat on disk:/dev/mapper/arch
2016-10-05 18:49:25.500: [ SKGFD]running stat on disk:/dev/mapper/vote_ocr
2016-10-05 18:49:25.500: [ SKGFD]running stat on disk:/dev/mapper/data
2016-10-05 18:49:25.500: [ SKGFD]running stat on disk:/dev/mapper/vg01-LogVol02
2016-10-05 18:49:25.500: [ SKGFD]running stat on disk:/dev/mapper/vg01-Log_ora
2016-10-05 18:49:25.500: [ SKGFD]running stat on disk:/dev/mapper/vg01-LogVol00
2016-10-05 18:49:25.500: [ SKGFD]running stat on disk:/dev/mapper/control
2016-10-05 18:49:25.500: [ SKGFD]Fetching UFS disk :/dev/mapper/control:
2016-10-05 18:49:25.500: [ SKGFD]Fetching UFS disk :/dev/mapper/vg01-LogVol00:
2016-10-05 18:49:25.500: [ SKGFD]Fetching UFS disk :/dev/mapper/vg01-Log_ora:
2016-10-05 18:49:25.500: [ SKGFD]Fetching UFS disk :/dev/mapper/vg01-LogVol02:
2016-10-05 18:49:25.500: [ SKGFD]Fetching UFS disk :/dev/mapper/data:
2016-10-05 18:49:25.500: [ SKGFD]Fetching UFS disk :/dev/mapper/vote_ocr:
2016-10-05 18:49:25.500: [ SKGFD]Fetching UFS disk :/dev/mapper/arch:
2016-10-05 18:49:25.500: [ SKGFD]OSS discovery with :/dev/mapper/*:
2016-10-05 18:49:25.500: [ CSSD]clssnmvDiskVerify: Successful discovery of 0 disks
2016-10-05 18:49:25.500: [ CSSD]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2016-10-05 18:49:25.500: [ CSSD]clssnmvFindInitialConfigs: No voting files found
2016-10-05 18:49:25.500: [ CSSD](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found. Retrying discovery in 15 seconds
2016-10-05 18:49:25.862: [ CSSD]clssscSelect: cookie accept request 0x7f3d4806ed00
2016-10-05 18:49:25.863: [ CSSD]clssscevtypSHRCON: getting client with cmproc 0x7f3d4806ed00
2016-10-05 18:49:25.863: [ CSSD]clssgmRegisterClient: proc(3/0x7f3d4806ed00), client(89/0x7f3d480630a0)
2016-10-05 18:49:25.863: [ CSSD]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(0x7f3d4806ed00) client(0x7f3d480630a0)
2016-10-05 18:49:25.863: [ CSSD]clssgmDiscEndpcl: gipcDestroy 0x17d6
2016-10-05 18:49:26.028: [ CSSD]clssscSelect: cookie accept request 0x7f3d48084620
2016-10-05 18:49:26.028: [ CSSD]clssscevtypSHRCON: getting client with cmproc 0x7f3d48084620
2016-10-05 18:49:26.028: [ CSSD]clssgmRegisterClient: proc(4/0x7f3d48084620), client(89/0x7f3d48096b90)
2016-10-05 18:49:26.028: [ CSSD]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(0x7f3d48084620) client(0x7f3d48096b90)
2016-10-05 18:49:26.028: [ CSSD]clssgmDiscEndpcl: gipcDestroy 0x17ec
2016-10-05 18:49:26.866: [ CSSD]clssscSelect: cookie accept request 0x7f3d4806ed00
2016-10-05 18:49:26.866: [ CSSD]clssscevtypSHRCON: getting client with cmproc 0x7f3d4806ed00
2016-10-05 18:49:26.866: [ CSSD]clssgmRegisterClient: proc(3/0x7f3d4806ed00), client(90/0x7f3d480916b0)
2016-10-05 18:49:26.866: [ CSSD]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(0x7f3d4806ed00) client(0x7f3d480916b0)
2016-10-05 18:49:26.866: [ CSSD]clssgmDiscEndpcl: gipcDestroy 0x1802
2016-10-05 18:49:27.031: [ CSSD]clssscSelect: cookie accept request 0x7f3d48084620
2016-10-05 18:49:27.031: [ CSSD]clssscevtypSHRCON: getting client with cmproc 0x7f3d48084620
2016-10-05 18:49:27.031: [ CSSD]clssgmRegisterClient: proc(4/0x7f3d48084620), client(90/0x7f3d480630a0)
2016-10-05 18:49:27.031: [ CSSD]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(0x7f3d48084620) client(0x7f3d480630a0)
2016-10-05 18:49:27.031: [ CSSD]clssgmDiscEndpcl: gipcDestroy 0x1818
2016-10-05 18:49:27.291: [ CSSD]clssgmExecuteClientRequest(): type(37) size(80) only connect and exit messages are allowed before lease acquisition proc(0x7f3d480717d0) client((nil))
2016-10-05 18:49:27.293: [ CSSD]clssgmDeadProc: proc 0x7f3d480717d0
2016-10-05 18:49:27.293: [ CSSD]clssgmDestroyProc: cleaning up proc(0x7f3d480717d0) con(0x1730) skgpidospid 3053 with 0 clients, refcount 0
2016-10-05 18:49:27.293: [ CSSD]clssgmDiscEndpcl: gipcDestroy 0x1730
^C
#
#
# pwd
/grid/grid_home/log/node1
# ls -l /dev/asm-*
brw-rw---- 1 oragrid asmadmin 8,48 10?5 18:44 /dev/asm-diskc
brw-rw---- 1 oragrid asmadmin 8,64 10?5 18:44 /dev/asm-diske
brw-rw---- 1 oragrid asmadmin 8, 112 10?5 18:44 /dev/asm-diskg
# kfed read /dev/asm-diskc| grep kfdhdb.grpname
kfdhdb.grpname: OCRVOTE ; 0x048: length=7
# kfed read /dev/asm-diske| grep kfdhdb.grpname
kfdhdb.grpname: DATA ; 0x048: length=4
# kfed read /dev/asm-diskg| grep kfdhdb.grpname
kfdhdb.grpname: ARCH ; 0x048: length=4
#
#
#
#
#
#
#
# crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
#
#
#
# ssh node2
root@node2's password:
Last login: Wed Oct5 18:44:42 2016 from node1
# crsctl check crs
-bash: crsctl: command not found
# su - oragrid
$ cd $ORACLE_HOME/bin
$ su
??:
# crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
# exit
exit
$ exit
logout
# exit
logout
Connection to node2 closed.
#
#
#
# whoami
root
#
#
# crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'node1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'node1'
CRS-2677: Stop of 'ora.mdnsd' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'node1'
CRS-2673: Attempting to stop 'ora.crf' on 'node1'
CRS-2677: Stop of 'ora.drivers.acfs' on 'node1' succeeded
CRS-2677: Stop of 'ora.crf' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'node1'
CRS-2677: Stop of 'ora.gipcd' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'node1'
CRS-2677: Stop of 'ora.gpnpd' on 'node1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'node1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
#
#
#
#
#
# crsctl start crs -excl -nocrs
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd' on 'node1'
CRS-2676: Start of 'ora.mdnsd' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'node1'
CRS-2676: Start of 'ora.gpnpd' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'node1'
CRS-2672: Attempting to start 'ora.gipcd' on 'node1'
CRS-2676: Start of 'ora.cssdmonitor' on 'node1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'node1'
CRS-2672: Attempting to start 'ora.diskmon' on 'node1'
CRS-2676: Start of 'ora.diskmon' on 'node1' succeeded
CRS-2676: Start of 'ora.cssd' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'node1'
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'node1'
CRS-2672: Attempting to start 'ora.ctssd' on 'node1'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'node1'
CRS-2676: Start of 'ora.drivers.acfs' on 'node1' succeeded
CRS-2676: Start of 'ora.ctssd' on 'node1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'node1'
CRS-2676: Start of 'ora.asm' on 'node1' succeeded
#
#
# exit
exit
$
$
$
$
$ echo $ORACLE_SID
+ASM1
$ sqlplus /nolog
SQL*Plus: Release 11.2.0.4.0 Production on Wed Oct 5 19:00:05 2016
Copyright (c) 1982, 2013, Oracle.All rights reserved.
SQL> connect /as sysasm
Connected.
SQL> select open_mode from v$instance;
select open_mode from v$instance
*
ERROR at line 1:
ORA-00904: "OPEN_MODE": invalid identifier
SQL> desc v$instance;
Name Null? Type
----------------------------------------- -------- ----------------------------
INSTANCE_NUMBER NUMBER
INSTANCE_NAME VARCHAR2(16)
HOST_NAME VARCHAR2(64)
VERSION VARCHAR2(17)
STARTUP_TIME DATE
STATUS VARCHAR2(12)
PARALLEL VARCHAR2(3)
THREAD# NUMBER
ARCHIVER VARCHAR2(7)
LOG_SWITCH_WAIT VARCHAR2(15)
LOGINS VARCHAR2(10)
SHUTDOWN_PENDING VARCHAR2(3)
DATABASE_STATUS VARCHAR2(17)
INSTANCE_ROLE VARCHAR2(18)
ACTIVE_STATE VARCHAR2(9)
BLOCKED VARCHAR2(3)
SQL> select status from v$instance;
STATUS
------------
STARTED
SQL>
SQL>
SQL>
SQL>
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
$ ps -ef | grep -i asm
oragrid 8024 10 18:59 ? 00:00:00 asm_pmon_+ASM1
oragrid 8026 10 18:59 ? 00:00:00 asm_psp0_+ASM1
oragrid 8028 12 18:59 ? 00:00:02 asm_vktm_+ASM1
oragrid 8032 10 18:59 ? 00:00:00 asm_gen0_+ASM1
oragrid 8034 10 18:59 ? 00:00:00 asm_diag_+ASM1
oragrid 8036 10 18:59 ? 00:00:00 asm_ping_+ASM1
oragrid 8038 10 18:59 ? 00:00:00 asm_dia0_+ASM1
oragrid 8040 10 18:59 ? 00:00:00 asm_lmon_+ASM1
oragrid 8042 10 18:59 ? 00:00:00 asm_lmd0_+ASM1
oragrid 8044 10 18:59 ? 00:00:00 asm_lms0_+ASM1
oragrid 8048 10 18:59 ? 00:00:00 asm_lmhb_+ASM1
oragrid 8050 10 18:59 ? 00:00:00 asm_mman_+ASM1
oragrid 8052 10 18:59 ? 00:00:00 asm_dbw0_+ASM1
oragrid 8054 10 18:59 ? 00:00:00 asm_lgwr_+ASM1
oragrid 8056 10 18:59 ? 00:00:00 asm_ckpt_+ASM1
oragrid 8058 10 18:59 ? 00:00:00 asm_smon_+ASM1
oragrid 8060 10 18:59 ? 00:00:00 asm_rbal_+ASM1
oragrid 8062 10 18:59 ? 00:00:00 asm_gmon_+ASM1
oragrid 8064 10 18:59 ? 00:00:00 asm_mmon_+ASM1
oragrid 8066 10 18:59 ? 00:00:00 asm_mmnl_+ASM1
oragrid 8068 10 18:59 ? 00:00:00 asm_lck0_+ASM1
oragrid 8099 10 18:59 ? 00:00:00 oracle+ASM1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
oragrid 8209 68650 19:00 pts/0 00:00:00 grep -i asm
$
$
$
$
$ sqlplus /nolog
SQL*Plus: Release 11.2.0.4.0 Production on Wed Oct 5 19:00:53 2016
Copyright (c) 1982, 2013, Oracle.All rights reserved.
SQL> connect /as sysasm
Connected.
SQL> show parameter disk
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups string
asm_diskstring string
SQL> show parameter spfile;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string
SQL>
SQL>
SQL> show parameter
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
_dummy_instance boolean TRUE
asm_diskgroups string
asm_diskstring string
asm_power_limit integer 1
asm_preferred_read_failure_groups string
audit_file_dest string /grid/grid_home/rdbms/audit
audit_sys_operations boolean FALSE
audit_syslog_level string
background_core_dump string partial
background_dump_dest string /grid/grid_home/log/diag/asm/+
asm/+ASM1/trace
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
cluster_database boolean TRUE
cluster_database_instances integer 4
cluster_interconnects string
core_dump_dest string /grid/grid_home/log/diag/asm/+
asm/+ASM1/cdump
cpu_count integer 4
db_cache_size big integer 0
db_ultra_safe string OFF
db_unique_name string +ASM
diagnostic_dest string /grid/grid_home/log
event string
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
file_mapping boolean FALSE
filesystemio_options string none
ifile file
instance_name string +ASM1
instance_number integer 1
instance_type string ASM
large_pool_size big integer 0
ldap_directory_sysauth string no
listener_networks string
local_listener string
lock_name_space string
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
lock_sga boolean FALSE
max_dump_file_size string unlimited
memory_max_target big integer 1076M
memory_target big integer 1076M
nls_calendar string
nls_comp string BINARY
nls_currency string
nls_date_format string
nls_date_language string
nls_dual_currency string
nls_iso_currency string
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
nls_language string AMERICAN
nls_length_semantics string BYTE
nls_nchar_conv_excp string FALSE
nls_numeric_characters string
nls_sort string
nls_territory string AMERICA
nls_time_format string
nls_time_tz_format string
nls_timestamp_format string
nls_timestamp_tz_format string
os_authent_prefix string ops$
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
os_roles boolean FALSE
parallel_execution_message_size integer 16384
pga_aggregate_target big integer 0
processes integer 360
remote_listener string
remote_login_passwordfile string EXCLUSIVE
remote_os_authent boolean FALSE
remote_os_roles boolean FALSE
service_names string +ASM
sessions integer 564
sga_max_size big integer 1088M
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
sga_target big integer 0
shadow_core_dump string partial
shared_pool_reserved_size big integer 30M
shared_pool_size big integer 0
sort_area_size integer 65536
spfile string
sql_trace boolean FALSE
statistics_level string TYPICAL
timed_os_statistics integer 0
timed_statistics boolean TRUE
trace_enabled boolean TRUE
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
use_large_pages string TRUE
user_dump_dest string /grid/grid_home/log/diag/asm/+
asm/+ASM1/trace
workarea_size_policy string AUTO
SQL>
SQL>
SQL>
SQL>
SQL>alter system set asm_diskstring='/dev/asm-*';
System altered.
SQL> alter diskgroup data mount;
Diskgroup altered.
SQL> alter diskgroup ocrvote mount;
Diskgroup altered.
SQL> alter diskgroup arch mount;
Diskgroup altered.
SQL> show parameter spfile;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string
SQL> show parameter disk
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups string DATA, OCRVOTE, ARCH
asm_diskstring string /dev/asm-*
SQL> create spfile from memory;
create spfile from memory
*
ERROR at line 1:
ORA-17502: ksfdcre:4 Failed to create file
+OCRVOTE/node-cluster/asmparameterfile/registry.253.917395543
ORA-15177: cannot operate on system aliases
SQL> create spfile from memory;
File created.
SQL> startup force mount;
ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance
ASM instance started
Total System Global Area 1135747072 bytes
Fixed Size 2260728 bytes
Variable Size 1108320520 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted
ASM diskgroups volume enabled
SQL> show parameter spfile
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string /grid/grid_home/dbs/spfile+ASM
1.ora
SQL> show parameter disk
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups string DATA, OCRVOTE, ARCH
asm_diskstring string /dev/asm-*
SQL>
SQL>
SQL>
SQL>
SQL> create spfile='+OCRVOTE' FROMpfile;
create spfile='+OCRVOTE' FROMpfile
*
ERROR at line 1:
ORA-01078: failure in processing system parameters
LRM-00109: could not open parameter file '/grid/grid_home/dbs/init+ASM1.ora'
SQL> create pfile from spfile;
File created.
SQL> create spfile='+OCRVOTE' FROMpfile;
File created.
SQL> startup force;
ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance
ASM instance started
Total System Global Area 1135747072 bytes
Fixed Size 2260728 bytes
Variable Size 1108320520 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted
ASM diskgroups volume enabled
SQL> show parameter spfile;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string +OCRVOTE/node-cluster/asmparam
eterfile/registry.253.92446232
5
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
$ asmcmd
ASMCMD> lsdg
State Type RebalSectorBlock AUTotal_MBFree_MBReq_mir_free_MBUsable_file_MBOffline_disksVoting_filesName
MOUNTEDEXTERNN 512 40961048576 8544 8135 0 8135 0 NARCH/
MOUNTEDEXTERNN 512 40961048576 11296 8543 0 8543 0 NDATA/
MOUNTEDEXTERNN 512 40961048576 5728 5364 0 5364 0 NOCRVOTE/
ASMCMD> lsdsk
Path
/dev/asm-diskc
/dev/asm-diske
/dev/asm-diskg
ASMCMD> pwd
+
ASMCMD> ls
ARCH/
DATA/
OCRVOTE/
ASMCMD> cd ocrvote
ASMCMD> ls
node-cluster/
ASMCMD> cd node-cluster
ASMCMD> ls
ASMPARAMETERFILE/
OCRFILE/
ASMCMD> cd asmparameterfile
ASMCMD> ls
REGISTRY.253.924462325
ASMCMD> exit
$ crsctl query css votedisk
Located 0 voting disk(s).
$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3100
Available space (kbytes) : 259020
ID : 16088418
Device/File Name : +OCRVOTE
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user
$ crsctl replace votedisk +OCRVOTE
Successful addition of voting disk 8a8c153dabef4f71bfe5c3ebc4074c1e.
Successfully replaced voting disk group with +OCRVOTE.
CRS-4266: Voting file(s) successfully replaced
$
$
$ crsctl query css votedisk
##STATE File Universal Id File Name Disk group
------- ----------------- --------- ---------
1. ONLINE 8a8c153dabef4f71bfe5c3ebc4074c1e (/dev/asm-diskc)
Located 1 voting disk(s).
$
$
$
$ pwd
/grid/grid_home/log/node1/cssd
$ su
??:
#
#
# crsctl stop cluster -all
CRS-2673: Attempting to stop 'ora.ctssd' on 'node1'
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'node1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'node1' succeeded
CRS-2796: The command may not proceed when Cluster Ready Services is not running
CRS-2677: Stop of 'ora.ctssd' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'node1'
CRS-2677: Stop of 'ora.cssd' on 'node1' succeeded
# crsctl check cluster -all
**************************************************************
node1:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
**************************************************************
node2:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
**************************************************************
# crsctl start cluster -all
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'node1'
CRS-2676: Start of 'ora.cssdmonitor' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'node1'
CRS-2672: Attempting to start 'ora.diskmon' on 'node1'
CRS-2676: Start of 'ora.diskmon' on 'node1' succeeded
CRS-2676: Start of 'ora.cssd' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'node1'
CRS-2676: Start of 'ora.ctssd' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'node1'
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'node1'
CRS-5702: Resource 'ora.evmd' is already running on 'node2'
CRS-2676: Start of 'ora.evmd' on 'node1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'node1'
CRS-2676: Start of 'ora.asm' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'node1'
CRS-2676: Start of 'ora.crsd' on 'node1' succeeded
CRS-5702: Resource 'ora.crsd' is already running on 'node2'
# crsctl check cluster -all
**************************************************************
node1:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
node2:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
# crsctl check cluster -all
**************************************************************
node1:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
node2:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
# crsctl check cluster -all
**************************************************************
node1:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
node2:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
# ssh node2
root@node2's password:
Last login: Wed Oct5 18:55:56 2016 from node1
# shutdown -r now
Broadcast message from root@node2
(/dev/pts/1) at 19:14 ...
The system is going down for reboot NOW!
# exit
logout
Connection to node2 closed.
# shutdown -r now
Broadcast message from root@node1
(/dev/pts/0) at 19:14 ...
The system is going down for reboot NOW!
2016/10/5 11:15:23
$ ping 192.168.1.155
?? Ping 192.168.1.155 ?? 32 ?????:
?? 192.168.1.155 ???: ??=32 ??<1ms TTL=64
?? 192.168.1.155 ???: ??=32 ??<1ms TTL=64
192.168.1.155 ? Ping ????:
???: ??? = 2,??? = 2,?? = 0 (0% ??),
?????????(??????):
?? = 0ms,?? = 0ms,?? = 0ms
$ ssh 192.168.1.155
Connecting to 192.168.1.155:22...
Connection established.
To escape to local shell, press 'Ctrl+Alt+]'.
Last login: Wed Oct5 18:59:04 2016 from 192.168.1.1
#
#
#
#
# ps -ef | grep d.bin
root 2587 15 19:15 ? 00:00:01 /grid/grid_home/bin/ohasd.bin reboot
oragrid 2914 10 19:16 ? 00:00:00 /grid/grid_home/bin/mdnsd.bin
oragrid 2927 11 19:16 ? 00:00:00 /grid/grid_home/bin/gpnpd.bin
oragrid 2937 11 19:16 ? 00:00:00 /grid/grid_home/bin/gipcd.bin
root 2951 12 19:16 ? 00:00:00 /grid/grid_home/bin/osysmond.bin
oragrid 2995 12 19:16 ? 00:00:00 /grid/grid_home/bin/ocssd.bin
root 3127 30010 19:16 pts/0 00:00:00 grep d.bin
# su - oragrid
$ crsctl check cluster -all
**************************************************************
node1:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4534: Cannot communicate with Event Manager
**************************************************************
$ crsctl check cluster -all
**************************************************************
node1:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4534: Cannot communicate with Event Manager
**************************************************************
$ crsctl check cluster -all
**************************************************************
node1:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4534: Cannot communicate with Event Manager
**************************************************************
$ crsctl check cluster -all
**************************************************************
node1:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4534: Cannot communicate with Event Manager
**************************************************************
node2:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
**************************************************************
$ crsctl check cluster -all
**************************************************************
node1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
node2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.ARCH.dg ora....up.type ONLINE ONLINE node1
ora.DATA.dg ora....up.type ONLINE ONLINE node1
ora....ER.lsnr ora....er.type ONLINE ONLINE node1
ora....N1.lsnr ora....er.type ONLINE ONLINE node1
ora.OCRVOTE.dg ora....up.type ONLINE ONLINE node1
ora.asm ora.asm.type ONLINE ONLINE node1
ora.cvu ora.cvu.type ONLINE ONLINE node1
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE node1
ora....SM1.asm application ONLINE ONLINE node1
ora....E1.lsnr application ONLINE ONLINE node1
ora.node1.gsdapplication OFFLINE OFFLINE
ora.node1.onsapplication ONLINE ONLINE node1
ora.node1.vipora....t1.type ONLINE ONLINE node1
ora....SM2.asm application ONLINE ONLINE node2
ora....E2.lsnr application ONLINE ONLINE node2
ora.node2.gsdapplication OFFLINE OFFLINE
ora.node2.onsapplication ONLINE OFFLINE
ora.node2.vipora....t1.type ONLINE ONLINE node2
ora.oc4j ora.oc4j.typeONLINE ONLINE node1
ora.ons ora.ons.type ONLINE ONLINE node1
ora.orcc.db ora....se.type ONLINE OFFLINE
ora....ry.acfs ora....fs.type ONLINE ONLINE node1
ora.scan1.vipora....ip.type ONLINE ONLINE node1
$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.ARCH.dg ora....up.type ONLINE ONLINE node1
ora.DATA.dg ora....up.type ONLINE ONLINE node1
ora....ER.lsnr ora....er.type ONLINE ONLINE node1
ora....N1.lsnr ora....er.type ONLINE ONLINE node1
ora.OCRVOTE.dg ora....up.type ONLINE ONLINE node1
ora.asm ora.asm.type ONLINE ONLINE node1
ora.cvu ora.cvu.type ONLINE ONLINE node1
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE node1
ora....SM1.asm application ONLINE ONLINE node1
ora....E1.lsnr application ONLINE ONLINE node1
ora.node1.gsdapplication OFFLINE OFFLINE
ora.node1.onsapplication ONLINE ONLINE node1
ora.node1.vipora....t1.type ONLINE ONLINE node1
ora....SM2.asm application ONLINE ONLINE node2
ora....E2.lsnr application ONLINE ONLINE node2
ora.node2.gsdapplication OFFLINE OFFLINE
ora.node2.onsapplication ONLINE ONLINE node2
ora.node2.vipora....t1.type ONLINE ONLINE node2
ora.oc4j ora.oc4j.typeONLINE ONLINE node1
ora.ons ora.ons.type ONLINE ONLINE node1
ora.orcc.db ora....se.type ONLINE ONLINE node1
ora....ry.acfs ora....fs.type ONLINE ONLINE node1
ora.scan1.vipora....ip.type ONLINE ONLINE node1
$ asmcmd
ASMCMD> lsdsk
Path
/dev/asm-diskc
/dev/asm-diske
/dev/asm-diskg
ASMCMD> exit
$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3100
Available space (kbytes) : 259020
ID : 16088418
Device/File Name : +OCRVOTE
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user
$ crsctl query css votedisk
##STATE File Universal Id File Name Disk group
------- ----------------- --------- ---------
1. ONLINE 8a8c153dabef4f71bfe5c3ebc4074c1e (/dev/asm-diskc)
Located 1 voting disk(s).
$
$
$
$
2016/10/5 11:18:41
页:
[1]