Check ASM Diskgroup Space and Directory Size

The script below can be used to check the Disk Group Space Free and also Check Directory sizes for each Disk Group

The script is written by somebody at Pythian but i cannot re-collect the original link to the blog.

eg:
./asmcmd_du.sh

DiskGroup Total_MB Free_MB % Free
——— ——– ——- ——
DATAC1 15962112 11215880 70
DBFS_DG 415296 403740 97
RECOC1 3996000 3460272 86

./asmcmd_du.sh DATAC1

DiskGroup Total_MB Free_MB % Free
——— ——– ——- ——
DATAC1 15962112 10155732 63

DATAC1 subdirectories size

Subdir Used MB Mirror MB
—— ——- ———
CARATST/ 55646 111356
ECCTST/ 174912 349856
—— ——- ———
Total 2799978 5600788

Script Below :

[code language=”bash”]
#!/bin/bash
# Shadab Mohammad — 2016
#
# – If no parameter specified, show a du of each DiskGroup
# – If a parameter, print a du of each subdirectory
#

D=$1

#
# Colored thresholds (Red, Yellow, Green)
#
CRITICAL=90
WARNING=75

#
# Set the ASM env
#
ORACLE_SID=`ps -ef | grep pmon | grep asm | awk ‘{print $NF}’ | sed s’/asm_pmon_//’ | egrep "^[+]"`
export ORAENV_ASK=NO
. oraenv > /dev/null 2>&1

#
# A quick list of what is running on the server
#
ps -ef | grep pmon | grep -v grep | awk ‘{print $NF}’ | sed s’/.*_pmon_//’ | egrep "^([+]|[Aa-Zz])" | sort | awk -v H="`hostname -s`" ” | sed s’/, $//’

#
# Manage parameters
#
if [[ -z $D ]]
then # No directory provided, will check all the DG
DG=`asmcmd lsdg | grep -v State | awk ‘{print $NF}’ | sed s’/\///’`
SUBDIR="No" # Do not show the subdirectories details if no directory is specified
else
DG=`echo $D | sed s’/\/.*$//g’`
fi

#
# A header
#
printf "\n%25s%16s%16s%14s" "DiskGroup" "Total_MB" "Free_MB" "% Free"
printf "\n%25s%16s%16s%14s\n" "———" "——–" "——-" "——"

#
# Show DG info
#
for X in ${DG}
do
asmcmd lsdg ${X} | tail -1 |\
awk -v DG="$X" -v W="$WARNING" -v C="$CRITICAL" ‘\
BEGIN \
{COLOR_BEGIN = "\033[1;" ;
COLOR_END = "\033[m" ;
RED = COLOR_BEGIN"31m" ;
GREEN = COLOR_BEGIN"32m" ;
YELLOW = COLOR_BEGIN"33m" ;
COLOR = GREEN ;
}
{ FREE = sprintf("%12d", $8/$7*100) ;
if ((100-FREE) > W) {COLOR=YELLOW ;}
if ((100-FREE) > C) {COLOR=RED ;}
printf("%25s%16s%16s%s\n", DG, $7, $8, COLOR FREE COLOR_END) ; }’
done
printf "\n"

#
# Subdirs info
#
if [ -z ${SUBDIR} ]
then
(for DIR in `asmcmd ls ${D}`
do
echo ${DIR} `asmcmd du ${D}/${DIR} | tail -1`
done) | awk -v D="$D" ‘ BEGIN { printf("\n\t\t%40s\n\n", D " subdirectories size") ;
printf("%25s%16s%16s\n", "Subdir", "Used MB", "Mirror MB") ;
printf("%25s%16s%16s\n", "——", "——-", "———") ;}
{
printf("%25s%16s%16s\n", $1, $2, $3) ;
use += $2 ;
mir += $3 ;
}
END { printf("\n\n%25s%16s%16s\n", "——", "——-", "———") ;
printf("%25s%16s%16s\n\n", "Total", use, mir) ;} ‘
fi

#************************************************************************#
#* E N D O F S O U R C E *#
#************************************************************************#
[/code]

Oracle 12c Dataguard RAC Primary to RAC Standby with Dataguard Broker (on Exadata)

Setup for the Environment
——————-

Exadata X4-2 Quater Rack n Primary and Standby Site
Grid Version : 12.1.0.2
DB Home Version : 12.1.0.2

Primary, 2 Node RAC, db_unique_name = primeprd
Instance 2 : primeprd1
Instance 2 : primeprd2
TNS entry: PRIMEPRD, but below entry in tnsnames.ora files on both node with DB Home user, most likely it will be Oracle user.

PRIMEPRD =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = primaryhost.domain.com)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = primeprd)
)
)

Standby, 2 Node RAC, db_unique_name= primedr
Instance 2 : primedr1
Instance 2 : primedr2
TNS ENTRY: PRIMEDR, but below entry in tnsnames.ora files on both node with DB Home user, most likely it will be Oracle user.

PRIMEDR =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = standbyhost.domain.com)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = primedr)
)
)

1. Enable Force Logging on the Database

[code language=”sql”]
alter database force logging;
[/code]

Database altered.

2.1 change the below parameter in Primary database.

[code language=”sql”]

ALTER SYSTEM SET db_recovery_file_dest=’+RECOC1′ scope=both sid=’*’;

System altered.

alter system set LOG_ARCHIVE_CONFIG=’DG_CONFIG=(primeprd,primedr)’ scope=both sid=’*’;

alter system set LOG_ARCHIVE_DEST_1=’LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=primeprd’ scope=both sid=’*’;

alter system set LOG_ARCHIVE_DEST_2=’SERVICE=primedr LGWR ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=primedr’ scope=both sid=’*’;

alter system set log_archive_format=’%t_%s_%r.arc’ scope=spfile sid=’*’;

alter system set log_archive_max_processes=8 scope=both sid=’*’;

alter system set fal_server=primedr scope=both sid=’*’;

alter system set fal_client=primeprd scope=both sid=’*’;

alter system set standby_file_management=AUTO scope=both sid=’*’;

alter system set db_file_name_convert=(‘+DATAC1/primedr/’, ‘+DATAC1/primeprd/’, ‘+RECOC1/primedr/’, ‘+RECOC1/primeprd/’) scope=both sid=’*’;

alter system set log_file_name_convert=(‘+DATAC1/primedr/’, ‘+DATAC1/primeprd/’, ‘+RECOC1/primedr/’, ‘+RECOC1/primeprd/’) scope=both sid=’*’;

— Query to check if parameters are set correctly —

set linesize 500 pages 0
col value for a90
col name for a50
select name, value
from v$parameter
where name in (‘db_name’,’db_unique_name’,’log_archive_config’, ‘log_archive_dest_1′,’log_archive_dest_2’,
‘log_archive_dest_state_1′,’log_archive_dest_state_2’, ‘remote_login_passwordfile’,
‘log_archive_format’,’log_archive_max_processes’,’fal_server’,’db_file_name_convert’,
‘log_file_name_convert’, ‘standby_file_management’);

select name, value
from v$parameter
where name in (‘db_name’,’db_unique_name’,’log_archive_config’, ‘log_archive_dest_1′,’log_archive_dest_2’,
‘log_archive_dest_state_1′,’log_archive_dest_state_2’, ‘remote_login_passwordfile’,
‘log_archive_format’,’log_archive_max_processes’,’fal_server’,’db_file_name_convert’,
‘log_file_name_convert’, ‘standby_file_management’);

[/code]

3.1. verify the logfile using the below querys.

[code language=”sql”]
select group#,THREAD#,bytes/1024/1024,status,members from v$log;
select member from v$logfile;
[/code]

3.2. Add the below logfile and standby logfile in primary database, make sure you have large redolog files. In this day and age, 100MB does not really cut it.

[code language=”sql”]
ALTER DATABASE ADD LOGFILE THREAD 1 GROUP 1 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD LOGFILE THREAD 1 GROUP 2 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD LOGFILE THREAD 1 GROUP 3 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD LOGFILE THREAD 1 GROUP 4 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 5 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 6 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 7 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 8 (‘+DATAC1′,’+RECOC1’) SIZE 1G;

select * from v$standby_log;

— Add Standby Redolog files, number of standby redo log files = Redo logfile Groups + 2 —

ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 9 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 10 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 11 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 12 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 13 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 2 GROUP 14 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 2 GROUP 15 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 2 GROUP 16 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 2 GROUP 17 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 2 GROUP 18 (‘+DATAC1′,’+RECOC1’) SIZE 1G;
[/code]

4. Take the rman backup using below command.

[code language=”sql”]
rman target / nocatalog
run
{
sql ‘alter system archive log current’;
backup as compressed backupset database plus archivelog format ‘/u03/rmanbkp/online/Primary_bkp_for_stndby_%U’;
backup current controlfile for standby format ‘/u03/rmanbkp/online/stby.ctl’;
sql ‘alter system archive log current’;
}
[/code]

5.1. create the init file using the below command in primary database.

[code language=”sql”]
create pfile=’/u03/rmanbkp/prime/pfile_for_standby.txt’ from spfile;
[/code]

— Primary PFILE —

[code language=”sql”]
primeprd1.__data_transfer_cache_size=0
primeprd2.__data_transfer_cache_size=0
primeprd1.__db_cache_size=7046430720
primeprd2.__db_cache_size=7147094016
primeprd1.__java_pool_size=67108864
primeprd2.__java_pool_size=67108864
primeprd1.__large_pool_size=301989888
primeprd2.__large_pool_size=301989888
primeprd2.__oracle_base=’/u03/app/oracle’#ORACLE_BASE set from environment
primeprd1.__oracle_base=’/u03/app/oracle’#ORACLE_BASE set from environment
primeprd1.__pga_aggregate_target=3221225472
primeprd2.__pga_aggregate_target=3221225472
primeprd1.__sga_target=9663676416
primeprd2.__sga_target=9663676416
primeprd1.__shared_io_pool_size=469762048
primeprd2.__shared_io_pool_size=469762048
primeprd1.__shared_pool_size=1644167168
primeprd2.__shared_pool_size=1577058304
primeprd1.__streams_pool_size=100663296
primeprd2.__streams_pool_size=67108864
*.audit_file_dest=’/u03/app/oracle/admin/primeprd/adump’
*.audit_trail=’db’
*.cluster_database=TRUE
*.compatible=’12.1.0.2.0′
*.control_files=’+DATAC1/PRIMEPRD/CONTROLFILE/current.660.914322763′,’+DATAC1/PRIMEPRD/CONTROLFILE/current.665.914322763′
*.cpu_count=16
*.db_block_size=8192
*.db_create_file_dest=’+DATAC1′
*.db_domain=”
*.db_file_name_convert=’+DATAC1/primedr/’,’+DATAC1/primeprd/’,’+RECOC1/primedr/’,’+RECOC1/primeprd/’
*.db_name=’primeprd’
*.db_recovery_file_dest=’+DATAC1′
*.db_recovery_file_dest_size=536870912000
*.db_writer_processes=6
*.ddl_lock_timeout=30
*.diagnostic_dest=’/u03/app/oracle’
*.dispatchers='(PROTOCOL=TCP) (SERVICE=primeprdXDB)’
*.fal_client=’primeprd’
*.fal_server=’primedr’
primeprd2.instance_number=2
primeprd1.instance_number=1
*.log_archive_config=’dg_config=(primeprd,primedr)’
*.log_archive_dest_1=’LOCATION=+RECOC1 valid_for=(ALL_LOGFILES,ALL_ROLES) db_unique_name=primeprd’
*.log_archive_dest_2=’service=primedr LGWR ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=primedr’
*.log_archive_dest_state_1=’ENABLE’
*.log_archive_dest_state_2=’ENABLE’
*.log_archive_format=’log%t_%s_%r.arc’
*.log_file_name_convert=’+DATAC1/primedr/’,’+DATAC1/primeprd/’,’+RECOC1/primedr/’,’+RECOC1/primeprd/’
*.nls_length_semantics=’CHAR’
*.open_cursors=2000
*.optimizer_adaptive_features=FALSE
*.pga_aggregate_target=3072m
*.processes=1500
*.remote_login_passwordfile=’exclusive’
*.resource_manager_cpu_allocation=48
*.resource_manager_plan=’DEFAULT’
*.session_cached_cursors=100
*.sessions=2272
*.sga_max_size=17179869184
*.sga_target=9216m
*.standby_file_management=’AUTO’
*.temp_undo_enabled=TRUE
primeprd2.thread=2
primeprd1.thread=1
primeprd2.undo_tablespace=’UNDOTBS2′
primeprd1.undo_tablespace=’UNDOTBS1′
[/code]

— DR PFILE —
[code language=”sql”]
primedr1.__data_transfer_cache_size=0
primedr2.__data_transfer_cache_size=0
primedr1.__db_cache_size=7046430720
primedr2.__db_cache_size=7147094016
primedr1.__java_pool_size=67108864
primedr2.__java_pool_size=67108864
primedr1.__large_pool_size=301989888
primedr2.__large_pool_size=301989888
primedr2.__oracle_base=’/u03/app/oracle’#ORACLE_BASE set from environment
primedr1.__oracle_base=’/u03/app/oracle’#ORACLE_BASE set from environment
primedr1.__pga_aggregate_target=3221225472
primedr2.__pga_aggregate_target=3221225472
primedr1.__sga_target=9663676416
primedr2.__sga_target=9663676416
primedr1.__shared_io_pool_size=469762048
primedr2.__shared_io_pool_size=469762048
primedr1.__shared_pool_size=1644167168
primedr2.__shared_pool_size=1577058304
primedr1.__streams_pool_size=100663296
primedr2.__streams_pool_size=67108864
*.audit_file_dest=’/u03/app/oracle/admin/primeprd/adump’ # Create on Both Nodes
*.audit_trail=’db’
*.cluster_database=TRUE
*.compatible=’12.1.0.2.0′
#*.control_files=’+DATAC1/PRIMEPRD/CONTROLFILE/current.660.914322763′,’+DATAC1/PRIMEPRD/CONTROLFILE/current.665.914322763′
*.cpu_count=16
*.db_block_size=8192
*.db_create_file_dest=’+DATAC1′
*.db_domain=”
*.db_file_name_convert=’PRIMEPRD’,’PRIMEDR’ ##### Changed for dataguard
*.db_unique_name=’primedr’ ##### Changed for dataguard
*.db_name=’primeprd’
*.db_recovery_file_dest=’+RECOC1′
*.db_recovery_file_dest_size=536870912000
*.db_writer_processes=6
*.ddl_lock_timeout=30
*.diagnostic_dest=’/u03/app/oracle’
*.dispatchers='(PROTOCOL=TCP) (SERVICE=primedrXDB)’ ##### Changed for dataguard
*.fal_client=’primedr’ ##### Changed for dataguard
*.fal_server=’primeprd’ ##### Changed for dataguard
primedr2.instance_number=2
primedr1.instance_number=1
*.log_archive_config=’dg_config=(primeprd,primedr)’
*.log_archive_dest_1=’LOCATION=+RECOC1 valid_for=(ALL_LOGFILES,ALL_ROLES) db_unique_name=primedr’ ##### Changed for dataguard
*.log_archive_dest_2=’service=primeprd LGWR ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=primeprd’ ##### Changed for dataguard
*.log_archive_dest_state_1=’ENABLE’
*.log_archive_dest_state_2=’ENABLE’
*.log_archive_format=’log%t_%s_%r.arc’
*.log_file_name_convert=’PRIMEPRD’,’PRIMEDR’ ##### Changed for dataguard
*.remote_listener=’exadr-scan:1521′ ##### Changed for dataguard
*.nls_length_semantics=’CHAR’
*.open_cursors=2000
*.optimizer_adaptive_features=FALSE
*.pga_aggregate_target=3072m
*.processes=1500
*.remote_login_passwordfile=’exclusive’
*.resource_manager_cpu_allocation=48
*.resource_manager_plan=’DEFAULT’
*.session_cached_cursors=100
*.sessions=2272
*.sga_max_size=17179869184
*.sga_target=9216m
*.standby_file_management=’AUTO’
*.temp_undo_enabled=TRUE
primedr2.thread=2
primedr1.thread=1
primedr2.undo_tablespace=’UNDOTBS2′
primedr1.undo_tablespace=’UNDOTBS1′
[/code]

cp -ip pfile_for_standby.txt pfile_to_apply.ora

vi pfile_to_apply.ora

— Actual PFILE after modifying all parameters —

[code language=”sql”]
primedr1.__data_transfer_cache_size=0
primedr2.__data_transfer_cache_size=0
primedr1.__db_cache_size=7046430720
primedr2.__db_cache_size=7147094016
primedr1.__java_pool_size=67108864
primedr2.__java_pool_size=67108864
primedr1.__large_pool_size=301989888
primedr2.__large_pool_size=301989888
primedr2.__oracle_base=’/u03/app/oracle’#ORACLE_BASE set from environment
primedr1.__oracle_base=’/u03/app/oracle’#ORACLE_BASE set from environment
primedr1.__pga_aggregate_target=3221225472
primedr2.__pga_aggregate_target=3221225472
primedr1.__sga_target=9663676416
primedr2.__sga_target=9663676416
primedr1.__shared_io_pool_size=469762048
primedr2.__shared_io_pool_size=469762048
primedr1.__shared_pool_size=1644167168
primedr2.__shared_pool_size=1577058304
primedr1.__streams_pool_size=100663296
primedr2.__streams_pool_size=67108864
*.audit_file_dest=’/u03/app/oracle/admin/primeprd/adump’
*.audit_trail=’db’
*.cluster_database=TRUE
*.compatible=’12.1.0.2.0′
#*.control_files=’+DATAC1/PRIMEPRD/CONTROLFILE/current.660.914322763′,’+DATAC1/PRIMEPRD/CONTROLFILE/current.665.914322763′
*.cpu_count=16
*.db_block_size=8192
*.db_create_file_dest=’+DATAC1′
*.db_domain=”
*.db_file_name_convert=’PRIMEPRD’,’PRIMEDR’
*.db_unique_name=’primedr’
*.db_name=’primeprd’
*.db_recovery_file_dest=’+RECOC1′
*.db_recovery_file_dest_size=536870912000
*.db_writer_processes=6
*.ddl_lock_timeout=30
*.diagnostic_dest=’/u03/app/oracle’
*.dispatchers='(PROTOCOL=TCP) (SERVICE=primedrXDB)’
*.fal_client=’primedr’
*.fal_server=’primeprd’
primedr2.instance_number=2
primedr1.instance_number=1
*.log_archive_config=’dg_config=(primeprd,primedr)’
*.log_archive_dest_1=’LOCATION=+RECOC1 valid_for=(ALL_LOGFILES,ALL_ROLES) db_unique_name=primedr’
*.log_archive_dest_2=’service=primeprd LGWR ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=primeprd’
*.log_archive_dest_state_1=’ENABLE’
*.log_archive_dest_state_2=’ENABLE’
*.log_archive_format=’log%t_%s_%r.arc’
*.log_file_name_convert=’PRIMEPRD’,’PRIMEDR’
*.remote_listener=’exadr-scan:1521′
*.nls_length_semantics=’CHAR’
*.open_cursors=2000
*.optimizer_adaptive_features=FALSE
*.pga_aggregate_target=3072m
*.processes=1500
*.remote_login_passwordfile=’exclusive’
*.resource_manager_cpu_allocation=48
*.resource_manager_plan=’DEFAULT’
*.session_cached_cursors=100
*.sessions=2272
*.sga_max_size=17179869184
*.sga_target=9216m
*.standby_file_management=’AUTO’
*.temp_undo_enabled=TRUE
primedr2.thread=2
primedr1.thread=1
primedr2.undo_tablespace=’UNDOTBS2′
primedr1.undo_tablespace=’UNDOTBS1′
[/code]

6. Create same RMAN backup directory in DR and copy the backup Piece and init file from Production to DR server.

Also do not forget to create directory for adump in path ‘/u03/app/oracle/admin/primeprd/adump’

[code language=”sql”]
startup nomount pfile=’/u03/rmanbkp/prime/pfile_to_apply.ora’;
[/code]

[code language=”sql”]
rman target sys/*****@primeprd auxiliary /

Recovery Manager: Release 12.1.0.2.0 – Production on Tue Jun 28 10:05:31 2016

Copyright (c) 1982, 2014, Oracle and/or its affiliates. All rights reserved.

connected to target database: PRIMEPRD (DBID=111111111111111)
connected to auxiliary database: PRIMEPRD (not mounted)

show all;

using target database control file instead of recovery catalog
RMAN configuration parameters for database with db_unique_name PRIMEPRD are:
CONFIGURE RETENTION POLICY TO REDUNDANCY 1; # default
CONFIGURE BACKUP OPTIMIZATION OFF;
CONFIGURE DEFAULT DEVICE TYPE TO ‘SBT_TAPE’;
CONFIGURE CONTROLFILE AUTOBACKUP ON;
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO ‘/u03/rmanbkp/prime/autobackup_control_file_%F’;
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE SBT_TAPE TO ‘%F’; # default
CONFIGURE DEVICE TYPE SBT_TAPE PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default
CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE SBT_TAPE TO 1; # default
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE SBT_TAPE TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE CHANNEL DEVICE TYPE DISK FORMAT ‘/u03/rmanbkp/prime’;
CONFIGURE MAXSETSIZE TO UNLIMITED; # default
CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
CONFIGURE ENCRYPTION ALGORITHM ‘AES128’; # default
CONFIGURE COMPRESSION ALGORITHM ‘BASIC’ AS OF RELEASE ‘DEFAULT’ OPTIMIZE FOR LOAD TRUE ; # default
CONFIGURE RMAN OUTPUT TO KEEP FOR 7 DAYS; # default
CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
CONFIGURE SNAPSHOT CONTROLFILE NAME TO ‘/u03/app/oracle/product/12.1.0.2/payapps/dbs/snapcf_primeprd1.f’; # default

DUPLICATE TARGET DATABASE FOR STANDBY NOFILENAMECHECK;

[/code]

9. start the MRP process using the below command in SQL prompt.

[code language=”sql”]
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT;
[/code]

10. In case if you get error while receiving logging onto standby use the below command and create password files on both nodes of production and dr

— For Oracle 12c, do not create password files with this method. Go to section below for Dataguard Broker Setup

[code language=”sql”]
orapwd file=/u03/app/oracle/product/12.1.0.2/payapps/dbs/orapwprimeprd1 force=y ignorecase=y
orapwd file=/u03/app/oracle/product/12.1.0.2/payapps/dbs/orapwprimeprd2 force=y ignorecase=y
[/code]

[code language=”sql”]
orapwd file=/u03/app/oracle/product/12.1.0.2/payapps/dbs/orapwprimedr1 force=y ignorecase=y
orapwd file=/u03/app/oracle/product/12.1.0.2/payapps/dbs/orapwprimedr2 force=y ignorecase=y

select * from v$pwfile_users;
[/code]

Check by logging in from sqlplus from both production site and DR site from both nodes

From DR both nodes

[code language=”sql”]
sqlplus sys/*****@primeprd as sysdba
[/code]

From Prod Both nodes

[code language=”sql”]
sqlplus sys/******@primedr as sysdba
[/code]

Stop and start recovery again in DR

[code language=”sql”]
alter database recover managed standby database cancel;

alter database recover managed standby database using current logfile disconnect from session;

select process,status from v$managed_standby;

[/code]

11. Create the spfile and add the newly created control files

[code language=”sql”]
show parameter control_file

NAME TYPE VALUE
———————————— ———– ——————————
control_file_record_keep_time integer 7
control_files string +DATAC1/PRIMEDR/CONTROLFILE/current.636.915704919, +RECOC1/PRIMEDR/CONTROLFILE/current.15794.915704923

[/code]

Add the controlfile string in the pfile you created earlier called pfile_to_apply.ora

cd /u03/rmanbkp/prime/
vi pfile_to_apply.ora

*.control_files=’+DATAC1/PRIMEDR/CONTROLFILE/current.636.915704919′,’+RECOC1/PRIMEDR/CONTROLFILE/current.15794.915704923′

[code language=”sql”]
shutdown immediate;
ORA-01109: database not open

Database dismounted.
ORACLE instance shut down.

startup nomount pfile=’/u03/rmanbkp/prime/pfile_to_apply.ora’;

create spfile=’+DATAC1/primedr/spfileprimedr.ora’ from pfile=’/u03/rmanbkp/prime/pfile_to_apply.ora’;

shutdown immediate;
[/code]

cd /u03/app/oracle/product/12.1.0.2/payapps/dbs/

vi initprimedr1.ora

–add only one line pointing to the SPFILE in ASM

spfile=’+DATAC1/primedr/spfileprimedr.ora’

[code language=”sql”]
startup mount pfile=’/u03/app/oracle/product/12.1.0.2/payapps/dbs/initprimedr1.ora’;

show parameter spfile

NAME TYPE VALUE
———————————— ———– ——————————
spfile string +DATAC1/primedr/spfileprimedr.
ora

alter database recover managed standby database using current logfile disconnect from session;

[/code]

12. Create cluster server using the below command.

[code language=”bash”]
srvctl add database -d primedr -o /u03/app/oracle/product/12.1.0.2/payapps -p +DATAC1/primedr/spfileprimedr.ora -r physical_standby -a DATAC1,RECOC1 -s MOUNT
srvctl add instance -d primedr -i primedr1 -n exadrdbadm01
srvctl add instance -d primedr -i primedr2 -n exadrdbadm02

srvctl add service -db onlndr -s onlnprod -preferred onlndr1,onlndr2
srvctl add service -db primedr -s primeprod -preferred primedr1,primedr2

srvctl config service -s onlnprod -db onlndr
srvctl config service -s primeprod -db primedr

srvctl stop database -d primedr

— if gives error stop from sqlplus

srvctl start database -d primedr -o mount

[/code]

— Start Recovery Again —

[code language=”sql”]
alter database recover managed standby database using current logfile disconnect from session;
[/code]

13.1. Add the below entries in both production nodes tnsnames.ora file as a oracle user. (Not required from ORacle 12c 12.1.0.2 onwards)

primeprd_DGMGRL =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = ********)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = ********)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = primeprd_DGMGRL)
)
)

13.2. Add the below entries in both DR nodes tnsnames.ora file as a oracle user.

primedr_DGMGRL =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = ********)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = ********)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = primedr_DGMGRL)
)
)

13.3. Add the below entries in production node1 listener.ora file as a grid user.

(SID_DESC =
(GLOBAL_DBNAME = primeprd_DGMGRL)
(ORACLE_HOME = /u03/app/oracle/product/12.1.0.2/payapps)
(SID_NAME = primeprd1)
)

13.4. Add the below entries in production node2 listener.ora file as a grid user.

(SID_DESC =
(GLOBAL_DBNAME = primeprd_DGMGRL)
(ORACLE_HOME = /u03/app/oracle/product/12.1.0.2/payapps)
(SID_NAME = primeprd2)
)

13.5. Add the below entries in DR node1 listener.ora file as a grid user.

(SID_DESC =
(GLOBAL_DBNAME = primedr_DGMGRL)
(ORACLE_HOME = /u03/app/oracle/product/12.1.0.2/payapps)
(SID_NAME = primedr1)
)

13.6. Add the below entries in DR node2 listener.ora file as a grid user.

(SID_DESC =
(GLOBAL_DBNAME = primedr_DGMGRL)
(ORACLE_HOME = /u03/app/oracle/product/12.1.0.2/payapps)
(SID_NAME = primedr2)
)

13.7. Using the below command to restart the listener.

[code language=”sql”]
$ srvctl stop listener
$ srvctl start listener
[/code]

14.1. Set the below parameter in primary database.

[code language=”sql”]
alter system set dg_broker_config_file1 = ‘+DATAC1/PRIMEPRD/dr1primeprd.dat’ scope=both sid=’*’;
alter system set dg_broker_config_file2 = ‘+DATAC1/PRIMEPRD/dr2primeprd.dat’ scope=both sid=’*’;
alter system set dg_broker_start = true scope=both sid=’*’;
[/code]

14.2. Set the below parameter in standby database.

[code language=”sql”]
alter system set dg_broker_config_file1 = ‘+DATAC1/PRIMEDR/dr1primedr.dat’ scope=both sid=’*’;
alter system set dg_broker_config_file2 = ‘+DATAC1/PRIMEDR/dr2primedr.dat’ scope=both sid=’*’;
alter system set dg_broker_start = true scope=both sid=’*’;
[/code]

15.1. Configure the dataguard brocker using the below command from production database server

[code language=”bash”]
/home/oracle::11g- dgmgrl

connect sys/*****

create configuration dg_primeprd as primary database is primeprd connect identifier is primeprd;

add database primedr as connect identifier is primedr maintained as physical;

Configuration "dg_primeprd" created with primary database "primeprd"

add database primedr as connect identifier is primedr maintained as physical;
Error: ORA-16698: member has a LOG_ARCHIVE_DEST_n parameter with SERVICE attribute set

Failed.

remove configuration;
Removed configuration
[/code]

Set the log_archive_dest_2 settings from both the Primary and Standby databases to be nothing.

[code language=”sql”]
alter system set log_archive_dest_2=” scope=both sid=’*’;
[/code]

Disable then Enable the broker parameter on both the Primary and Standby databases.

— Primary
[code language=”sql”]
alter system set dg_broker_start=false scope=both sid=’*’;
alter system set dg_broker_start=true scope=both sid=’*’;
[/code]

— Standby
[code language=”sql”]
alter system set dg_broker_start=false scope=both sid=’*’;
alter system set dg_broker_start=true scope=both sid=’*’;
[/code]

On the Primary database create the broker configuration for the Primary and Standby database and this time it should work fine with no issues since the log archive destination 2 setting is not set, this is the workaround/solution.

[code language=”bash”]
/home/oracle::11g– dgmgrl

connect sys/*****

create configuration dg_primeprd as primary database is primeprd connect identifier is primeprd;

add database primedr as connect identifier is primedr maintained as physical;

enable configuration;
[/code]

Set back the log_archive_dest_2 on both production and dr

— In Production Database —

[code language=”sql”]
alter system set log_archive_dest_2=’service=primedr LGWR ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=primedr’ scope=both sid=’*’;
[/code]

— In DR Database —

[code language=”sql”]
alter system set log_archive_dest_2=’service=primeprd LGWR ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=primeprd’ scope=both sid=’*’;
[/code]

If in Broker you get below error

[code language=”bash”]
show configuration

Configuration – dg_primeprd

Protection Mode: MaxPerformance
Members:
primeprd – Primary database
primedr – Physical standby database
Error: ORA-16664: unable to receive the result from a database

Fast-Start Failover: DISABLED

Configuration Status:
ERROR (status updated 40 seconds ago)

Check in Broker Log File.

Mon Jul 04 14:41:46 2016
Error 1017 received logging on to the standby
————————————————————
Check that the primary and standby are using a password file
and remote_login_passwordfile is set to SHARED or EXCLUSIVE,
and that the SYS password is same in the password files.
returning error ORA-16191

Failed to send message to site . Error code is ORA-01017.

[/code]

Then re-create password files in ASM in 12c like below, the correct format is pwd

Primary

[code language=”bash”]
srvctl config database -d primeprd
orapwd file=’+DATAC1/PRIMEPRD/pwdprimeprd’ entries=10 dbuniquename=primeprd password=****** force=y ignorecase=y
[/code]

Standby

[code language=”bash”]
srvctl config database -d primedr
orapwd file=’+DATAC1/PRIMEDR/pwdprimedr’ entries=10 dbuniquename=primedr password=****** force=y ignorecase=y
[/code]

Check configuration now and the error should have vanished and the status is success

[code language=”bash”]
show configuration

Configuration – dg_primeprd

Protection Mode: MaxPerformance
Members:
primeprd – Primary database
primedr – Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 18 seconds ago)
[/code]

— Try doing Switchover using DGMGRL Now —

[code language=”bash”]
switchover to primedr
Performing switchover NOW, please wait…
Error: ORA-16644: apply instance not available

Failed.
Unable to switchover, primary database is still "primeprd"

edit database primedr set state=’APPLY-ON’ with apply instance=’primedr1′;

show database verbose primedr

Database – primedr

Role: PHYSICAL STANDBY
Intended State: APPLY-ON
Transport Lag: 0 seconds (computed 1 second ago)
Apply Lag: 0 seconds (computed 0 seconds ago)
Average Apply Rate: 6.00 KByte/s
Active Apply Rate: 755.00 KByte/s
Maximum Apply Rate: 827.00 KByte/s
Real Time Query: OFF
Instance(s):
primedr1 (apply instance)
primedr2

switchover to primedr
Performing switchover NOW, please wait…
New primary database "primedr" is opening…
Oracle Clusterware is restarting database "primeprd" …
Switchover succeeded, new primary is "primedr"

show configuration

Configuration – dg_primeprd

Protection Mode: MaxPerformance
Members:
primedr – Primary database
primeprd – Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:

SUCCESS (status updated 40 seconds ago)

edit database primeprd set state=’APPLY-ON’ with apply instance=’primeprd1′;

show database verbose primeprd

Database – primeprd

Role: PHYSICAL STANDBY
Intended State: APPLY-ON
Transport Lag: 0 seconds (computed 0 seconds ago)
Apply Lag: 0 seconds (computed 0 seconds ago)
Average Apply Rate: 307.00 KByte/s
Active Apply Rate: 0 Byte/s
Maximum Apply Rate: 0 Byte/s
Real Time Query: OFF
Instance(s):
primeprd1 (apply instance)
primeprd2

DGMGRL

show configuration

Configuration – dg_primeprd

Protection Mode: MaxPerformance
Members:
primedr – Primary database
primeprd – Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 55 seconds ago)

Connect to DR site broker to intiate switchover

switchover to primeprd ;
Performing switchover NOW, please wait…
Operation requires a connection to instance "primeprd1" on database "primeprd"
Connecting to instance "primeprd1"…
Connected as SYSDBA.
New primary database "primeprd" is opening…
Oracle Clusterware is restarting database "primedr" …
Switchover succeeded, new primary is "primeprd

show configuration

Configuration – dg_primeprd

Protection Mode: MaxPerformance
Members:
primeprd – Primary database
primedr – Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 11 seconds ago
[/code]

Oracle Dataguard Broker 12c with password file in ASM

Configure 12c dataguard broker using the below command onPrimary database server

primeprd – > tnsnames for primary database

primedr-> tnsnames for standby database

/home/oracle::11g–>dgmgrl

DGMGRL> connect sys/********

DGMGRL> create configuration dg_primeprd as primary database is primeprd connect identifier is primeprd;

DGMGRL> add database primedr as connect identifier is primedr maintained as physical;

Configuration “dg_primeprd” created with primary database “primeprd”

DGMGRL> add database primedr as connect identifier is primedr maintained as physical;
Error: ORA-16698: member has a LOG_ARCHIVE_DEST_n parameter with SERVICE attribute set

Failed.
DGMGRL> remove configuration;
Removed configuration
Set the log_archive_dest_2 settings from both the Primary and Standby databases to be nothing.

alter system set log_archive_dest_2=” scope=both sid=’*’;
Disable then Enable the broker parameter on both the Primary and Standby databases.

— Primary
alter system set dg_broker_start=false scope=both sid=’*’;
alter system set dg_broker_start=true scope=both sid=’*’;

— Standby
alter system set dg_broker_start=false scope=both sid=’*’;
alter system set dg_broker_start=true scope=both sid=’*’;

On the Primary database create the broker configuration for the Primary and Standby database and this time it should work fine with no issues since the log archive destination 2 setting is not set, this is the workaround/solution.

/home/oracle::11g–>dgmgrl

DGMGRL> connect sys/**********

DGMGRL> create configuration dg_primeprd as primary database is primeprd connect identifier is primeprd;

DGMGRL> add database primedr as connect identifier is primedr maintained as physical;

DGMGRL> enable configuration;
If in Broker you get below error Error: ORA-16664: unable to receive the result from a database

DGMGRL> show configuration

Configuration – dg_primeprd

Protection Mode: MaxPerformance
Members:
primeprd – Primary database
primedr – Physical standby database
Error: ORA-16664: unable to receive the result from a database

Fast-Start Failover: DISABLED

Configuration Status:
ERROR (status updated 40 seconds ago)

Check in Broker Log File.

Mon Jul 04 14:41:46 2016
Error 1017 received logging on to the standby
————————————————————
Check that the primary and standby are using a password file
and remote_login_passwordfile is set to SHARED or EXCLUSIVE,
and that the SYS password is same in the password files.
returning error ORA-16191

Failed to send message to site . Error code is ORA-01017.

Re-create password files in ASM in 12c like below, the correct format is pwd<dbuniquename>

Primary

srvctl config database -d primeprd

orapwd file=’+DATAC1/PRIMEPRD/pwdprimeprd’ entries=10 dbuniquename=primeprd password=*** force=y ignorecase=y
Standby

srvctl config database -d primedr
orapwd file=’+DATAC1/PRIMEDR/pwdprimedr’ entries=10 dbuniquename=primedr password=*** force=y ignorecase=y

DGMGRL> show configuration

Configuration – dg_primeprd

Protection Mode: MaxPerformance
Members:
primeprd – Primary database
primedr – Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 18 seconds ago)

— Try Switchover Now —
DGMGRL> switchover to primedr
Performing switchover NOW, please wait…
Error: ORA-16644: apply instance not available

Failed.
Unable to switchover, primary database is still “primeprd”

DGMGRL> edit database primedr set state=’APPLY-ON’ with apply instance=’primedr1′;

DGMGRL> show database verbose primedr

Database – primedr

Role: PHYSICAL STANDBY
Intended State: APPLY-ON
Transport Lag: 0 seconds (computed 1 second ago)
Apply Lag: 0 seconds (computed 0 seconds ago)
Average Apply Rate: 6.00 KByte/s
Active Apply Rate: 755.00 KByte/s
Maximum Apply Rate: 827.00 KByte/s
Real Time Query: OFF
Instance(s):
primedr1 (apply instance)
primedr2
DGMGRL> switchover to primedr
Performing switchover NOW, please wait…
New primary database “primedr” is opening…
Oracle Clusterware is restarting database “primeprd” …
Switchover succeeded, new primary is “primedr”
DGMGRL> show configuration

Configuration – dg_primeprd

Protection Mode: MaxPerformance
Members:
primedr – Primary database
primeprd – Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 40 seconds ago)
DGMGRL> edit database primeprd set state=’APPLY-ON’ with apply instance=’primeprd1′;

DGMGRL> show database verbose primeprd

Database – primeprd

Role: PHYSICAL STANDBY
Intended State: APPLY-ON
Transport Lag: 0 seconds (computed 0 seconds ago)
Apply Lag: 0 seconds (computed 0 seconds ago)
Average Apply Rate: 307.00 KByte/s
Active Apply Rate: 0 Byte/s
Maximum Apply Rate: 0 Byte/s
Real Time Query: OFF
Instance(s):
primeprd1 (apply instance)
primeprd2

DGMGRL>
DGMGRL> show configuration

Configuration – dg_primeprd

Protection Mode: MaxPerformance
Members:
primedr – Primary database
primeprd – Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 55 seconds ago)

Connect to DR site broker to intiate switchover

DGMGRL> switchover to primeprd ;
Performing switchover NOW, please wait…
Operation requires a connection to instance “primeprd1” on database “primeprd”
Connecting to instance “primeprd1″…
Connected as SYSDBA.
New primary database “primeprd” is opening…
Oracle Clusterware is restarting database “primedr” …
Switchover succeeded, new primary is “primeprd

DGMGRL> show configuration

Configuration – dg_primeprd

Protection Mode: MaxPerformance
Members:
primeprd – Primary database
primedr – Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS (status updated 11 seconds ago

Add Members for Online Relog File Group on RAC ASM

Let’s say you have already created logfiles from Group 1 to Group 8 on a 2 node RAC cluster on Diskgroup :+RECOC1. Now it is always highly recommended to mutiplex the relog files. This is how we will add new members to existing groups.

First let us see the commands to create the logfile groups on +RECOC1

— Add Redo LogFile’s Thread 1 —

alter database add logfile thread 1 group 1 (‘+RECOC1’) size 1G;

alter database add logfile thread 1 group 2 (‘+RECOC1’) size 1G;

alter database add logfile thread 1 group 3 (‘+RECOC1’) size 1G;

alter database add logfile thread 1 group 4 (‘+RECOC1’) size 1G;
— Add Redo LogFile’s Thread 2 —

alter database add logfile thread 2 group 5 (‘+RECOC1’) size 1G;

alter database add logfile thread 2 group 6 (‘+RECOC1’) size 1G;

alter database add logfile thread 2 group 7 (‘+RECOC1’) size 1G;

alter database add logfile thread 2 group 8 (‘+RECOC1’) size 1G;

 

Now we will add membersto all above created groups  in diskgroup +DATAC1

ALTER DATABASE
ADD LOGFILE MEMBER ‘+DATAC1’
TO GROUP 1;

ALTER DATABASE
ADD LOGFILE MEMBER ‘+DATAC1’
TO GROUP 2;

ALTER DATABASE
ADD LOGFILE MEMBER ‘+DATAC1’
TO GROUP 3;

ALTER DATABASE
ADD LOGFILE MEMBER ‘+DATAC1’
TO GROUP 4;

ALTER DATABASE
ADD LOGFILE MEMBER ‘+DATAC1’
TO GROUP 5;

ALTER DATABASE
ADD LOGFILE MEMBER ‘+DATAC1’
TO GROUP 6;

ALTER DATABASE
ADD LOGFILE MEMBER ‘+DATAC1’
TO GROUP 7;

ALTER DATABASE
ADD LOGFILE MEMBER ‘+DATAC1’
TO GROUP 8;

 

 

ACFS on Exadata 12c 12.1.0.2

ACFS stands for ASM Cluster filesystem, which is basically a cluster filesystem on top of ASM Diskgroups. It is similar to DBFS but kind of more elegant.

Until this momen it was not available on Exadata, but now ACFS is officially supported on Exadata from 12.1.0.2 onwards. To utilize this feature both your Grid Infrastructure and RDBMS home should be atleast on 12.1.0.2 on Exadata.

I just created a test ACFS filesystem on my X4-2 Quater Rack. And its amazing !

acfs

Move SPFILE to ASM RAC

— Move SPFILE to ASM RAC —

Already we have a PFILE called initmove.ora which was created previously from spfile

SQL> show parameter spfile

NAME TYPE VALUE
———————————— ———– ——————————
spfile string /u02/app/oracle/product/11.2.0
/dbhome_1/dbs/spfiletest1.ora

As you can see the spfile is in Filesystem

SQL> create spfile=’+DATAC1/test/spfiletest.ora’ from pfile=’/u03/rman/initmove.ora’;

File created.

SQL> exit

Create a pfile called inittest.ora and add only below line only in it ( format is init<sid>.ora )
spfile=’+DATAC1/test/spfiletest.ora’

SQL> shutdown immediate;

Now Startup database with this pfile and automatically it will point to the SPFILE which we created in the first step

SQL> startup nomount pfile=’/u03/rman/inittest.ora’;

SQL> show parameter spfile;

NAME TYPE VALUE
———————————— ———– ——————————
spfile string +DATAC1/eccdr/spfiletest.ora
SQL> exit

As you see now the SPFILE is in ASM disk group. Modify the database with SRVCTL incase you didnt add, add like below

srvctl modify database -d test-p +DATAC1/eccdr/spfiletest.ora

Copy File from Filesystem to ASM

In this exercise we will copy a file residing on a filesystem onto ASM disk.

[sourcecode language=”sql”]

SQL> create or replace directory asm_dir as ‘+FRA/BACKUP’;

Directory created.

SQL> grant read, write on directory asm_dir to public;

Grant succeeded.

SQL> create or replace directory export_dump as ‘/filesystem/exportdump’;

Directory created.

SQL> grant read,write on directory export_dump to public;

Grant succeeded.

[/sourcecode]

— Copy file from Filesystem into ASM —

[sourcecode language=”sql”]

SQL> exec dbms_file_transfer.COPY_FILE(‘export_dump’,’backup_dump.dmp’,’asm_dir’,’backup_dump.dmp’);

PL/SQL procedure successfully completed.

[/sourcecode]

Multiple Standby Databases Dataguard Oracle

We will create 2 standby databases from a single RAC primary database. 11gR2 allows upto 30 standby databases.  The most important part is to consider the dataguard parameters, the rest is a normal dataguard creation. The important parameters are as below :

db_file_name_convert

log_archive_config

log_archive_dest_1

log_archive_dest_2

log_archive_dest_3

LOG_ARCHIVE_DEST_STATE_1

LOG_ARCHIVE_DEST_STATE_2

LOG_ARCHIVE_DEST_STATE_3

log_file_name_convert

remote_login_passwordfile

STANDBY_FILE_MANAGEMENT

fal_client

fal_server

sec_case_sensitive_logon    (if on 11gR2)
All database are running on Grid + ASM 11.2.0.3.4 with RAC primary and RAC standby’s. Please note the procedure is identical to a single instance setup. In Rac the only special thing you do is register the standby database and instances to the OCR via srvctl command.

We all call our primary database SWX and our PHYSICAL STANDBY databases as SWXSTANDBY, SWXSTANDYBY2. We will use RMAN Duplicate Active Standby database feature of 11gR2. If your standby databases are in different data centesr and the bandwidht isn’t suffcient you can use the tradionatl RMAN  duplicate command.

1. Create tnsnames.ora entry on Primary database. Use RDBMS owner for the tnsnames.ora not Grid infrastructure owner

SWX =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = *.*.*.213)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = swx1)(UR=A)
)
)

SWXSTANDBY =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = *.*.*.219)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = swxstandby)(UR=A)
)
)

SWXSTANDBY2 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST =  *.*.*.224)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = swxstandby2)(UR=A)
)

2. Add an SID entry on the listener.ora as Grid infrastructure user

SID_LIST_EASYORADBA_LISTENER=(SID_LIST=(SID_DESC=(SID_NAME=swx1)(GLOBAL_DBNAME=swx)(ORACLE_HOME=/grid/app/11.2.0/grid/)))

3. Create a pfile and the below datagaurd parameters to the pfile

#########################STANDBY PARAMETERS########################
*.DB_UNIQUE_NAME=’swx’
*.db_file_name_convert=’+DATA/SWXSTANDBY2/’,’+DATA/SWXSTANDBY/’,’+DATA/SWX/’,’+FRA/SWXSTANDBY2/’,’+FRA/SWXSTANDBY/’,’+FRA/SWX/’
*.log_archive_config=’dg_config=(swxstandby2,swxstandby,swx)’
*.log_archive_dest_1=’LOCATION=+FRA valid_for=(ALL_LOGFILES,ALL_ROLES) db_unique_name=swx’
*.log_archive_dest_2=’service=swxstandby LGWR ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=swxstandby’
*.log_archive_dest_3=’service=swxstandby2 LGWR ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=swxstandby2′
*.LOG_ARCHIVE_DEST_STATE_1=’ENABLE’
*.LOG_ARCHIVE_DEST_STATE_2=’ENABLE’
*.LOG_ARCHIVE_DEST_STATE_3=’ENABLE’
*.LOG_ARCHIVE_FORMAT=’log%t_%s_%r.arc’
*.log_file_name_convert=’+DATA/SWXSTANDBY2/’,’+DATA/SWXSTANDBY/’,’+DATA/SWX/’,’+FRA/SWXSTANDBY2/’,’+FRA/SWXSTANDBY/’,’+FRA/SWX/’
*.remote_login_passwordfile=’exclusive’
*.STANDBY_FILE_MANAGEMENT=’AUTO’
*.fal_client=’SWX’
*.fal_server=’SWXSTANDBY2′,’SWXSTANDBY’
*.sec_case_sensitive_logon=FALSE
########################################

4. Restart the database with the new pfile and create spfile from it and bounce the database again

5. Login to the First DR database server as Oracle user and create tnsnames.ora entry

SWX =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = *.*.*.213)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = swx)(UR=A)
)
)

SWXSTANDBY =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = *.*.*.218)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = swxstandby)(UR=A)
)
)

SWXSTANDBY2 =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = *.*.*.224)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = swxstandby2)(UR=A)
)
)

6. Add SID listener entry in listener.ora file as Grid infrastructure user.

SID_LIST_EASYORADBA_LISTENER=(SID_LIST=(SID_DESC=(SID_NAME=swx1)(GLOBAL_DBNAME=swx)(ORACLE_HOME=/grid/app/11.2.0/grid)))

7. Create a pfile with the dataguard parameters from the pfile created before on primary and add below parameters.

#########################STANDBY DATABASE PARAMETERS########################
*.DB_UNIQUE_NAME=’SWXSTANDBY’
*.db_file_name_convert=’+DATA/SWX/’,’+DATA/SWXSTANDBY/’,’+DATA/SWXSTANDBY2/’,’+FRA/SWX/’,’+FRA/SWXSTANDBY/’,’+FRA/SWXSTANDBY2/’
*.log_archive_config=’dg_config=(swxstandby2,swxstandby,swx)’
*.log_archive_dest_1=’LOCATION=+FRA valid_for=(ALL_LOGFILES,ALL_ROLES) db_unique_name=swxstandby’
*.log_archive_dest_2=’service=swx LGWR ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=swx’
*.log_archive_dest_3=’service=swxstandby2 LGWR ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=swxstandby2′
*.LOG_ARCHIVE_DEST_STATE_1=’ENABLE’
*.LOG_ARCHIVE_DEST_STATE_2=’ENABLE’
*.LOG_ARCHIVE_DEST_STATE_3=’ENABLE’
*.LOG_ARCHIVE_FORMAT=’log%t_%s_%r.arc’
*.log_file_name_convert=’+DATA/SWX/’,’+DATA/SWXSTANDBY/’,’+DATA/SWXSTANDBY2/’,’+FRA/SWX/’,’+FRA/SWXSTANDBY/’,’+FRA/SWXSTANDBY2/’
*.remote_login_passwordfile=’exclusive’
*.STANDBY_FILE_MANAGEMENT=’AUTO’
*.fal_client=’SWXSTANDBY’
*.fal_server=’SWX’,’SWXSTANDBY2′
*.sec_case_sensitive_logon=FALSE
########################################

8. Start the database with the pfile and create spfile from it

9. Clone the database from RMAN using active option

rman target sys/***@swx auxiliary sys/***@swxstandby

RMAN> duplicate target database for standby from active database nofilenamecheck dorecover;

10. Register the database and instance with srvctl command

srvctl add database -d SWXSTANDBY -o /oracle/app/oracle/product/11.2.0/dbhome_1 -c RAC -r PHYSICAL_STANDBY -a DATA,FRA
srvctl add instance -d SWXSTANDBY -i swx1 -n csftest-2
srvctl stop database -d SWXSTANDBY
sqlplus “/as sysdba”
srvctl start database -d swxstandby
srvctl modify database -d swxstandby -s mount
srvctl config database -d swxstandby

## TO ACTIVATE ACTIVE STANDBY DATABASE ###
srvctl start database -d swxstandby -o open

11. Login to the Second DR database server as Oracle user and create tnsnames.ora entry

SWX =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = *.*.*.213)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = swx)(UR=A)
)
)

SWXSTANDBY2 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = *.*.*.15)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = SWXSTANDBY2)(UR=A)
)
)

SWXSTANDBY =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = *.*.*.218)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = SWXSTANDBY)(UR=A)
)
)

12. Create a listener.ora entry as grid user

SID_LIST_EASYORADBA_LISTENER=(SID_LIST=(SID_DESC=(SID_NAME=swx1)(GLOBAL_DBNAME=swx)(ORACLE_HOME=/grid/app/11.2.0/grid)))

13. Create a pfile with the dataguard parameters from the pfile created before on primary and add below parameters.

#########################STANDBY DATABASE PARAMETERS########################
*.DB_UNIQUE_NAME=’SWXSTANDBY2′
*.db_file_name_convert=’+DATA/SWX/’,’+DATA/SWXSTANDBY/’,’+DATA/SWXSTANDBY2/’,’+FRA/SWX/’,’+FRA/SWXSTANDBY/’,’+FRA/SWXSTANDBY2/’
*.log_archive_config=’dg_config=(swxstandby2,swxstandby,swx)’
*.log_archive_dest_1=’LOCATION=+FRA valid_for=(ALL_LOGFILES,ALL_ROLES) db_unique_name=swxstandby2′
*.log_archive_dest_2=’service=swx LGWR ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=swx’
*.log_archive_dest_3=’service=swxstandby LGWR ASYNC valid_for=(ONLINE_LOGFILE,PRIMARY_ROLE) db_unique_name=swxstandby’
*.LOG_ARCHIVE_DEST_STATE_1=’ENABLE’
*.LOG_ARCHIVE_DEST_STATE_2=’ENABLE’
*.LOG_ARCHIVE_DEST_STATE_3=’ENABLE’
*.LOG_ARCHIVE_FORMAT=’log%t_%s_%r.arc’
*.log_file_name_convert=’+DATA/SWX/’,’+DATA/SWXSTANDBY/’,’+DATA/SWXSTANDBY2/’,’+FRA/SWX/’,’+FRA/SWXSTANDBY/’,’+FRA/SWXSTANDBY2/’
*.remote_login_passwordfile=’exclusive’
*.STANDBY_FILE_MANAGEMENT=’AUTO’
*.fal_client=’SWXSTANDBY2′
*.fal_server=’SWX’,’SWXSTANDBY’
*.sec_case_sensitive_logon=FALSE
########################################

14. Start the database with the pfile and create spfile from it

15. Clone the database from RMAN using active option

rman target sys/***@swx auxiliary sys/***@swxstandby

RMAN> duplicate target database for standby from active database nofilenamecheck dorecover;

16. Register the database and instance with srvctl command

srvctl add database -d swxstandby2 -o /oracle/app/oracle/product/11.2.0/dbhome_1 -c RAC -r PHYSICAL_STANDBY -n swx -a DATA,FRA
srvctl add instance -d swxstandby2 -i swx1 -n suncsftst

Now that we  have  a fully functional RAC PRIMARY with 2 PHYSICAL STANDBYs now let us consider the 2 most important things in Dataguard: Switchover and Failover

————–

Switchover

—————-

No extra steps are necessary when performing a switchover. All bystander physical
standby databases automatically apply redo data received from the new primary
database.

————–

Failover

—————-

The steps for performing a failover to a physical standby database depend on the
Redo Apply progress of the new primary database and any bystander physical
standby databases at the time of the failover.
• If the new primary database has applied more redo than all of the bystander
physical standby databases, no additional steps are required. Only the original
primary database needs to be reinstated, using the steps documented in Oracle
Data Guard Concepts and Administration[5], Section 12.4.1 “Flashing Back a Failed
Primary Database into a Physical Standby Database.”.
• If any bystander physical standby database has applied more redo than the new
primary database, then perform the following steps to reinstate the bystander
physical standby

SQL*Plus Physical/Physical Failover with Physical Bystander Ahead

1. Determine STANDBY_BECAME_PRIMARY_SCN from the new
primary.
SQL> select STANDBY_BECAME_PRIMARY_SCN from
v$database;
2. On the bystander physical standby, flash back to
STANDBY_BECAME_PRIMARY_SCN from the new primary
database.
SQL> flashback database to scn
<STANDBY_BECAME_PRIMARY_SCN>;
3. On the bystander physical standby, delete divergent archived redo logs
created at the time of, or after, the failover.
RMAN> delete archivelog from scn
<STANDBY_BECAME_PRIMARY_SCN>;
4. On the new primary database, enable the redo transport destination
for this bystander physical standby and archive the current redo log.
SQL> alter system set
log_archive_destination_2=enable;
SQL> alter system archive log current;
5. After the logs have been received by the bystander physical standby,
start Redo Apply on the bystander physical standby.

SQL> alter database recover managed standby
database using current logfile through all
switchover disconnect;
The bystander standby database is now reinstated.

For more information refer to this excellent Oracle document.

http://www.oracle.com/technetwork/database/features/availability/maa10gr2multiplestandbybp-1-131937.pdf

Interesting Case-Study on Multiple Standby Databases. Apparently Apple uses it for one of their Large databases.

http://www.oracle.com/technetwork/database/features/availability/311400-134359.pdf

ORA-29701: unable to connect to Cluster Synchronization Service

Problem
——
Error on starting ASM from SQLPLUS or ASMCMD

SQL> startup;
ORA-01078: failure in processing system parameters
ORA-29701: unable to connect to Cluster Synchronization Service
SQL> exit
Disconnected

Solution
——–

As Grid User

$ crsctl start res ora.cssd
CRS-2672: Attempting to start ‘ora.cssd’ on ‘prmdb’
CRS-2672: Attempting to start ‘ora.diskmon’ on ‘prmdb’
CRS-2676: Start of ‘ora.diskmon’ on ‘prmdb’ succeeded

$ asmcmd
Connected to an idle instance.
ASMCMD> startup
ORA-00099: warning: no parameter file specified for ASM instance

ASM instance started

Total System Global Area 317333504 bytes
Fixed Size 2221120 bytes
Variable Size 289946560 bytes
ASM Cache 25165824 bytes
ORA-15110: no diskgroups mounted

$ sqlplus “/as sysasm”

SQL*Plus: Release 11.2.0.3.0 Production on Thu Nov 15 12:53:09 2012

Copyright (c) 1982, 2011, Oracle. All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 – 64bit Production
With the Automatic Storage Management option

SQL> select name,state from v$asm_diskgroup;

NAME STATE
—————————— ———–
DATA MOUNTED