Monday, May 11, 2020

Relocated PDBs in Oracle Database 18c

Oracle 18c allows RMAN backups created before the non-CDB or PDB was migrated into a different target CDB can be used for recovery operations. The COMPATIBLE parameter of the source and Oracle Cloud must be set to 18.0.0 or higher

The RMAN commands used to backup and recovery CDBs and PDBs are the same as those used for non-CDBs, with few variations in the syntax.

The backup and recovery operations performed on non-CDBs can also be performed on CDBs and PDBs. This includes the following:
– Full and incremental backups
– Complete and point-in-time recovery (PITR)
– Reporting operations (such as listing and cross-checking backups)
– Flashback Database

We need take metadata for the existing backups and available to the destination CDB. To export metadata user needs to run DBMS_PDB.EXPORTRMANBACKUP procedure on the source database.

EXECUTE DBMS_PDB.exportrmanbackup();
Or
EXECUTE DBMS_PDB.exportrmanbackup('EMP_PDB');

In case if you are unplug you no need to run this command as unplug already includes the metadata.

Convert NON-CDB to PDB : As we have metadata, now we can covert the NON-CDB instance to PDB.

Step1: Open the non-CDB instance in read-only mode and describe and shutdown

SQL> SHUTDOWN IMMEDIATE;
SQL> STARTUP OPEN READ ONLY;
SQL> BEGIN
DBMS_PDB.DESCRIBE( pdb_descr_file => '/oracle/empdb.xml');
END;
SQL> SHUTDOWN IMMEDIATE;

Step2: Create the new pluggable database using the non-CDB description file that we have taken in above step

SQL> CREATE PLUGGABLE DATABASE empdb_pdb USING '/oracle/empdb.xml' COPY;
SQL> ALTER SESSION SET CONTAINER= empdb_pdb;
SQL> @$ORACLE_HOME/rdbms/admin/noncdb_to_pdb.sql
SQL> ALTER PLUGGABLE DATABASE OPEN;
SQL> ALTER PLUGGABLE DATABASE SAVE STATE;

Step3:- Restore and recovery using pre-plugin backup

SQL> ALTER PLUGGABLE DATABASE empdb_pdb CLOSE IMMEDIATE;

RMAN> SET PREPLUGIN CONTAINER=db18cpdb;
RMAN> RESTORE PLUGGABLE DATABASE empdb_pdb FROM PREPLUGIN;
RMAN> RECOVER PLUGGABLE DATABASE empdb_pdb FROM PREPLUGIN;

Sometimes users may come across RMAN-06054 error in that case you need to CATALOG the missing archive log and start the recovery again.

RMAN-06054: media recovery requesting unknown archived log for thread 3 sequence 94983
RMAN>SET PREPLUGIN CONTAINER= empdb_pdb;
RMAN>CATALOG PREPLUGIN ARCHIVELOG 'oracle/archivelog/arc_empdb_pdb_3_94983.arc';
RMAN>RECOVER PLUGGABLE DATABASE empdb_pdb FROM PREPLUGIN;

Perform normal recovery and open the database

RMAN>RECOVER PLUGGABLE DATABASE empdb_pdb;
RMAN>ALTER PLUGGABLE DATABASE empdb_pdb OPEN;


Reference:
Check preplugin backups available to the CDB instance
RMAN> LIST PREPLUGIN BACKUP OF PLUGGABLE DATABASE empdb_pdb;

Preplugin backups are usable only on the destination CDB into which you plug in the source non-CDB or PDB

Regards
Satishbabu Gunukula, Oracle ACE
http://www.oracleracexpert.com

Sunday, May 3, 2020

How to Register or Catalog Backups, Archive logs, data file copy, FRA...etc

When you backup using RMAN all backups  will be registered with catalog. If you have taken backup without catalog and you can also register manually.

Backup pieces copied manually to different location to catalog backup piece run below command.
RMAN> CATALOG BACKUPPIECE /oracle/backup/DB1_1978087_6446005710.bkp' ;

In case if you have taken backup using o/s command and to catalog a datafile copy use below command.
RMAN> CATALOG DATAFILECOPY '/oracle/backup/employee01.bak' LEVEL 0;

In case, archive log location full and you have moved backups to different location using o/s command then you can register the archive logs using below command

RMAN> CATALOG ARCHIVELOG ‘/oracle/backup2/db1_arch_99089_0876589.arc'
If you are using Flash recovery area and want to catalog all files in the FRA use below command.
RMAN> CATALOG recovery area;

This command will prompt to confirm that you want the found files to be cataloged. If you do not want to prompt use “noprompt” clause.
RMAN> CATALOG recovery area noprompt;

If you want to catalog only files Start with specific name or belongs to specific SID (db1) use below command
RMAN> CATALOG START WITH '/oracle/backup/db1'

If you run CROSSCHECK, it will go through the files that match criteria and will be marked as available, and files not found will be marked as EXPIRED.

RMAN> CROSSCHECK BACKUP ;
If you want to DELETE EXPIRED backups you can use below command, note that EXPIRED copy will no longer appear in RMAN. You have to be extra cautious to delete backups.

RMAN> DELETE EXPIRED BACKUP ;

If you find extra copies and want to delete those extra copies use DELETE BACKUPPIECE command:
RMAN> DELETE BACKUPPIECE 4555;
Hope this helps managing your backups taken outside of RMAN recovery catalog .

Thanks & Regards 
Satishbabu Gunukula, Oracle ACE
http://oracleracexpert.com



Wednesday, April 29, 2020

ORA-19815: WARNING: db_recovery_file_dest_size of 128849010 bytes is 99.03% used, and has 14136528 remaining bytes available.

You will receive ORA-19815 is reaching maximum size of flash backup recovery area (FRA) and note that this limitation will be managed by db_recovery_file_dest_size parameter

If you got this error as one time then you might be running adhoc load or some batch job. Also you will get below message in alert.log file. Please follow one of the choice to fix the issue.

ORA-19815: WARNING: db_recovery_file_dest_size of 128849010 bytes is 99.03% used, and has 14136528 remaining bytes available.
2020-02-21T11:31:19.892420-07:00

************************************************************************
You have following choices to free up space from recovery area:
1. Consider changing RMAN RETENTION POLICY. If you are using Data Guard,
then consider changing RMAN ARCHIVELOG DELETION POLICY.
2. Back up files to tertiary device such as tape using RMAN
BACKUP RECOVERY AREA command.
3. Add disk space and increase db_recovery_file_dest_size parameter to
reflect the new space.
4. Delete unnecessary files using RMAN DELETE command. If an operating
system command was used to delete files, then use RMAN CROSSCHECK and
DELETE EXPIRED commands.

In case if you archive logs are going to FRA then you should run RMAN backups and delete the unnecessary files to avoid Database hung or CRASH situation.

RMAN> backup archivelog all delete input;

If the Flash recovery area is 100% then you might receive below error when starting up the database

Database mounted.
ORA-03113: end-of-file on communication channel
Process ID: 9789
Session ID: 321 Serial number: 3


Pls make sure you follow below steps to add more space to flash recovery area (FRA)

SQL> startup mount;
SQL> alter system set db_recovery_file_dest_size=<size>G scope=both;
Replace <size> with a number larger than the current and ensure is sufficient space in the destination
SQL> alter database open;


To see what FRA has run below command

SQL> select * from v$flash_recovery_area_usage;
FILE_TYPE            PERCENT_SPACE_USED PERCENT_SPACE_RECLAIMABLE NUMBER_OF_FILES     CON_ID
-------------------- ------------------ ------------------------- --------------- ----------
CONTROL FILE                          0                         0               0          0
REDO LOG                              0                         0               0          0
ARCHIVED LOG                          0                         0               0          0
BACKUP PIECE                          0                         0               0          0
IMAGE COPY                            0                         0               0          0
FLASHBACK LOG                     99.76                         0             192          0
FOREIGN ARCHIVED LOG                  0                         0               0          0
AUXILIARY DATAFILE C                  0                         0               0          0

In my case I am using for FLASHBACK LOG, these logs are different from redo logs or undo. They are a separate, optional recording of database activity.

These logs are automatically deleted when FRA is our of space and logs is not needed for recvoery within the window specified in DB_FLASHBACK_RETENTION_TARGET parameter

To determine the disk quota and current disk usage in the flash recovery area, query the view
V$RECOVERY_FILE_DEST.

Thanks & Regards,
Satishbabu Gunukula, Oracle ACE
http://www.oracleracexpert.com

Friday, April 17, 2020

When using Data Guard broker Error: ORA-16627: operation disallowed since no member would remain to support protection mode

You will encounter this error when PROTECTION mode is not consistent betweeen PRIMARY and STANDBY. Make sure you are using DGMGRL to update the configurationn.

DGMGRL> show configuration;
Configuration - oradb_dg_fsfconf
Protection Mode: MaxPerformance
Members:
oradb - Primary database
oradb_stdby - Physical standby database

Fast-Start Failover: Disabled

Configuration Status:
SUCCESS (status updated 13 seconds ago)

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS MAXAVAILABILITY;

Error: ORA-16627: operation disallowed since no member would remain to support protection mode Failed.
=============

Make sure you verify the entire configuration using verbose.

DGMGRL> show configuration verbose;
Configuration - oradb_dg_fsfconf
Protection Mode: MaxPerformance
Members:
oradb - Primary database
oradb_stdby - Physical standby database

Properties:
FastStartFailoverThreshold = '30'
OperationTimeout = '30'
TraceLevel = 'USER'
FastStartFailoverLagLimit = '30'
CommunicationTimeout = '180'
ObserverReconnect = '0'
FastStartFailoverAutoReinstate = 'TRUE'
FastStartFailoverPmyShutdown = 'TRUE'
BystandersFollowRoleChange = 'ALL'
bserverOverride = 'FALSE'
ExternalDestination1 = ''
ExternalDestination2 = ''
PrimaryLostWriteAction = 'CONTINUE'
ConfigurationWideServiceName = 'ORADB_CFG'

Fast-Start Failover: Disabled

Configuration Status:
SUCCESS

DGMGRL> show database verbose ORADB;
Database - oradb
Role: PRIMARY
Intended State: TRANSPORT-ON
Instance(s):
ORADB

Properties:
DGConnectIdentifier = 'oradb'
ObserverConnectIdentifier = ''
FastStartFailoverTarget = ''
PreferredObserverHosts = ''
LogShipping = 'ON'
RedoRoutes = ''
LogXptMode = 'ASYNC'
DelayMins = '0'
Binding = 'optional'
MaxFailure = '0'
ReopenSecs = '300'
NetTimeout = '30'
RedoCompression = 'DISABLE'
PreferredApplyInstance = ''
ApplyInstanceTimeout = '0'
ApplyLagThreshold = '30'
TransportLagThreshold = '30'
TransportDisconnectedThreshold = '30'
ApplyParallel = 'AUTO'
ApplyInstances = '0'
StandbyFileManagement = ''
ArchiveLagTarget = '0'
LogArchiveMaxProcesses = '0'
LogArchiveMinSucceedDest = '0'
DataGuardSyncLatency = '0'
LogArchiveTrace = '0'
LogArchiveFormat = ''
DbFileNameConvert = ''
LogFileNameConvert = ''
ArchiveLocation = ''
AlternateLocation = ''
StandbyArchiveLocation = ''
StandbyAlternateLocation = ''
InconsistentProperties = '(monitor)'
InconsistentLogXptProps = '(monitor)'
LogXptStatus = '(monitor)'
SendQEntries = '(monitor)'
RecvQEntries = '(monitor)'
HostName = 'scllqa0078'
StaticConnectIdentifier = '(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=seroradb-db)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ORADB_DGMGRL)(INSTANCE_NAME=ORADB)(SERVER=DEDICATED)))'
TopWaitEvents = '(monitor)'
SidName = '(monitor)'

Log file locations:
Alert log : /home/oracle/diag/rdbms/oradb/ORADB/trace/alert_ORADB.log
Data Guard Broker log : /home/oracle/diag/rdbms/oradb/ORADB/trace/drcORADB.log

Database Status:
SUCCESS
DGMGRL> show database verbose ORADB_STDBY;

Database - oradb_stdby
Role: PHYSICAL STANDBY
Intended State: APPLY-ON
Transport Lag: 0 seconds (computed 1 second ago)
Apply Lag: 0 seconds (computed 1 second ago)
Average Apply Rate: 26.00 KByte/s
Active Apply Rate: 938.00 KByte/s
Maximum Apply Rate: 940.00 KByte/s
Real Time Query: OFF
Instance(s):
ORADB

Properties:
DGConnectIdentifier = 'oradb_stdby'
ObserverConnectIdentifier = ''
FastStartFailoverTarget = ''
PreferredObserverHosts = ''
LogShipping = 'ON'
RedoRoutes = ''
LogXptMode = 'ASYNC'
DelayMins = '0'
Binding = 'optional'
MaxFailure = '0'
ReopenSecs = '300'
NetTimeout = '30'
RedoCompression = 'DISABLE'
PreferredApplyInstance = ''
ApplyInstanceTimeout = '0'
ApplyLagThreshold = '30'
TransportLagThreshold = '30'
TransportDisconnectedThreshold = '30'
ApplyParallel = 'AUTO'
ApplyInstances = '0'
StandbyFileManagement = ''
ArchiveLagTarget = '0'
LogArchiveMaxProcesses = '0'
LogArchiveMinSucceedDest = '0'
DataGuardSyncLatency = '0'
LogArchiveTrace = '0'
LogArchiveFormat = ''
DbFileNameConvert = ''
LogFileNameConvert = ''
ArchiveLocation = ''
AlternateLocation = ''
StandbyArchiveLocation = ''
StandbyAlternateLocation = ''
InconsistentProperties = '(monitor)'
InconsistentLogXptProps = '(monitor)'
LogXptStatus = '(monitor)'
SendQEntries = '(monitor)'
RecvQEntries = '(monitor)'
HostName = 'scllqa0120'
StaticConnectIdentifier = '(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oraoradb-dg)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ORADB_STDBY_DGMGRL)(INSTANCE_NAME=ORADB)(SERVER=DEDICATED)))'
TopWaitEvents = '(monitor)'
SidName = '(monitor)'

Log file locations:
Alert log : /home/oracle/diag/rdbms/oradb_stdby/ORADB/trace/alert_ORADB.log
Data Guard Broker log : /home/oracle/diag/rdbms/oradb_stdby/ORADB/trace/drcORADB.log

Database Status:
SUCCESS

Make sure you update the LogXptMode and set the protection mode.

DGMGRL> EDIT DATABASE ORADB SET PROPERTY 'LogXptMode'='SYNC';
Property "LogXptMode" updated
DGMGRL> EDIT DATABASE ORADB_STDBY SET PROPERTY 'LogXptMode'='SYNC';
Property "LogXptMode" updated

DGMGRL> show configuration;
Configuration - oradb_dg_fsfconf

Protection Mode: MaxPerformance
Members:
oradb - Primary database
oradb_stdby - Physical standby database

Fast-Start Failover: Disabled

Configuration Status:
SUCCESS (status updated 32 seconds ago)

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS MAXAVAILABILITY;
Succeeded.

DGMGRL> show configuration;

Configuration - oradb_dg_fsfconf
Protection Mode: MaxAvailability
Members:
oradb - Primary database
oradb_stdby - Physical standby database

Fast-Start Failover: Disabled

Configuration Status:
SUCCESS (status updated 57 seconds ago)
Regards,
Satishbabu Gunukula, Oracle ACE

Wednesday, April 8, 2020

ORA-00230: operation disallowed: snapshot control file enqueue unavailable

User might encounter below error, in case if RMAN backup is already running and user might have triggered another backup. The second backup must wait control file enqueuer to create a snapshot of the controlfile.

RMAN-03002: failure of backup command at 04/10/2020 16:40:33
ORA-00230: operation disallowed: snapshot control file enqueue unavailable

You can run below command to find out the backup that is running and causing the wait

SELECT S.SID, USERNAME , PROGRAM, MODULE,
ACTION, LOGON_TIME FROM V$SESSION s, V$ENQUEUE_LOCK l
WHERE l.SID = s.SID AND l.TYPE = 'CF' AND l.ID1 = 0 AND l.ID2 = 2;

To avoid this error make sure that backups are not overlapping each other.

Regards,
Satishbabu Gunukula, Oracle ACE
http://oracleracexpert.com