Authentication enhancements in Oracle 23c

Oracle 23c offers longer passwords improved security in authentication now supports up to 1024 bytes

• Oracle Data pump Export and import support longer encryption passwords up to 2024 bytes long

• Oracle Call interface (OCI) and Oracle C++ Call interface support up to 1024 bytes long password for user authentication.
• JDBC think driver support up to 1024 characters for password
• Oracle Database (including Autonomous) and clients supports password up to 1024 bytes

You can login into Oracle Database using Microsoft Azure Active Directory single sing-on OAuth2 access token. Multicloud feature integrates Oracle Database and Azure AD and you can perform this integration on

• Oracle 19.16 and later (Back ported) but not for Oracle 21c.
• Oracle Autonomous Database on Dedicated/Shared Exadata Infrastructure
• Oracle Exadata Clod Service
• Oracle Base Database Service

You can map AD users to Oracle Database schema and roles and also you can login ODP.NET can login into Oracle Database Using Microsoft Azure Active Directory. The UTL_HTTP support SHA-256/512 and XDB HTTP supports SHA512, authentication and updated Kerberos Library support.

The password length helps accommodating Oracle Identity Access management (IAM) and Identity Cloud service (IDCS) and helps enabling uniform password rules.

Oracle 23c offers many Improvements in Kerberos security and MIT Kerberos version 1.20.1 supports cross domain, windows credential guard and multiple principals.

KERBEROS5_CC_NAME and KERBEROS5_PRINCIPAL can be specified in tnsnames.ora and the values must match for user authentication.

kuser =
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=orahost)(PORT=1521)) (CONNECT_DATA=(SERVICE_NAME=ORCL))
(SECURITY=(KERBEROS5_CC_NAME = /tmp/kuser/krb.cc) (KERBEROS5_PRINCIPAL = kprinc)))

Kerberos parameters can be specified in Sqlnet.ora file but note that some parameters you can set at server level, and some are at client level and few you can set on both.

You can set below parameters on both client and server

SQLNET.AUTHENTICATION_SERVICES=(KERBEROS5)

SQLNET.AUTHENTICATION_KERBEROS5_SERVICE=oracle

SQLNET.KERBEROS5_CONF=<Kerberos_configfile_path >

SQLNET.KERBEROS5_CONF_MIT=(TRUE)

SQLNET.FALLBACK_AUTHENTICATION=FALSE

SQLNET.KERBEROS5_CLOCKSKEW=1200

The below parameter is not required on the server, but in case if your client in Microsoft Windows then you may want to consider setting OSMSFT:// or MSLSA

SQLNET.KERBEROS5_CC_NAME= <Kerberos_CC_name_withpath>

This setting is not usually required for the client or the server.

SQLNET.KERBEROS5_REALMS=<Kerberos_realms_path >

Only set this parameter on the server

SQLNET.KERBEROS5_KEYTAB=<Kerberos_keytab_path > 

Thanks & Regards,

https://oracleracexpert.com

ORA-29861: domain index is marked LOADING/FAILED/UNUSABLE

When users performing the transactions on tables where DOMAIN,SPATIAL indexes are part of it they may receive below error and unable to proceed.

SQL> DELETE FROM USER_URL where CXT_ID=8484884;
ERROR at line 1: ORA-29861: domain index is marked LOADING/FAILED/UNUSABLE

The main reason for this error is user might have copied the table/index using export/import or different method but the CREATE INDEX and failed.

Find out what caused the failure of CREATE INDEX. In many cases not having enough space on the Tablespace where you are creating the index is the culprit.

SQL> SELECT INDEX_NAME, TABLE_NAME STATUS,DOMIDX_STATUS,DOMIDX_OPSTATUS FROM DBA_INDEXES WHERE DOMIDX_OPSTATUS='FAILED';

INDEX_NAME     TABLE_NAME STATUS   DOMIDX_STATUS  DOMIDX_OPSTATUS
------------------ --------------- ---------- --------------------- -------------------------  
USER_URL_IDX  USER_URL      VALID    VALID                    FAILED


When verified I found that schema refreshed recently, and index not created successfully.

ORA-31693: Table data object "FIN"."USER_URL" failed to load/unload and is being skipped due to error:
ORA-29861: domain index is marked LOADING/FAILED/UNUSABLE

If you need workaround to proceed with transactions immediately, then drop the index.
The permanent fix will be drop and recreate index successfully.

To avoid this type of issues, make sure you perform following checks.

1. Before refresh verify the tablespace or file systems has enough space and add required space
2. After import check the data pump or import log file and fix all failures
3. Run utlrp.sql to recompile all invalid objects.

Thanks & Regards

https://oracleracexpert.com

ORA-02304: invalid object identifier literal and ORA-39083 errors during impdp

Users might receive below error during the import.

ORA-39083: Object type TYPE:"ORCL"."OID_TAB1" failed to create with error:
ORA-02304: invalid object identifier literal

Failing sql is:
CREATE EDITIONABLE TYPE "ORCL"."OID_TAB1" OID 'FB3CF41KK327B011D053F114ABBC878C' AS OBJECT (
className1 VARCHAR2(200),
id1 NUMBER)

Run below query in the database and check for TPYE_ODI exists or not

 
SQL> select OWNER, TYPE_NAME from DBA_TYPES where TYPE_OID='FB3CF41KK327B011D053F114ABBC878C ';

OWNER TYPE_OID
------------------------------------------ --------------------------------
ORCL      FB3CF41KK327B011D053F114ABBC878C

In most cases you will see the ODI is already exists in the database. The ODI already exists, and you cannot create ODI with same name that’s why it failed. You need to follow one of the options.

1. Create the object with new ODI – You can remove the ODI clause from the statement and run again. The database will generate new ODI for the object.

2. Drop the existing TYPE_ODI and reimport - DROP the object and reimport so that object will create with same ODI

3. In “impdp” use the parameter transform=oid:n - Using the parameter during the impdp will import TYPES with new ODI.

I have used all 3 methods based upon different scenarios.

Thanks & Regards,

http://oracleracexpert.com

Oracle 19c import issues ORA-31693, ORA-02354, ORA-39002, ORA-39405

When performing import come across the following issue

ORA-31693: Table data object "ORCL"."ITEMCG" failed to load/unload and is being skipped due to error:
ORA-02354: error in exporting/importing data
ORA-39840: A data load operation has detected data stream format error .
ORA-39844: Bad stream format detected: [klaprs_62] [139751105749101] [139751105749012] [4] [2] [2065583] [] []

User may encounter this issue when new column added to a table with cokumn optimization enabled and the same column was modified.

Below are the workarounds

• Use access_method=EXTERNAL_TABLE during export

• Prior export add and drop a dummy column to the problematic table using

SQL> ALTER TABLE <table_name> ADD dummy number;
SQL> ALTER TABLE <table_name> DROP dummy number;

• Take export of failed table and import using CONTENT=DATA_ONLY as tableau structure already imported.

The fix for this bug was initially available on 19.13 and above see if it helps.

Also, I come across below issues when importing data

ORA-39002: invalid operation
ORA-39405: Oracle Data Pump does not support importing from a source database with TSTZ version 33 into a target database with TSTZ version 32.

User may encounter this issue when export from higher time zone version and importing into lower time zone version.

• Patch the target database to higher or equal to source time zone patch or DST TZ version
• Create a database with same time zone and perform export/import

For latest DST patches refer,

• Oracle RDBMS and OJVM DST-related notes
• Updated DST Transitions and New Time Zones in Oracle RDBMS and OJVM Time Zone File Patches (Doc ID 412160.1)

Hope this helps,

Thanks & Regards

https://oracleracexpert.com

BlockChain Tables in Oracle 21c

Blockchain tables are designed to allow insert operations only, updates and any modifications are not allowed, and delete operations are restricted. The rows are organized into chains by storing previous row’s hash values in the current row and chain of rows is verifiable by all participants.

Oracle 19c introduced Immutable tables concept, that provides protection against unauthorized data modifications. The blockchain tables concept introduced in Oracle 21c and can be backported to 19c using a patch 32431413, but COMPATIBLE parameter must be set to 19.10.0 or late.

These tables are useful to implement Blockchain applications to handle tamper-resistant blockchain transactions with verifiable crypto-secure data management practices. The blockchain tables prevent unauthorized changes or deletion by criminals, hackers and fraud and protect critical company data. Blockchain tables has hidden columns as well and these values are managed by the database.

Blockchain tables and regular tables can be used in queries and transactions, also you can create indexes and partitions.

Blockchain tables hidden columns

Column Name	Data Type	Description
ORABCTAB_INST_ID$	NUMBER (22)	Instance ID of the database instance into which the row is inserted.
ORABCTAB_CHAIN_ID$	NUMBER (22)	Chain ID of the chain, in the database instance, into which the row is inserted.  0 through 31 are valid values.
ORABCTAB_SEQ_NUM$	NUMBER(22)	Sequence number of the row on the chain
ORABCTAB_CREATION_TIME$	TIMESTAMP WITH TIME ZONE	Row created time in UTC format
ORABCTAB_USER_NUMBER$	NUMBER (22)	Database User ID who inserted the row.
ORABCTAB_HASH$	RAW(2000)	Hash value of the row
ORABCTAB_SIGNATURE$	RAW(2000)	User signature of the row
ORABCTAB_SIGNATURE_ALG$	NUMBER(22)	Signature algorithm used to produce the user signature of a signed row.
ORABCTAB_SIGNATURE_CERT$	RAW(16)	GUID of the certificate associated with the signature on a signed row.
ORABCTAB_SPARE$	RAW(2000)	Reserved for future use.

Important Guidelines for Blockchain Tables

• In case of Oracle RAC instance, a block chain table contains 32 chains and chain will have unique combination of instance ID and chain ID. It is recommended to create index on the combination of Instance ID, chain ID and sequence number.

• The SHA2-512 hashing algorithm used to handle hash value

• In case Oracle Data Guard, to avoid data loss consider using Maximum availability or Maximum protection mode

• To specify retention period for Blockchain table use “NO DROP” Clause in the CREATE BLOCKCHAIN TABLE statement to specify retention period.

• To specify retention period for rows in Blockchain table use “NO DELETE” Clause in the CREATE BLOCKCHAIN TABLE statement to specify retention period for rows

Restrictions for Blockchain tables

• There are many restrictions when using blockchain tables

• Many datatypes are not supported such as nested table , varray, REF , ROWID, UROWID, LONG, object type, BFILE, XMLType , , , TIMESTAMP WITH TIME ZONE, TIMESTAMP WITH LOCAL TIME ZONE

• Point-in-time recovery or flashback database will undo changes on all database objects including blockchain tables.

• Blockchain Retention policies depends on system time and any changes to systems time must be audited.

• Max number of user column allowed are 980

• Blockchain tables doesn’t allow or support below operations

Column level restrictions

o Adding, dropping, and renaming columns

o Dropping partitions

Row level restrictions

o Update or merge rows

o Defining BEFORE ROW triggers that fire for update operations are not allowed.

Table level restrictions

o Truncate table

o Inserting data using parallel DML

o Sharded tables

o During distributed transactions, Inserting data into a blockchain table using Active Data Guard DML redirection is not supported

o Direct-path loading

o Flashback table

o Cannot convert a regular table to a blockchain table or vice versa.

o XA transactions

Database level restrictions

o Export and Import can be done as regular tables, without the system-generated hidden columns.

o Creating blockchain tables in CDB or application root

o Creating Oracle Label Security (OLS) policies

o Using the DBMS_REDEFINITION package for Online redefinition

o Creating Oracle Virtual Private Database (VPD) policies

o When using Transient Logical Standby and rolling upgrades, the DDL and DML on blockchain tables are not replicated and supported

o when using Logical Standby and Oracle GoldenGate, the DDL and DML on blockchain tables succeed on the primary database but are not replicated to standby databases

o Creating Automatic Data Optimization (ADO) policies

Thanks & Regards

https://oracleracexpert.com, Oracle ACE

Webinar: Oracle Data pump enhancements in Oracle 21c

This Webinar helps you to understand Oracle data pump new features in Oracle 21c and take an advantage of these features.

Date and time: Dec 21st 2022 8:00am-9:00am
Pacific Daylight Time (San Francisco, GMT-07:00)

This Webinar covers following Topics.

• Introduction to Oracle Data Pump
• CHECKSUM, CHECKSUM_ALGORITHM
• VERIFY_ONLY and VERIFY_CHECKSUM
• INCLUDE and EXCLUDE in the Same Operation
• Index Compression
• Transportable Tablespace Enhancements
• JSON Data Type Support
• Export/Import from Cloud Object Store
• Q&A

To register for this Webinar, please send an email to SatishbabuGunukula@gmail.com

Please use Zoom Link to join the Webinar.  

Note that registrations are limited and first come and first serve basis.
You will receive an email confirmation with meeting session link.

For presentation link Click here 

Thanks & Regards,
http://www.oracleracexpert.com

DBMS_CLOUD Package in Oracle 21c

In order to work with data in object stores you can either use DBMS_CLOUD package or manually define external table. The package was DBMS_CLOUD package introduced in Oracle Autonomous Database to work with an object store, we can also use in on-prem for versions 19c and 21c but you need to install manually. You can refer Oracle note How To Setup And Use DBMS_CLOUD Package (Doc ID 2748362.1) for setup and other details.

You can run below command to create directory, grant access on directory and DBMS_CLOUD package to emp_usr user.

CREATE OR REPLACE DIRECTORY EMP_DIR AS '/oracle/emp_dir';

GRANT EXECUTE ON DBMS_CLOUD TO emp_usr;

GRANT READ, WRITE ON DIRECTORY emp_dir TO emp_usr, C##CLOUD$SERVICE;

Note DBMS_CLOUD is owned by a separate schema C##CLOUD$SERVICE, this user locked by default. Both emp_usr, C##CLOUD$SERVICE user needs to have access.

Credential - You can create credential using the CREATE_CREDENTIAL procedure for your object store.

GRANT CREATE CREDENTIAL TO emp_usr;

BEGIN

DBMS_CLOUD.CREATE_CREDENTIAL (

CREDENTIAL_NAME => 'cloud_obj_cred',

USERNAME => '<username>,

PASSWORD => '<user auth token>'

) ;

END;

/

You can use UPDATE_CREDENTIALS procedure to make changes to the credentials

BEGIN

DBMS_CREDENTIAL.UPDATE_CREDENTIAL(

CREDENTIAL_NAME => 'cloud_obj_cred',

ATTRIBUTE => 'username',

VALUE => 'emp_usr');

END;

/

To drop credential you can use DROP_CREDENTIAL procedure.

BEGIN

DBMS_CLOUD.DROP_CREDENTIAL(CREDENTIAL_NAME => 'cloud_obj_cred’);

END;

/

You can disable and enable the credentials using DISABLE_CREDENTIAL and ENABLE_CREDENTIAL

BEGIN

DBMS_CREDENTIAL.DISABLE_CREDENTIAL('cloud_obj_cred’);

END;

/

BEGIN

DBMS_CREDENTIAL.ENABLE_CREDENTIAL('cloud_obj_cred’);

END;

/

Object store - For managing the objects in the cloud object store, please use below commands

To transfer a file from direct to cloud object store use the PUT_OBJECT procedure

BEGIN

DBMS_CLOUD.PUT_OBJECT (

CREDENTIAL_NAME => 'cloud_obj_cred’,

OBJECT_URI => 'https://xxxxxxxxxxxx/emp_file.txt',

DIRECTORY_NAME => 'emp_dir',

FILE_NAME => 'emp_file.txt');

END;

/

In order to transfer a object from cloud object store to the directory use the GET_OBJECT procedure

BEGIN

DBMS_CLOUD.GET_OBJECT (

CREDENTIAL_NAME => 'cloud_obj_cred’,

OBJECT_URI => 'https://xxxxxxxxxxxx/emp_file.txt',

DIRECTORY_NAME => 'emp_dir',

FILE_NAME => 'emp_file.txt');

END;

/

You can use dbms_cloud.list_objects to lists objects in the object store and use GET_METADATA function In order to get specific object metadata

SELECT * FROM DBMS_CLOUD.LIST_OBJECTS(CREDENTIAL_NAME => 'cloud_obj_cred’,LOCATION_URI => 'https://xxxxxxxxxxx');

SELECT * FROM DBMS_CLOUD.GET_METADATA(CREDENTIAL_NAME => 'cloud_obj_cred’,LOCATION_URI => 'https://xxxxxxxxxxx/emp_file.txt') AS METADATA FROM DUAL;

You can delete objects from the cloud object store using DELETE_OBJECT procedure

 

BEGIN

DBMS_CLOUD.DELETE_OBJECT(

CREDENTIAL_NAME => 'cloud_obj_cred’,

OBJECT_URI => 'https://xxxxxxxx/emp_file.txt');

END;

/

You can delete files from the directory using DELETE_FILE procedure

 

BEGIN

DBMS_CLOUD.DELETE_FILE(

DIRECTORY_NAME => 'emp_dir',

FILE_NAME => 'emp_file.txt');

END;

/

To list the files in a directory use LIST_FILES function

SELECT * FROM DBMS_CLOUD.LIST_FILES(DIRECTORY_NAME => 'emp_dir');

To export data into a cloud object store in the required format use EXPORT_DATA procedure

BEGIN

DBMS_CLOUD.EXPORT_DATA (

CREDENTIAL_NAME => 'cloud_obj_cred’,

FILE_URI_LIST => 'https://xxxxxxxxxxx/emp_file.csv',

QUERY => 'select * from emp',

FORMAT => '{"TYPE" : "csv"}');

END;

/

To delete all operation use DELETE_ALL_OPERATIONS procedure

BEGIN

DBMS_CLOUD.DELETE_ALL_OPERATIONS;

END;

/

To delete specific operation use DELETE_OPERATIONS procedure , but you need to provide operation ID

BEGIN

DBMS_CLOUD.DELETE_OPERATION(<id>);

END;

/

If you enter problems with DBMS_CLOUD with the user or role you can test the configuration using the same sample code that was used for the DBMS_CLOUD setup .

Thanks & Regards,

http://oracleracexpert.com, Oracle ACE  

Oracle Data pump import stuck Processing object type DATABASE_EXPORT/SCHEMA

You might come across issues when importing data using DataPump. I have faced the issue several times while importing Domain Index and other objects. You can encounter DOMAIN INDEX issue  related to CTXSYS schema. Note that data import may complete quickly but import gets stuck when creating DOMAIN INDEXES or INDEX and it may still run even after few days. 

Processing object type DATABASE_EXPORT/SCHEMA/TABLE/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type DATABASE_EXPORT/SCHEMA/TABLE/COMMENT
Processing object type DATABASE_EXPORT/SCHEMA/PACKAGE/PACKAGE_SPEC
Processing object type DATABASE_EXPORT/SCHEMA/PACKAGE/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type DATABASE_EXPORT/SCHEMA/PACKAGE/CODE_BASE_GRANT
Processing object type DATABASE_EXPORT/SCHEMA/FUNCTION/FUNCTION
Processing object type DATABASE_EXPORT/SCHEMA/FUNCTION/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type DATABASE_EXPORT/SCHEMA/PROCEDURE/PROCEDURE
Processing object type DATABASE_EXPORT/SCHEMA/PACKAGE/COMPILE_PACKAGE/PACKAGE_SPEC/ALTER_PACKAGE_SPEC
Processing object type DATABASE_EXPORT/SCHEMA/FUNCTION/ALTER_FUNCTION
Processing object type DATABASE_EXPORT/SCHEMA/PROCEDURE/ALTER_PROCEDURE
Processing object type DATABASE_EXPORT/SCHEMA/VIEW/VIEW
Processing object type DATABASE_EXPORT/SCHEMA/VIEW/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type DATABASE_EXPORT/SCHEMA/PACKAGE_BODIES/PACKAGE/PACKAGE_BODY
Processing object type DATABASE_EXPORT/SCHEMA/TABLE/INDEX/INDEX
Processing object type DATABASE_EXPORT/SCHEMA/TABLE/INDEX/FUNCTIONAL_INDEX/INDEX
Processing object type DATABASE_EXPORT/SCHEMA/TABLE/CONSTRAINT/CONSTRAINT
Processing object type DATABASE_EXPORT/SCHEMA/TABLE/CONSTRAINT/REF_CONSTRAINT
Processing object type DATABASE_EXPORT/SCHEMA/TABLE/INDEX/DOMAIN_INDEX/INDEX

Pls note that domain or normal index rebuild may take time based upon the index size during the import and also other reasons may effect the operation. Here are few reasons that you need to look …

1. Did you gather Stats before the import – If yes, exclude STATS and manually gather stats after import completed.

2. Make sure all tablespaces have enough space – Sometimes imports get stuck due to not having enough space

3. Make sure you have enough STREAMS_POOL_SIZE - The value should be at least 512M or more

If you are still facing the issue then you need to run below commands to identify the SQL Statement A Data Pump Process Is Executing

a) Find out the the datapump import job running or not using below SQL

SQL> select owner_name, job_name, operation, job_mode, from dba_datapump_jobs where state='EXECUTING' ;

You can run below command to identify the session used by datapump job.

SQL> select owner_name, job_name, session_type from dba_datapump_sessions;

b)  If the job is still running on step 4 then Identify The Current SQL Statement A Data Pump Process Is Executing (refer Oracle support Doc ID 1528301.1) and identify the object

After you have tried all the options if the index creating is still taking time then the last option to EXCLUDE the object taking time and manually create after import operation is successful…

You can get the object DDL command using below query

SQL> select dbms_metadata.get_ddl('INDEX','<Index_Name>','<Schema_Name>') from dual;

or 

SQL> select dbms_metadata.get_ddl('TABLE','<Table_Name>','<Schema_Name') from dual;

You can exclude the index by adding below clause in the import command…

Exclude= INDEX:"LIKE ‘Index_name _that got stuck_%'"

After import is successful you can create the object manually in my case it is index…

To monitor Data Pump jobs query views DBA_DATAPUMP_JOBS AND DBA_DATAPUMP_SESSIONS. You can also query V$SESSION_LONGOPS to see the progress of data pump job.

The below script very useful to identify database role, version, registry status, patch level and Invalid objects. I would highly suggest to run this script for any maintenance activity you perform on a Database.

SET PAGESIZE 2000
SET LINESIZE 500
COL OBJECT_NAME FORMAT A30
COL OBJECT_TYPE FORMAT A30
COL COMP_ID FORMAT A10

COL COMP_NAME FORMAT A45  
COL OWNER FORMAT A15
COL STATUS FORMAT A10

COL VERSION FORMAT A10  
/* Database Role and Version */
select NAME, PLATFORM_ID, DATABASE_ROLE from v$database;
select * from V$version where banner like 'Oracle Database%';
/* Database Component Registry status */
select comp_id, comp_name, status, version from dba_registry;
/* Database patch Level*/
select * from dba_registry_history;
/* INVALID objects in the DB count, by type & in detail */
select count(*) "INVALID Objects Count" from dba_objects where status !='VALID';
select owner, object_type, count(*) from dba_objects where status !='VALID' group by owner, object_type order by owner, object_type;
select owner, object_type, object_name, status from dba_objects where status !='VALID' order by owner, object_type, object_name;

Hope this helps

Regards,

http://oracleracexpert.com, Oracle ACE

Oracle Data pump enhancements in Oracle 21c

Oracle 21c offering new enhancements for Oracle data pump and take an advantage of these features

• CHECKSUM, CHECKSUM_ALGORITHM
• VERIFY_ONLY and VERIFY_CHECKSUM
• INCLUDE and EXCLUDE in the Same Operation
• Index Compression
• Transportable Tablespace Enhancements
• JSON Data Type Support

CHECKSUM, CHECKSUM_ALGORITHM
These parameters enables the export to perform checksum validation for each of the dump files, you can enable using CHECKSUM_ALGORITHM or CHECKSUM. In order to use these the COMPATIBLE parameter must be set to 21.0 or higher

$ expdp \'/ AS SYSDBA\' dumpfile=test.dmp schemas=testuser DIRECTORY=test_dir LOGFILE=test.log CHECKSUM=YES checksum_algorithm=SHA256

VERIFY_ONLY and VERIFY_CHECKSUM
The VERIFY_ONLY , VERIFY_CHECKSUM parameter uses the checksum to validate dump files during the import. The VERIFY_CHECKSUM and VERIFY_ONLY parameters are mutually exclusive

$ impdp \'/ AS SYSDBA\' dumpfile=test.dmp DIRECTORY=test_dir LOGFILE=test.log verify_checksum=yes

INCLUDE and EXCLUDE in the Same Operation
From Oracle 21c, the Include and exclude objects within the same export or import job

$ expdp \'/ AS SYSDBA\' dumpfile=test.dmp schemas=testuser DIRECTORY=test_dir LOGFILE=test.log include=table exclude=statistics

Index Compression
In Oracle database 21c, we can compress indexes while importing using the INDEX_COMPRESSION_CLAUSE and the TRANSFORM parameter

$ impdp \'/ AS SYSDBA\' dumpfile=test.dmp FULL=Y DIRECTORY=test_dir TRANSFORM=TABLE_COMPRESSION_CLAUSE:\"COMPRESS BASIC\" TRANSFORM=INDEX_COMPRESSION_CLAUSE:\"COMPRESS ADVANCED LOW\" EXCLUDE=CONSTRAINT

Master table "SYS"."SYS_IMPORT_FULL_01" successfully loaded/unloaded Starting "SYS"."SYS_EXPORT_TRANSPORTABLE_01": "/******** AS SYSDBA" FULL=Y DIRECTORY=test_dir TRANSFORM=TABLE_COMPRESSION_CLAUSE:\"COMPRESS BASIC\" TRANSFORM=INDEX_COMPRESSION_CLAUSE:\"COMPRESS ADVANCED LOW\" EXCLUDE=CONSTRAINT
Processing object type TABLE_EXPORT/TABLE/TABLE
Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
. . imported "TEST"."EMP" 188.08 KB 107 rows
. . imported "TEST"."DEPT" 90.50 KB 90 rows
. . imported "TEST"."SAL" 40.50 KB 38 rows
Processing object type TABLE_EXPORT/TABLE/COMMENT
Processing object type TABLE_EXPORT/TABLE/INDEX/INDEX
Processing object type TABLE_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type TABLE_EXPORT/TABLE/TRIGGER
Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
Processing object type TABLE_EXPORT/TABLE/STATISTICS/MARKER
Job "SYS"."SYS_IMPORT_FULL_01" successfully completed at Fri Dec 10 09:22:27 2021 elapsed 0 00:00:32

Transportable Tablespace Enhancements
Before Oracle 21c any failure you cannot able to resume transportable tablespace jobs , but now Oracle Data Pump Resumes Transportable Tablespace Jobs and also Parallelizes Metadata Operations using PARALLEL parameter

$ expdp \'/ AS SYSDBA\' dumpfile=test.dmp TRANSPORT_TABLESPACES=testusr DIRECTORY=test_dir TRANSPORT_FULL_CHECK=YES LOGFILE=test.log REUSE_DUMPFILES=YES PARALLEL=2

Starting "SYS"."SYS_EXPORT_TRANSPORTABLE_01": "/******** AS SYSDBA" dumpfile=test.dmp TRANSPORT_TABLESPACES= testusr TRANSPORT_FULL_CHECK=YES LOGFILE=test.log REUSE_DUMPFILES=YES PARALLEL=2
ORA-39396: Warning: exporting encrypted data using transportable option without password
Processing object type TRANSPORTABLE_EXPORT/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type TRANSPORTABLE_EXPORT/INDEX/STATISTICS/BITMAP_INDEX/INDEX_STATISTICS Processing object type TRANSPORTABLE_EXPORT/STATISTICS/TABLE_STATISTICS Processing object type TRANSPORTABLE_EXPORT/PLUGTS_BLK
Processing object type TRANSPORTABLE_EXPORT/STATISTICS/MARKER
Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PLUGTS_BLK Processing object type TRANSPORTABLE_EXPORT/TABLE
Processing object type TRANSPORTABLE_EXPORT/INDEX/INDEX
Processing object type TRANSPORTABLE_EXPORT/COMMENT
Processing object type TRANSPORTABLE_EXPORT/CONSTRAINT/CONSTRAINT
Processing object type TRANSPORTABLE_EXPORT/CONSTRAINT/REF_CONSTRAINT Processing object type TRANSPORTABLE_EXPORT/TRIGGER
Processing object type TRANSPORTABLE_EXPORT/INDEX/BITMAP_INDEX/INDEX Processing object type TRANSPORTABLE_EXPORT/INDEX/DOMAIN_INDEX/SECONDARY_TABLE/INDEX/INDEX Processing object type TRANSPORTABLE_EXPORT/INDEX/DOMAIN_INDEX/SECONDARY_TABLE/TABLE
Processing object type TRANSPORTABLE_EXPORT/INDEX/DOMAIN_INDEX/SECONDARY_TABLE/CONSTRAINT
Processing object type TRANSPORTABLE_EXPORT/INDEX/DOMAIN_INDEX/INDEX Processing object type TRANSPORTABLE_EXPORT/MATERIALIZED_VIEW Master table "SYS"."SYS_EXPORT_TRANSPORTABLE_01" successfully loaded/unloaded

JSON Data Type Support
Oracle Data Pump enables export and import of Oracle Database native JSON objects, We can also export and import of tables containing the JSON Datatype in full, tablespaces and table modes using Transportable Tablespace

Oracle 21c also offers export/import from Oracle autonomous Database and Cloud Object stores

Thanks & Regards,

http://oracleracexpert.com, Oracle ACE

Convert Partitioned Table to Non-partitioned Table and vice versa

You might come across the situation that you need to convert Partitioned Table to Non-partitioned Table and vice versa.

You can RUN below command to identify weather the Table is partitioned or not

SQL> SELECT TABLE_NAME, PARTITIONED FROM USER_TABLES WHERE TABLE_NAME=‘EMP’;

TABLE_NAME PAR
--------------------- ---
EMP YES

I suggest using data pump to convert Partitioned Table to Non-partitioned Table. Below are the steps
Take a backup of the table that you want to convert to NON-Partitioned table

$ expdp SCOTT/xxxxx directory=EXP_DIR dumpfile=emp_expdp.dmp log=emp_expdp.log tables=EMP

Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
Processing object type TABLE_EXPORT/TABLE/STATISTICS/MARKER
Processing object type TABLE_EXPORT/TABLE/TABLE
. . exported "SCOTT"."EMP":"YEAR_2001" 15.398 MB 40023 rows
. . exported "SCOTT"."EMP":"YEAR_2002" 23.456 MB 89898 rows
. . exported "SCOTT"."EMP":"YEAR_2003" 89.675 MB 100453 rows
Master table "SCOTT"."SYS_EXPORT_TABLE_01" successfully loaded/unloaded

Import the table into TEST schema using PARTITIONS_OPTIONS parameter

$ impdp TEST/xxxxx directory=EXP_DIR dumpfile=emp_expdp.dmp log=imp_emp.log remap_schema=SCOTT:TEST partition_options=merge

Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
. . imported “TEST"."EMP":"YEAR_2001" 15.398 MB 40023 rows
. . imported "TEST"."EMP":"YEAR_2002" 23.456 MB 89898 rows
. . imported "TEST"."EMP":"YEAR_2003" 89.675 MB 100453 rows
Processing object type TABLE_EXPORT/TABLE/STATISTICS/MARKER
Job "TEST"."SYS_IMPORT_FULL_01" successfully completed at Sun July 5 08:44:21 2020 elapsed 0 00:00:10

This command will merge all partitions while importing as single table.

RUN below command to verify and you should see the out as
SQL> SELECT TABLE_NAME, PARTITIONED FROM USER_TABLES WHERE TABLE_NAME=‘EMP’;

TABLE_NAME PAR
---------------------- ---
EMP NO

DROP PARTATION

If you need to drop the partition use below command

Method 1: Drop the specific partition
SQL> ALTER TABLE EMP DROP PARTITION YEAR_2001;

Method 2: DELETE all rows part of partition and remove partition
SQL> DELETE FROM EMP PARTITION (YEAR_2001;);
SQL> ALTER TABLE EMP DROP PARTITION YEAR_2001;

Method 3:- Drop partition and update indexes
SQL> ALTER TABLE EMP DROP PARTITION YEAR_2001 UPDATE INDEXES;

ADD/MODFY partition

You can use ALTER TABLE ADD PARTITION, MODIFY PARTITION statement on a table

For ex:-
SQL>ALTER TABLE EMP MODIFY PARTITION BY RANGE (JOIN_DATE)
(PARTITION YEAR_2001 VALUES LESS THAN (to_date('01-JAN-2002','dd-mon-yyyy')),
PARTITION YEAR_2002 VALUES LESS THAN (to_date('01-JAN-2003','dd-mon-yyyy')));

SQL> ALTER TABLE EMP ADD PARTITION YEAR_2003 VALUES LESS THAN ( '01-JAN-2004' ) ;

Please refer oracle documentation for detailed PARTITIONS options and methods..etc

Thanks

http://oracleracexpert.com, Oracle ACE

Generate Table, Index or tablespace DDLs in oracle

When migrating database from one server to another you need Tablespace creation DDL. Also it is very useful when copying specific table or index or exp/imp from one database to databases.

Here are few ways to generate the tablespace script from source database.

SQL>SET ECHO OFF;
SQL>SET HEADING OFF;
SQL>SET LINESIZE 1000;
SQL> SET LONG 60000;
SQL> SET FEEDBACK OFF;
SQL>SPOOL TBS_DDL.SQL
SQL>SELECT DBMS_METADATA.GET_DDL('TABLESPACE',DBA_TABLESPACES.TABLESPACE_NAME) FROM DBA_TABLESPACES;
SQL>SPOOL OFF

You can see below command if you want the TABLESPACE creation script specific to one Table space.

SQL> SELECT DBMS_METADATA.GET_DDL('TABLESPACE','&TABLESPACE_NAME') FROM dual;

You can use below command to generate specific TABLE, INDEX Script from a user

Syntax:-
select dbms_metadata.get_ddl('TABLE','<TABEL NAME>','<SCHEMA>') from dual;
select dbms_metadata.get_ddl('INDEX','<INDEX NAME>','<SCHEMA>') from dual;

Ex:-
select dbms_metadata.get_ddl('TABLE','EMP_SAL','EMP') from dual;
select dbms_metadata.get_ddl('INDEX','EMPNO_IDX','EMP') from dual;

You can use below command to all TABLE, INDEX Script from a user. First, connect to user

SQL>SET ECHO OFF;
SQL>SET HEADING OFF;
SQL>SET LINESIZE 1000;
SQL> SET LONG 60000;
SQL> SET FEEDBACK OFF;

SQL> SELECT DBMS_METADATA.GET_DDL('TABLE',U.TABLE_NAME)

FROM USER_TABLES U;

SQL> SELECT DBMS_METADATA.GET_DDL('INDEX',U.INDEX_NAME)

FROM USER_INDEXES U;  

In case if you need a View DDL, you can run below command;
SQL>select DBMS_METADATA.GET_DDL('VIEW',<View_Name>’) from dual;
or
SQL>select DBMS_METADATA.GET_DDL('VIEW','<view_name>','<schema_name>') from DUAL;

You can run below command to generate the DDL statements of a SCHEMA
set pages 30000
set linesize 1000
set lines 500

SQL> SELECT DBMS_METADATA.GET_DDL('USER','<schema_name>') FROM dual;

You can run below command to generate DDL statements for more than one schema

SQL> SELECT DBMS_METADATA.GET_DDL('USER',U.USERNAME) FROM DBA_USERS U WHERE USERNAME IN ('USER1','USER2');

You can run below command to generate DDL statement of the System Grant/ role granted/object granted to a schema owner

SQL> SELECT DBMS_METADATA.GET_GRANTED_DDL('SYSTEM_GRANT','<schema_name>') from dual;
SQL> SELECT DBMS_METADATA.GET_GRANTED_DDL('ROLE_GRANT','<schema_name>') from dual;
SQL> SELECT DBMS_METADATA.GET_GRANTED_DDL('OBJECT_GRANT',<Schema_Name>’) from dual;

You can run below command to generate DDL statement of the role
SQL> SELECT DBMS_METADATA.GET_DDL('ROLE','RESOURCE') from dual;

You can also use SQL Developer to get the DDL script for a specific object.

You can refer oracle documentation for more details on DBMS_METADATA

Thanks & Regards

http://oracleracexpert.com

ORA-39346: data loss in character set conversion for object TABLE_STATISTICS

When doing export/import using data pump encountered below error. This happens when invalid or corrupt characters are stored in the database.

When using export/import instead of data pump, no errors reported.

Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/REF_CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/TRIGGER
Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
ORA-39346: data loss in character set conversion for object TABLE_STATISTICS:"ORCL"."TABLE1"
ORA-39346: data loss in character set conversion for object TABLE_STATISTICS:"ORCL"."TABLEMAT"
Job "SYS"."SYS_IMPORT_SCHEMA_01" completed with 2 error(s) at Sun Apr 11 12:05:25 2021 elapsed 0 00:20:13

After research found that this is a known issue due to Bug 13696958 .

The fix for Bug 13696958 

added warning messages in 12.1 datapump import for characters that couldn't be converted. Previously, a fix for Bug 16173664 was made available to report the affected object in the import log.

Workaround/fix:-

Needs to apply interim patch 21342624 
and execute post-install step:

$ datapatch

If the patch 21342624 installed then impdp log will report the objects that are experiencing data loss during characterset conversion. If the fix then character data is checked even when the client and database character sets are the same, to prevent input of invalid character data.

That being said the fix for Bug 21342624 does not stop the error messages or repair any bad data on the source database

In case of any further issues , patch questions or concerns, please contact Oracle Support.

Refer Oracle support notes: -

Unpublished Bug 21342624 - DATA LOSS DUE TO CHARACTER SET NOT SHOWING CORRECT SET OF OBJECTS DURING IMPORT.

Note 1317012.1 - How To Use MOS Patch Planner To Check And Request The Conflict Patches?

Thanks & Regards,
http://oracleracexpert.com, Oracle ACE

ORA-39095: Dump file space has been exhausted: Unable to allocate 8192 bytes

Uses can receive below error during the export in case if you did not provide enough file names

$expdp xxx/xxxx directory=EXP_DIR DUMPFILE=exp_user.dmp LOGFILE=exp_user.log SCHEMAS=EMPUSER FILESIZE=20m
…..

ORA-39095: dump file space has been exhausted: unable to allocate 4096 bytes
Job "EMPUSER"."sys_export_table_01" stopped due to fatal error at 10:05:03

You will also receive this error if you specify one dump file or a number less than parallelism value. The slave processes locking the file does not release the lock even after finishing as it wait other processes to write to the file.

$expdp xxx/xxxx directory= EXP_DIR DUMPFILE=exp_user.dmp LOGFILE=exp_user.log SCHEMAS=EMPUSER PARALLEL=5

ORA-39095: dump file space has been exhausted: unable to allocate 8192 bytes

DONOT use PARALLEL clause to avoid this error.

In case if you have space issue on file system, then you can generate multiple dumps on multiple file systems or disks.


Thanks,
http://oracleracexpert.com
Oracle ACE

Oracle Database Upgrade to 12c and Available Methods

Oracle 12c offers many new features and upgrading database can bring many advantages to organization. There are various upgrade and migration methods available and the best method to use for your upgrade/migration scenario depends on the source database version, the source and destination operating systems, your downtime requirements, and the personal preference of the DBA. Based upon factors there is a method available to best fit your organization needs.

We have several upgrade methods available

• DBUA
• command line
• Full Transportable Export/Import
• Transportable Tablespaces
• Data Pump Export/Import
• Original Export/Import

For complete Article Please check below link

https://www.linkedin.com/pulse/oracle-database-upgrade-12c-available-methods-gunukula-oracle-ace

Regards
Satishbabu Gunukula, Oracle ACE
http://www.oracleracexpert.com

ORA-39126: Worker unexpected fatal error in KUPW$WORKER.LOAD_METADATA

I have recently come across the below error when importing the data using DATA PUMP.

ORA-39126: Worker unexpected fatal error in KUPW$WORKER.LOAD_METADATA [SELECT..
......................................
ORA-39183: internal error -19 ocurred during decompression phase 2

ORA-06512: at "SYS.DBMS_SYS_ERROR", line 95
ORA-06512: at "SYS.KUPW$WORKER", line 9715

----- PL/SQL Call Stack -----
  object      line  object
  handle    number  name

0x64f77ff8     21979  package body SYS.KUPW$WORKER

Cause: Normally users will see ORA-39183 error when the dump got corrupted during the file transfer or user might not be using BINARY mode using FTP

I would recommend taking once more export and copy again, it should work.

Users might come across below known issue

ORA-39126: Worker unexpected fatal error in KUPW$WORKER.PUT_DDLS [PASSWORD_HISTORY:"<USER>"]
ORA-06502: PL/SQL: numeric or value error
LPX-00216: invalid character 0 (0x0)
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 95
ORA-06512: at "SYS.KUPW$WORKER", line 8358
Cause: In this case hashes of the old passwords stored in user_history$ are corrupted.

Workaround: Please follow below steps, if you error doesn’t match please contact Oracle support

1. EXCLUDE=PASSWORD_HISTORY on impdp statment
or
2. Delete the user_history$ information pertaining to that user.
    SQL> delete from user_history$ where user# in (select user# from user$ where name='USER1');

Please refer below metatlink notes before making any change

RA-39126: "Worker unexpected fatal error in KUPW$WORKER.PUT_DDLS [PASSWORD_HISTORY: ...]" (Doc ID 2114832.1)

DataPump Import Errors ORA-39183 Internal Error -19 Ocurred During Decompression Phase 2 ORA-6512 [ID 1524577.1]

Regards
Satishbabu Gunukula, Oracle ACE
http://oracleracexpert.com

ORA-31613: Master process DM00 failed during startup

While taking export backup using data pump user received following error.

ORA-31637: cannot create job SYS_EXPORT_SCHEMA_01 for user

ORA-06512: at "SYS.DBMS_SYS_ERROR", line 95

ORA-06512: at "SYS.KUPV$FT", line 1587

ORA-39062: error creating master process DM00

ORA-31613: Master process DM00 failed during startup.

Normally user receives this error when maximum numbers of processes are exceeded. You will see below error in trace files.

ORA-00020: maximum number of processes (150) exceeded

Solution: To fix the issue increase PROCESSES initialization parameter and restart the database.

If you don’t see the error ORA-00020 trace file then look for other error messages which will help to fix the issue. 

I had faced similar issue and  but don't see ORA-00020 error. After looking the other error messages I see that child processes are terminated and in my case restarting the database resolved the issue.

Regards,

Satishbabu Gunukula, Oracle ACE

http://oracleracexpert.com

Domain index creation hangs during datapump import

The impdp hangs while creating Domain index and never completes. User waited for 3 days but no luck. All the objects got imported except one domain index.


$ tail -f output
…..
Processing object type SCHEMA_EXPORT/TABLE/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type SCHEMA_EXPORT/TABLE/COMMENT
Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_SPEC
Processing object type SCHEMA_EXPORT/FUNCTION/FUNCTION
Processing object type SCHEMA_EXPORT/PROCEDURE/PROCEDURE
Processing object type SCHEMA_EXPORT/PACKAGE/COMPILE_PACKAGE/PACKAGE_SPEC/ALTER_PACKAGE_SPEC
Processing object type SCHEMA_EXPORT/FUNCTION/ALTER_FUNCTION
Processing object type SCHEMA_EXPORT/PROCEDURE/ALTER_PROCEDURE
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/VIEW/VIEW
Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_BODY
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/REF_CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/INDEX/DOMAIN_INDEX/INDEX
When I query data_datapump_jobs , I can see that import is still running.

OWNER_NAME JOB_NAME OPERATION JOB_MODE STATE DEGREE ATTACHED_SESSIONS DATAPUMP_SESSIONS
------------------------------ ------------------------------ ------------------------------ ------------------------------ ------------------------------ ---------- ----------------- -----------------
SYS SYS_IMPORT_SCHEMA_01 IMPORT SCHEMA EXECUTING 32 2 35
SAPCLD SYS_EXPORT_SCHEMA_01 EXPORT SCHEMA NOT RUNNING 0 0 0
When I query v$session_longops, I can see that 48915 out of 48916 MB done and job hangs

select * from (
select opname, target, sofar, totalwork,
units, elapsed_seconds, message
from v$session_longops order by start_time desc) …..

SYS_IMPORT_SCHEMA_01 48915 48916 MB 81659 SYS_IMPORT_SCHEMA_01: IMPORT : 48915 out of 48916 MB done
Solution: After research I found that there is a bug in 11.2.0.4 and applying below patch resolved the issue.
Patch 23521888: MERGE REQUEST ON TOP OF 11.2.0.4.0 FOR BUGS 20503463 16683112


Reference: Please find couple of datapump performance Bugs in 11.2.0.4
============================================================
For 11.2.0.4
EXPDP Performance Bugs:
MLR Patch 21443197 released on top of 11.2.0.4 contains the fixes for the bugs: 18082965 18469379 18793246 20236523 19674521 20532904 20548904

IMPDP Performance Bugs:
- Bug 19520061 - IMPDP: EXTREMELY SLOW IMPORT FOR A PARTITIONED TABLE
- Bug 13609098 - IMPORTING SMALL SECUREFILE LOBS USING DATA PUMP IS SLOW >>>>> This patch is already in lsinventory

Bug 21128593 : UPDATING THE MASTER TABLE AT THE END OF DP JOB IS SLOW STARTING WITH 12.1.0.2>> But patch is already in lsinventory

syntax error: unterminated quoted string literal

I have encountered below “sql syntax error” during data import into HANA Dynamic Tiering

* 257: sql syntax error: unterminated quoted string literal: line 1 col 13 (at pos 13) SQLSTATE: HY000
Here are the possible reasons for this error

1. You will see this error when data file name is wrong in the control file

2. The “quote” is not closed properly in the IMPORT command.

For ex:-
HANA=>IMPORT FROM '/hana/shared/DT/HDB00/work/employee.txt.ctl
As you see quote is not ended properly in above command

3. when data file is empty

Regards,
Satishbabu Gunukula, Oracle ACE

[SAP][ODBC Driver]Non-space text found after ending quote

I have encountered below error during data import into HANA Dynamic Tiering

* 403: internal error: [SAP][ODBC Driver]Non-space text found after ending quote
-- (dblib/db_RecScanner.cxx 2657) SQLSTATE: HY000

I found that data has quotes (“) which is causing the issue. But didn’t find any forum/link related to this error in HANA DT or solution.

At this situation SLT can help, I have used SLT to load the data and was able to load successfully.

During research I found that few users encountered same issue during the data import into SYBASE
They propose below recommendations

- Remove quotes from all values in data file
- TURN OFF the QUOTES as QUOTES ON is default.

Refer below link for more info for SYBASE error
https://scn.sap.com/thread/3638327

Other possible errors you may encounter:

* 403: internal error: [SAP][ODBC Driver]Cannot convert PatKey to a integer(10) (column EMPLOYEE_ID) SQLSTATE: HY00

Possible solution: Please choose appropriate data type.

Regards
Satishbabu Gunukula, Oracle ACE
http://www.oracleracexpert.com