Friday, 27 June 2025

SQL Troubleshooting Step By Step Lab in Oracle 19c RAC Database

 Hi All,


In this post i am sharing related to SQL Troubleshooting lab excise Step by step Process.

Firstly check the status of database services and load on the servers.



- here i am using 2 Node RAC Database , as we can see from the above snap 2 nodes database are running fine, will bit load on this server.

- Check the database status and confirmed its not a multitenant database 



- For SQL Troubleshooting Lab we can create a Table and load some data 

SQL> CREATE TABLE customer_orders (

    order_id     NUMBER,

    customer_id  NUMBER,

    order_date   DATE,

    order_amount NUMBER,

    status       VARCHAR2(10)

);

Table created.

SQL> select * from customer_orders;

no rows selected



- No data in that table , now i can load some data 

SQL> BEGIN

  FOR i IN 1 .. 500000 LOOP

  INSERT INTO customer_orders VALUES (

      i,

      MOD(i, 1000),

      SYSDATE - MOD(i, 365),

      ROUND(DBMS_RANDOM.VALUE(100, 10000)),

      CASE WHEN MOD(i, 3) = 0 THEN 'NEW'

           WHEN MOD(i, 3) = 1 THEN 'CLOSED'

           ELSE 'PENDING' END

    );

  END LOOP;

  COMMIT;

END;

 /


PL/SQL procedure successfully completed.

SQL> select count(*) from customer_orders;

  COUNT(*)

----------

    500000



- Now i have loaded 5 lakh rows of data.

- From this table we try to access the data and we can identify how much resource it consume like CPU memory and whether its taking full table scan or how we can see it.

-  this sql is insufficient.

SELECT * FROM customer_orders WHERE TO_CHAR(order_date, 'YYYY-MM-DD') = '2025-06-28' AND status = 'NEW';



- Here we can see more rows of data  , problems from this query is TO_CHAR() disables index usage , Causes Full table scan , also it leads to High CPU and I/O usage.

Now here is main point how we can troubleshoot this problematic sql .

- Generate explain Plan for this sql .

EXPLAIN PLAN FOR SELECT * FROM customer_orders WHERE TO_CHAR(order_date, 'YYYY-MM-DD') = '2025-06-28' AND status = 'NEW';


- here can see this sql is full table scan



- we can generate AWR and monitor this sql .

@$ORACLE_HOME/rdbms/admin/awrrpt.sql 

we can check the TOP sql   by Elapsed time / Buffer Gets also look for db file scattered read( sign of Full Table scan).

- Monitor the active sql's which are executing this sql will work from 19C db onwards.


SQL> SELECT sql_id, sql_text, status, elapsed_time, cpu_time

FROM v$sql_monitor WHERE status = 'EXECUTING';

 no rows selected


How to Troubleshoot the above Full table scan ? 

- From the above sql we identified problems which are

Issue 1 :  TO_CHAR() disables index usage , Causes Full table scan 

Fix for Issue 1 : 

SQL> SELECT * FROM customer_orders WHERE order_date = TO_DATE('2025-06-28', 'YYYY-MM-DD') AND status = 'NEW';

no rows selected

- Add index to that table .

SQL> CREATE INDEX idx_order_date_status ON customer_orders(order_date, status);

Index created.


- re-run the explain plan and see the output .

EXPLAIN PLAN FOR SELECT * FROM customer_orders WHERE order_date = TO_DATE('2025-06-28', 'YYYY-MM-DD') AND status = 'NEW';

Explained.





- Now its Index  range scan we avoid the full table scan . 

SQL Monitor report: should show reduced elapsed time
AWR: reduced cost/CPU/IO stats

Post Index creation You should see faster execution and lower logical reads .



SELECT * FROM customer_orders WHERE TO_CHAR(order_date, 'YYYY-MM-DD') = '2025-06-28' AND status = 'NEW';






without Index :



after index : 




Note: Generally we dont see when we used where clause with small amount of data request but in real time when we are accessing huge amount of data then we can understand clearly the power of Indexing .



Thanks ,
Srini


Saturday, 21 June 2025

Oracle 19C 2 Node RAC Dataguard complete setup step by step process

 Hi All,


In this post i am sharing Oracle 19C RAC 2 Node RAC Dataguard setup.

For this Lab  i have used 4 Virtual machines for Primary and standby servers configurations.

My Primary & Standby Environment details:

======================= =============



>> Primary node (we take as Production - its a live data / live business) .

>> we need to build the standby database for production High availability, in case of production server failures we can use standby server as primary server.

>> my Primary database is 2 nodes 19C RAC database with ASM storage, OS is OEL 7.

>> we need to configure standby servers with 2 Node 19C RAC with ASM Storage, OS OEL 7 .

on standby server we need to configure the Grid infrastructure on both standby nodes, then install the oracle 19C software only on both the standby servers. remain follow below steps to enable Dataguard.


>> Primary node host details /etc/hosts

[oracle@prodnode1 admin]$ cat /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhost localhost.localdomain localhost6 localhost6.localdomain6


#Public

192.168.0.21    prodnode1.localdomain.com       prodnode1

192.168.0.22    prodnode2.localdomain.com       prodnode2

192.168.0.23    prodnode3.localdomain.com       prodnode3


#Private

192.168.1.21    prodnode1-priv.localdomain.com  prodnode1-priv

192.168.1.22    prodnode2-priv.localdomain.com  prodnode2-priv

192.168.1.23    prodnode3-priv.localdomain.com  prodnode3-priv


#VIP

192.168.0.31    prodnode1-vip.localdomain.com   prodnode1-vip

192.168.0.32    prodnode2-vip.localdomain.com   prodnode2-vip

192.168.0.33    prodnode3-vip.localdomain.com   prodnode3-vip


#prod-scan (VIP)

192.168.0.41     prod-scan.localdomain.com      prod-scan

192.168.0.42     prod-scan.localdomain.com      prod-scan

192.168.0.43     prod-scan.localdomain.com      prod-scan


>> Standby node host details /etc/hosts



>> Primary node configuration ( archive log mode, force_loggining mode should be enable)


select name,force_logging,log_mode,FLASHBACK_ON from v$database;

shut immediate

startup mount


alter database archivelog;

alter database force logging;

alter database flashback on;


Check the db_unique_name 

SQL> show parameter db_uniq


NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

db_unique_name                       string      prod


set linesize 300 pages 100

col value for a45

col name for a25

select name, value

from v$parameter

where name in ('db_name',

'db_unique_name',

'log_archive_config',

'log_archive_dest_1',

'log_archive_dest_2',

'log_archive_dest_state_1',

'log_archive_dest_state_2',

'remote_login_passwordfile',

'log_archive_format',

'log_archive_max_processes',

'fal_client',

'fal_server',

'db_file_name_convert',

'log_file_name_convert',

'standby_file_management');




Configure the standby redo logs : ( 2+1*2) 

=========================

SQL> select GROUP#,THREAD#,BYTES/1024/1024,STATUS,MEMBERS FROM V$lOG;


 



add the standby redo logs >> these will useful only when you are using max availability and max protection modes.. not in the max performance mode..

ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 group 5('+DATA/PROD/STANDBYLOGS/stdby_01.log') SIZE 200M;  etc ... add +1 extra for each thread for primary .. in your primary if you have 2 groups 4 threads for standby you need to create 2 groups with 3 threads each .

SQL> SELECT INST_ID, GROUP#,THREAD#,SEQUENCE#,ARCHIVED,STATUS FROM GV$STANDBY_LOG ORDER BY 2,1;

no rows selected

Thanks,
Srini


Friday, 30 May 2025

Oracle RAC DBA Online Training - 7th June 2025

 

 Dear All,

Those who want to learn Oracle 19C RAC DBA online training from starting to Expert level follow the below course content.

Course Duration:  7 weekends ( Sat & Sun)  3 hours / per day ( 42 hours )

 

Staring the training from  7th June 2025 to 20th Jul 2025 (Total 7 weekends)

Weekend sessions: Saturday: 8:00 AM to 11 AM IST , 10:30 PM to 1:30AM ET

                                     Sunday:  8:00 AM to 11 AM IST , 10:30 PM to 1:30AM ET

 

Demo Session :   Oracle RAC DBA Demo session Weekend batch

Time: 7th June 2025 08:00 AM IST

Join Zoom Meeting :: Those who are interested to join Oracle RAC DBA online training . Join below invite only on 7th June 2025 8AM IST to 8:45AM IST.

 

Join Zoom Meeting https://us06web.zoom.us/j/87615430065?pwd=mkv32hOVeLcAbCQ0LDbm9uiBY5JXn7.1 Meeting ID: 876 1543 0065 Passcode: 5cgUgL



Course Content:

 

WhatsApp group:

https://chat.whatsapp.com/JTC9OhT5Aq1ETTuByC3MEr

 

Contact no:  +91 9392634440 ( whatsapp) , mail id :  racsinfotech@gmail.com

Registration Form :: https://docs.google.com/forms/d/1L8gWNTCP3WvOlklR62oag3XGXCPqd_CBjbDl2AtsK1s/viewform

  RacsinfoTech Website: https://racsinfotech.com/

 

RAC Course Contents:

1. Introduction to Oracle Custer ware / Cluster ware Architecture

>> Standalone Vs RAC

>> What is Oracle Cluster ware

>> Oracle Cluster ware Architecture and Cluster Services

>> RAC One Node Vs RAC database

>> Online Relocation

>> Networking in Cluster ware

>> Cluster ware services Start-up and shutdown Sequences

>> what are OCR and Voting disk

 

2. Linux Installation and Configuration step by step on Oracle VM.

>> Oracle Virtual box 7 version setup on windows.

>> Installation of Linux OS (7.x)

>> OS Post Installation checks and Tasks

>> Node1 VM to Node2 VM Clone .

 

3. Network Setting and Configuration setup.

>> Public IP, Private IP , VIP and Scan IP Configuration

>> DNS Server Configuration

>> VM Shared Disk Creation process

>> how to create ASM Disks using VM shared disk's.

 

4. Cluster Installation and Configuration steps.

>> Custer Pre-installation Tasks

   > System requirements Checks

   > Kernel parameters checks

   > Create required directory ,groups and users for grid & db install

   > Execute the ./runcluvfy from any one of the nodes using below command

     ex: ./runcluvfy.sh stage -pre crsinst -n racnode1,racnode2 -verbose)

>> ssh user password less /equivalence configuration on cluster nodes.

>> Start the Oracle Cluster Installation and Configuration steps using ./gridSetup.sh

>> Cluster Post Installation checks

ex:: ./cluvfy stage -post crsinst -n racnode1,racnode2 -verbose

 

5. Oracle RAC DB Installation and Configuration steps

>> Pre-installation tasks

   > System requirements Checks

   > Kernel parameters checks

   > Create required directory ,groups and users for db install

   > ssh user password less connection  setup / equivalence configuration

>> Installation and Configuration RAC DB.

>> RAC DB software Post Installation Tasks

 

6. Create Oracle ASM Disk groups >> +OCR,+DATA & +ARCH etc disk groups.

 

7. Oracle RAC Database Creation and Configuration

>> Pre-installation for Oracle RAC DB Configuration

./cluvfy stage -pre dbcfg -n racnode1,racnode2 -d Oracle_home -verbose

./cluvfy stage -pre dbinst -n node_list [-r {11gR1 | 11gR2 | 12cR1 | 12cR2}]  [-osdba osdba_group] [-d Oracle_home] [-fixup [-fixupdir fixup_dir] [-verbose]

 

>> Oracle RAC Database Creation and Configuration using dbca / command line using response file.

>> Post Installation Tasks for Oracle RAC DB Configuration

 

8. Oracle ASM Introduction & Architecture

>> Oracle ASM Storage Components.

>> what is Mirroring and Stripping in ASM.

>> what are the ASM instance parameters.

>> Oracle ASM Administration process / steps.

 

9. Oracle ASM Disk Groups Administering

>> How to Administering Oracle ASM Disk Groups.

>> How to Preform Adding Disk to Disk group and Dropping Disks from disk groups.

>> How to Preform Disk group Rebalance and Rebalance Power limit in ASM.

 

10. RAC Database Administration and Oracle Cluster ware Administration

 

##Oracle Cluster ware Administration

>> What are OCR and Voting Disks in Cluster.

>> What are the Cluster Resource and Services

>> What are crsctl vs srvctl utilities / tools.

>> How to Restarting Oracle Cluster ware and Services.

>> How to Restarting Oracle Cluster Databases.

>> How to Rebooting Cluster Nodes.

>> what is VIP, SCAN IP , Private and Public IP In RAC.

 

11. RAC Database Administration

>> Oracle RAC specific parameters In Init/spfile.

>> Difference between pfile and spfile in RAC database

>> Use of password file for RAC database

>> How to manage the Redo Log Files and Control file.

>> UNDO & TEMP Tablespaces use cases.

 

12. Backup and Recovery in Oracle RAC Database

>> Backup strategy Understanding in RAC.

>> Types of Backups in RAC.

>> Recoveries scenarios in RAC.

 

13. Networking setup in RAC ( Listeners in Cluster ware / Local )

>> Grid / ASM Listeners.

>> Remote / SCAN Listeners (Remote Listeners).

>> Local Database Listeners (Local Listeners).

 

14. RAC Database Multitenant Architecture vs Non Multitenant Architecture

>> Oracle Multitenant Architecture In-detail.

>> what are CDB and PDB databases.

>> How to Install the Pluggable database(PDB).

>> How to Clone Pluggable Database(PDB).

 

15. ASM Cluster Filesystem (ACFS) Configuration and Administration

>> How to Configure ACFS in Linux.

>> How to Administering & Managing ACFS.

 

16. Patching Oracle RAC and RAC Databases

>> What are the OPatch Vs opatchauto utilities.

>> How to Patch Oracle Cluster ware.

>> How to Patch RAC Oracle Home.

>> How to Patch RAC Databases.

>> what is Datapatch.

 

17. Oracle Cluster/Grid and Database Upgradation from lower version to Higher

>> Pre-Upgrade steps.

>> How many ways we can perform db and grid  upgrade.

>> Post-upgrade steps.

 

18. How to perform Node Addition and Deletion in RAC

>> Pre-requisites for node addition.

>> Node addition steps

>> post node addition steps.

>> Pre-requisites for node deletion.

>> Node deletion steps

>> post node deletion steps.

 

19. How to Convert the Single Instance to RAC using DBCA / OEM / RCONFIG etc.

 

**20. Interview Preparation Tips and Mock Interviews to crack Realtime Interviews.

**21. Resume / CV Preparation Tips and suggestions.


Thanks,

Srini

Tuesday, 6 May 2025

How to Apply APR-2025 RU patches in Oracle 19C Database step by step

 Dear All,

In this post i am sharing you Apply Apr-2025 RU patch step by step on Oracle 19C Database .

Apr-2025 RU patch Number :  

Patch 37641958: GI RELEASE UPDATE 19.27.0.0.0

 




Click on download patch option and read the readme file to check pre and post checks .

readme file : 

https://updates.oracle.com/Orion/Services/download?type=readme&aru=27123474

click on download patch :







Note : 
  • Document 19202504.9 Oracle Database 19c RU/RUR Apr 2025 Known Issues

After this download the latest opatch version - 6880880



copy the both opatch - patch 6880880 and Apr-2025 patch to database server and unzip it.



opatch readme file : https://updates.oracle.com/Orion/Services/download?type=readme&aru=27122750

>> First upgrade the opatch version .



After opatch upgrade : 






Download and unzip main patch on your database server 

[root@oemdb RU_19_27-GI]# unzip p37641958_190000_Linux-x86-64.zip -d /u01/software/



check inventory good :



check patch conflicts 





check invalids , take db full backup , take oracle home and customizations if anything backup before apply the main patches ..


SQL> select count(*) from dba_objects where status='INVALID';

  COUNT(*)
----------
         0

SQL> select name,open_mode from v$database;

NAME      OPEN_MODE
--------- --------------------
OEMDB     READ WRITE




Stop the database listener and database services 



export path ... 



Go to the patch location and apply the patch

Main APR-2025 RU patch ::

 $opatch apply 







apply second patch >> OCW patch 





Start up database and listener services 



Check the latest patch level ... 



Run datapatch verbose 



[oracle@oemdb OPatch]$ ./datapatch -verbose
SQL Patching tool version 19.27.0.0.0 Production on Wed May  7 01:47:58 2025
Copyright (c) 2012, 2025, Oracle.  All rights reserved.

Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_17547_2025_05_07_01_47_58/sqlpatch_invocation.log

Connecting to database...OK
Gathering database info...done
Bootstrapping registry and package to current versions...done
Determining current state...done

Current state of interim SQL patches:
  No interim patches found

Current state of release update SQL patches:
  Binary registry:
    19.27.0.0.0 Release_Update 250406131139: Installed
  SQL registry:
    Applied 19.3.0.0.0 Release_Update 190410122720 successfully on 07-MAY-25 12.43.26.306463 AM

Adding patches to installation queue and performing prereq checks...done
Installation queue:
  No interim patches need to be rolled back
  Patch 37642901 (Database Release Update : 19.27.0.0.250415 (37642901)):
    Apply from 19.3.0.0.0 Release_Update 190410122720 to 19.27.0.0.0 Release_Update 250406131139
  No interim patches need to be applied

Installing patches...
Patch installation complete.  Total patches installed: 1

Validating logfiles...done
Patch 37642901 apply: SUCCESS
  logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/37642901/27123174/37642901_apply_OEMDB_2025May07_01_49_39.log (no errors)
SQL Patching tool complete on Wed May  7 02:04:35 2025
[oracle@oemdb OPatch]$




Thanks,
Srini


Tuesday, 15 April 2025

How to drop Oracle 19C RAC Database step by step process

 Dear All,


In this post i am sharing step by step process to drop Oracle 19C RAC database.

Note : these steps only for demo purpose only****


srvctl status database -d prod



[oracle@prodnode1 bin]$ srvctl stop database -d prod

[oracle@prodnode1 bin]$ sqlplus / as sysdba


SQL*Plus: Release 19.0.0.0.0 - Production on Tue Apr 15 17:11:31 2025

Version 19.3.0.0.0


Copyright (c) 1982, 2019, Oracle.  All rights reserved.


Connected to an idle instance.


SQL> startup mount exclusive restrict;

ORACLE instance started.


Total System Global Area 2533358728 bytes

Fixed Size                  8899720 bytes

Variable Size             637534208 bytes

Database Buffers         1879048192 bytes

Redo Buffers                7876608 bytes

Database mounted.

SQL> drop database;

drop database

*

ERROR at line 1:

ORA-01586: database must be mounted EXCLUSIVE and not open for this operation



SQL> shut abort

ORACLE instance shut down.

SQL> startup mount restrict restrict;

SP2-0714: invalid combination of STARTUP options

SQL> startup mount restrict;

ORACLE instance started.


Total System Global Area 2533358728 bytes

Fixed Size                  8899720 bytes

Variable Size             637534208 bytes

Database Buffers         1879048192 bytes

Redo Buffers                7876608 bytes

Database mounted.

SQL> drop database;

drop database

*

ERROR at line 1:

ORA-01586: database must be mounted EXCLUSIVE and not open for this operation


SQL> shut abort

ORACLE instance shut down.

SQL> exit

Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.3.0.0.0





[oracle@prodnode1 bin]$ sqlplus / as sysdba


SQL*Plus: Release 19.0.0.0.0 - Production on Tue Apr 15 17:15:00 2025

Version 19.3.0.0.0


Copyright (c) 1982, 2019, Oracle.  All rights reserved.


Connected to an idle instance.


SQL> STARTUP NOMOUNT RESTRICT;

alter database mount;

ORACLE instance started.


Total System Global Area 2533358728 bytes

Fixed Size                  8899720 bytes

Variable Size             637534208 bytes

Database Buffers         1879048192 bytes

Redo Buffers                7876608 bytes

SQL>


Database altered.


SQL> SQL>

SQL>

SQL> drop database;

drop database

*

ERROR at line 1:

ORA-01586: database must be mounted EXCLUSIVE and not open for this operation



SQL> shut abort

ORACLE instance shut down.

SQL> exit

Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.3.0.0.0

[oracle@prodnode1 bin]$ sqlplus / as sysdba


SQL*Plus: Release 19.0.0.0.0 - Production on Tue Apr 15 17:17:01 2025

Version 19.3.0.0.0


Copyright (c) 1982, 2019, Oracle.  All rights reserved.


Connected to an idle instance.


SQL> startup mount restrict;

ORACLE instance started.


Total System Global Area 2533358728 bytes

Fixed Size                  8899720 bytes

Variable Size             637534208 bytes

Database Buffers         1879048192 bytes

Redo Buffers                7876608 bytes

Database mounted.

SQL> alter system set cluster_database=false scope=spfile;


System altered.


SQL> drop database;

drop database

*

ERROR at line 1:

ORA-01586: database must be mounted EXCLUSIVE and not open for this operation



SQL> shut abort

ORACLE instance shut down.

SQL> startup mount restrict exclusive;

ORACLE instance started.


Total System Global Area 2533358728 bytes

Fixed Size                  8899720 bytes

Variable Size             637534208 bytes

Database Buffers         1879048192 bytes

Redo Buffers                7876608 bytes

Database mounted.

SQL> drop database;


Database dropped.


Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.3.0.0.0

SQL>










Thanks,
Srini