vendredi 6 décembre 2019

How i moved a 34T Multitenant Database from Exadata to Non-Exadata server in less than One hour


1. INTRODUCTION


My customer requested me to migrate (move) a huge Multitenant Database (34T)  from Exadata to a non-Exadata server.
The requirements where :

  • The sources Environnements are : 
    • 2 RAC Nodes Database in Dataguard with 2 other 2 RAC Nodes
    • Oracle release 12cr2
    • Oracle servers : Exadata
    • Oracle Multitenant
  • The target Environnements are :
    • Standalone Database in Dataguard with an other Standalone Database.
    • Oracle release 12cr2
    • Oracle servers : San VBlock Linux
    • Oracle Multitenant


I designed and applied a solution with little Down Time : "Zero Down Time", and the D-Day, the migration took less than One hour.



In other words, i moved the Dataguard Exadata to a Standalone Dataguard in a time les than one hour.



2. PRINCIPLE


To minimise the migration time the D-Day, i chosed to use the "Cascade Standby" method, given that the two soure and target databases have the same release.




We assume:

Source 12cr2
Target 12cr2
Primary RAC : 
    Nodes : PR_N1 - PR_N2
    Unique DB Name : PR_EXA

Standby RAC:
Nodes : ST_N1 - ST_N2
Unique DB Name : ST_EXA
Primary  :  
Unique DB Name : PR_SAN

Standby:
Unique DB Name : ST_SAN



The approch was :
Two weeks before the migration :
  • Apply the same PSU, Bundel Patchs ... on the target machines as those on the source machines.  
  • Build the first SAN Vblock database PR_SAN as a  standby of  the primary Exadata database: PR_N1 - PR_N2.
  • Add it as a secondary standby of the primary Exadata database.
  • Build the second SAN Vblock database ST_SAN as standby of the first SAN database  PR_SAN.

The D-Day :
  • Do a failover on the first Standby SAN Vblock database PR_SAN, so it becomes a primary database
  • Create a broker to manage the new dataguard (SAN Vblock machines)
  • If necessary, create new services ...



3. BUILD THE FIRST SAN Vblock DATABASE


Two weeks before the migration, the approch was :
  • Open Streams network between the  PR_N1 -  PR_N2 servers and PR_SAN server 
Example :
            srvctl config vip -node PR_N1
VIP exists: network number 1, hosting node PR_N1
VIP Name:PR_N1-vip
        VIP IPv4 Address: aa.xx.xx.xx.
...
             VIP Name:PR_N1
         VIP IPv4 Address: bb.yy.yy.yy


   So ask for opening network streams between :
    •           PR_N1-vip  (aa.xx.xx.xx) &   PR_SAN : Port 1521 
    •           PR_SAN & PR_N1-vip  (aa.xx.xx.xx)   : Port 1522  ( This port has been used for Broker ...))      
    •   
    •           PR_N1  (bb.yy.yy.yy) &   PR_SAN : Port 1521 
    •           PR_SAN & PR_N1  (bb.yy.yy.yy)   : Port 1522  ( This port has been used for Broker ...))   
  • Open Streams network between the  ST_N1 -  ST_N2 servers and PR_SAN server
  • Create PR_SAN as a Standby of PR_N1 : it tooks 33h to finish

4. ADD TO THE EXISTING BROKER

  1. The current Source Broket configuration is  :

DGMGRL> show configuration;

Configuration - PR_N_dg_config

  Protection Mode: MaxPerformance
  Members:
  PR_EXA     - Primary database
    ST_EXA    - Physical standby database
      
Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS   (status updated 54 seconds ago)
               



2. Add PR_SAN as a secondary standby ...



You should infer that:

• PR_EXA database sends its redo stream to the ST_EXA database when PR_EXA is the primary (LOCAL is an alternative label

to specify the current database). 

• ST_EXA database sends its redo stream to the PR_EXA database when ST_EXA is the primary. 

• ST_EXA database forwards the redo stream to the PR_SAN database when ST_EXA is the secondary.

• PR_EXA database forwards the redo stream to the PR_SAN database when PR_EXA is the secondary.

Important :
Also notice that the configuration is not using the Real-Time Cascade feature because of the label “receiving archived redo”.

So :


DGMGRL>
ADD DATABASE "PR_SAN" AS CONNECT IDENTIFIER IS 'PR_SAN' MAINTAINED AS PHYSICAL;
edit database 'PR_EXA' set property redoroutes = '(localST_EXA)( ST_EXA:PR_SAN)';
edit database 'ST_EXA' set property redoroutes = '(PR_EXA:PR_SAN)(localPR_EXA)';
ENABLE CONFIGURATION;
               




We obtain then :




DGMGRL> show configuration;

Configuration - PR_N_dg_config

  Protection Mode: MaxPerformance
  Members:
  PR_EXA     - Primary database
    ST_EXA    - Physical standby database
      PR_SAN, - Physical standby database (receiving archived redo)

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS   (status updated 54 seconds ago)
               






So the Streams will be,




 PR_EXA================>ST_EXA
                                                          ||
                                                          ||
                                                          ||
                                                         \  /
                                                       PR_SAN
               



3. For PR_SAN database, add:

CONFIGURE ARCHIVELOG DELETION POLICY TO APPLIED ON ALL STANDBY;



5. BUILD THE STANDBY SAN Vblock DATABASE


  • Create ST_SAN as a Standby of PR_SAN
  • !!!! Do not configure a broker between them
  • Some configurations :
    • For  PR_SAN
alter system set log_archive_config='dg_config=(PR_SAN,PR_EXA,ST_EXA, ST_SAN)'  scope=spfile;

alter system set log_archive_dest_2 ='service="ST_SAN",LGWR ASYNC NOAFFIRM valid_for=(standby_logfile,all_roles) db_unique_name="ST_SAN"' scope=spfile;
               

  • For  ST_SAN  

Archivelog policy :
       CONFIGURE ARCHIVELOG DELETION POLICY TO APPLIED ON ALL STANDBY;
                                                                                             
Config parameters                                                                                        
             - alter system set standby_file_management= AUTO scope=spfile;
             - alter system set fal_client='ST_SAN' scope=spfile;
             - alter system set fal_server='PR_SAN' scope=spfile;                     
             - alter system set log_archive_config = 'dg_config=(ST_SAN,PR_SAN)' scope=spfile;

Now the archives log are applied on the three standby databases :



 PR_EXA================>ST_EXA
                                                          ||
                                                          ||
                                                          ||
                                                         \  /
                                                       PR_SAN
                                                          ||
                                                          ||
                                                          ||
                                                         \  /
                                                      ST_SAN


4. D-Day Migration


So, the D-Day, we can migrate 25T from Exdata to Non-Exadata environnement, the following steps will take less than one hour.

1. Remove  PR_SAN from the Exadata  broker
  

DGMGRL>
edit database   'PR_EXA' set property redoroutes = '';
edit database   'ST_EXA' set property redoroutes = '';
REMOVE DATABASE " PR_SAN" ;               

2.Do a failover on PR_SAN, so we open it as primary database : sequences will start from 1.

3.Configure a broker between PR_EXA & ST_EXA



DGMGRL> show configuration;

Configuration - SAN_dg_config

  Protection Mode: MaxPerformance
  Members:
  PR_SAN     - Primary database
    ST_SAN    - Physical standby database

Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS   (status updated 54 seconds ago)
               

4. 

  • Make some changes : TEMP tablespaces, services ...
  • Delete old archives log from Old Incarnation ...