Data Migration. External Table version 1

Transcription

1 Data Migration External Table version 1

2 Methods of moving data across platforms 2010 Version 1 2

3 External Table Methods of moving data across platforms Method 1 Database Link 2 SQL*Plus spools 3 UTL_FILE Package 4 SQL*Loader utility 5 Transportable tablespaces with RMAN 6 Export and Import utility 7 Data Pump utility 8 External table with ORACLE_DATAPUMP access driver 9 External table with ORACLE_LOADER access driver 10 3 rd party ETL tools 2010 Version 1 3

4 External Table Introduction 2010 Version 1 4

5 External Table Introduction External tables access data in external sources as if it were in a table in the database Introduced in 9i to be read-only in text-only datafiles Enhanced in 10g to be read-only or read-write, composed of flat files in a binary format Further enhanced in 11g to take advantage of COMPRESSION and ENCRYPTION options 2010 Version 1 5

6 External Table Introduction External tables are created using the SQL : CREATE TABLE... ORGANIZATION EXTERNAL statement Attributes include : TYPE: ORACLE_LOADER, ORACLE_DATAPUMP DEFAULT DIRECTORY ACCESS PARAMETERS LOCATION 2010 Version 1 6

7 External Table Introduction Use ORACLE_DATAPUMP Access Driver to Unload and Load data The data unloaded into flat files are read only by ORACLE_DATAPUMP driver Neither DML nor Index Creation Operation are allowed. The data unloaded can be used for another external tables in the same or a different database The degree of parallelization is tied to the number of files to which you are unloading. Dump files populated by different external tables can be aggregated into one and vice versa 2010 Version 1 7

8 External Table Introduction ORACLE_DATAPUMP access drive automatically resolves: Different character national character sets Endianness For the supported datatypes: Character, RAW, Number, Date, ROWID, etc Version 1 8

9 External Table Introduction Query External Tables Select Join Sort Vews and Synonyms Statistics Dropping External Tables Use DROP TABLE statement Only the table metadata is removed No effect on the actual data Views specific to external tables: *_EXTERNAL_TABLES *_EXTERNAL_LOCATIONS 2010 Version 1 9

10 How to move data using external tables 2010 Version 1 10

11 How to move data using external tables In source database: Create a DIRECTORY object in source database Create unloading external table Verify dump files created Ship the dump file to target server In target database: Create a DIRECTORY object in the target database Create load external table in the target database CTAS target table with external table 2010 Version 1 11

12 How to move data using external tables Source Tables Unload External table on file system Transfer using FTP Target External table on file system Load Table 2010 Version 1 12

13 Case Study 2010 Version 1 13

14 Case Study Create a unload DIRECTORY object in Source Database As user sys: Create directory unload as /unload ; Grant read, write on directory unload to <unload_user>; SQL> create directory unload as '/backup/unload'; Directory created. SQL> grant read, write on directory unload to unloader; Grant succeeded. Vefify a load directory object; SQL> SELECT * FROM dba_directories WHERE directory_name = 'UNLOAD'; OWNER DIRECTORY_NAME DIRECTORY_PATH SYS UNLOAD /backup/unload 2010 Version 1 14

15 Case Study Create a load DIRECTORY object in Target Database As user sys: Create directory load as /load ; Grant read, write on directory load to <load_user>; SQL> create directory load as '/backup/load'; Directory created. SQL> grant read, write on directory load to loader; Grant succeeded. Vefify a load directory object; SQL> SELECT * FROM dba_directories WHERE directory_name = 'LOAD'; OWNER DIRECTORY_NAME DIRECTORY_PATH SYS LOAD /backup/load 2010 Version 1 15

16 Case Study Create external table to unload CREATE TABLE ext_employees ORGANIZATION external ( TYPE ORACLE_DATAPUMP DEFAULT DIRECTORY unload ACCESS parameters (logfile ext_employees) LOCATION ('ext_employees.dmp') ) AS SELECT * FROM hr.employees; Vefify external table created ext_employees ; SQL> desc ext_employees Name Null? Type EMPLOYEE_ID NUMBER(6) FIRST_NAME VARCHAR2(20) LAST_NAME NOT NULL VARCHAR2(25) NOT NULL VARCHAR2(25) PHONE_NUMBER VARCHAR2(20) HIRE_DATE NOT NULL DATE 2010 Version 1 16

17 Case Study Vefify the dump file created : SQL>!ls -ltr /backup/unload 총 42 -rw-r--r-- 1 oracle dba 41 3월 23일 11:09 ext_employees.log -rw-r oracle dba 월 23일 11:09 ext_employees.dmp Vefify external table created ext_employees ; SQL> SELECT owner, table_name, type_name, default_directory_name FROM dba_external_tables WHERE owner = 'UNLOADER' AND table_name = 'EXT_EMPLOYEES'; OWNER TABLE_NAME TYPE_NAME DEFAULT_DIRECTORY_NAME UNLOADER EXT_EMPLOYEES ORACLE_DATAPUMP UNLOAD 2010 Version 1 17

18 Case Study Compress the dump file generated : SQL>!gzip ext_employees.dmp SQL>!ls ltr -rw-r--r-- 1 oracle dba 41 3월 23일 11:09 ext_employees.log -rw-r oracle dba 월 23일 11:09 ext_employees.dmp.gz ftp the compressed dump file from Source server to Target Server : d_svr=' ' d_pwd='oracle' d_user='oracle' load_dir='/backup/load' ftp -n -i ${d_svr} << EOF user ${d_user} ${d_pwd} bin cd ${load_dir} put ext_employees.dmp.gz bye EOF 2010 Version 1 18

19 Case Study Compress the dump file generated : SQL>!gzip ext_employees.dmp SQL>!ls ltr -rw-r--r-- 1 oracle dba 41 3월 23일 11:09 ext_employees.log -rw-r oracle dba 월 23일 11:09 ext_employees.dmp.gz ftp the compressed dump file from Source server to Target Server : d_svr=' ' d_pwd='oracle' d_user='oracle' load_dir='/backup/load' ftp -n -i ${d_svr} << EOF user ${d_user} ${d_pwd} bin cd ${load_dir} put ext_employees.dmp.gz bye EOF 2010 Version 1 19

20 Case Study Vefify the dump file transferred : SQL>!ls -ltr /backup/load -rw-r--r-- 1 oracle dba 월 23일 11:33 ext_employees.dmp.gz Uncompress the dump file transfered : SQL>!ls -ltr /backup/load -rw-r--r-- 1 oracle dba 월 23일 11:33 ext_employees.dmp 2010 Version 1 20

21 Case Study Extract external table ddl to load in source database SQL> set pages 0 long SQL> select dbms_metadata.get_ddl('table','ext_employees','unloader') from dual; CREATE TABLE "UNLOADER"."EXT_EMPLOYEES" ( "EMPLOYEE_ID" NUMBER(6,0), "FIRST_NAME" VARCHAR2(20), "LAST_NAME" VARCHAR2(25), " " VARCHAR2(25), xternal "PHONE_NUMBER" table created VARCHAR2(20), ext_employees ; "HIRE_DATE" DATE, "JOB_ID" VARCHAR2(10), "SALARY" NUMBER(8,2), "COMMISSION_PCT" NUMBER(2,2), "MANAGER_ID" NUMBER(6,0), "DEPARTMENT_ID" NUMBER(4,0) ) ORGANIZATION EXTERNAL ( TYPE ORACLE_DATAPUMP DEFAULT DIRECTORY "UNLOAD" ACCESS PARAMETERS (logfile ext_employees) LOCATION ('ext_employees.dmp') ) 2010 Version 1 21

22 Case Study Modify & create external table ddl to load in target database CREATE TABLE "LOADER"."EXT_EMPLOYEES" ( "EMPLOYEE_ID" NUMBER(6,0), "FIRST_NAME" VARCHAR2(20), "LAST_NAME" VARCHAR2(25), " " VARCHAR2(25), "PHONE_NUMBER" VARCHAR2(20), "HIRE_DATE" DATE, "JOB_ID" VARCHAR2(10), "SALARY" NUMBER(8,2), "COMMISSION_PCT" NUMBER(2,2), "MANAGER_ID" NUMBER(6,0), "DEPARTMENT_ID" NUMBER(4,0) ) ORGANIZATION EXTERNAL ( TYPE ORACLE_DATAPUMP DEFAULT DIRECTORY "LOAD" ACCESS PARAMETERS (logfile ext_employees) LOCATION ('ext_employees.dmp') ); 2010 Version 1 22

23 Case Study Vefify external table created ext_employees in target database SELECT owner, table_name, type_name, default_directory_name FROM dba_external_tables WHERE owner = 'LOADER' AND table_name = 'EXT_EMPLOYEES'; OWNER TABLE_NAME TYPE_NAME DEFAULT_DIRECTORY_NAME LOADER EXT_EMPLOYEES ORACLE_DATAPUMP LOAD 2010 Version 1 23