Oracle 12c Automatic Data Optimization (ADO) - ILM
Download as DOCX, PDF1 like1,402 views
Automatic Data Optimization (ADO) automatically moves and compresses data according to user-defined policies based on statistics collected by Heat Map. Heat Map tracks data access patterns at the row and segment levels. ADO policies can be defined to compress or move segments after a specified number of days with no modifications. When testing compression policies, ADO automatically compressed the SALES_ADO table after 20 days of no modifications, as determined by simulated Heat Map statistics.
![select object_name, segment_write_time
from dba_heat_map_segment
where object_name='SALES_ADO';
OBJECT_NAME SEGMENT_WRITE_TIME
------------------------------------------------------------
SALES_ADO 20-sep-13 11:03:46
16 rows selected.
SQL> !date
Thursday, 10 October 2013 11:05:30 AM EST
statistics are showing that the table was last accessed more 20 days back.
Step 10: To open and trigger the ADO policies jobs [Note: Not waiting for the
maintenance window for testing]
================================================================
SQL> conn sh/sh
Connected.
SQL> declare
v_executionid number;
begin
dbms_ilm.execute_ILM (ILM_SCOPE => dbms_ilm.SCOPE_SCHEMA,
execution_mode => dbms_ilm.ilm_execution_offline,
task_id => v_executionid);
end;
/
PL/SQL procedure successfully completed.
Step 11:
View the results of the job that completed the compression operation.
================================================================
SQL> select task_id, start_time as start_time from user_ilmtasks;
TASK_ID
----------
START_TIME
---------------------------------------------------------------------------
2
10-OCT-13 11.09.07.952584 AM](https://image.slidesharecdn.com/automaticdataoptimizationwithoracledatabase12c-131010003518-phpapp02/85/Oracle-12c-Automatic-Data-Optimization-ADO-ILM-7-320.jpg)
![For testing: Now set the threshold value for TBS_PERCENT_FREE to (97%) and
TBS_PERCENT_USED to 3%
SQL> conn sys/*** as sysdba
Connected.
SQL> exec dbms_ilm_admin.customize_ilm(dbms_ilm_admin.tbs_percent_free,97)
PL/SQL procedure successfully completed.
SQL> exec dbms_ilm_admin.customize_ilm(dbms_ilm_admin.tbs_percent_used,3)
PL/SQL procedure successfully completed.
SQL> col name format A21
SQL> col value format 9999
SQL> select * from dba_ilmparameters;
NAME VALUE
--------------------- -----
ENABLED 1
JOB LIMIT 10
EXECUTION MODE 3
EXECUTION INTERVAL 15
TBS PERCENT USED 3
TBS PERCENT FREE 97
6 rows selected.
maintenance window to open and trigger the ADO policies jobs [Note: Not
waiting for the maintenance window for testing]
================================================================
SQL> conn sh/sh
Connected.
SQL> declare
v_executionid number;
begin
dbms_ilm.execute_ILM (ILM_SCOPE => dbms_ilm.SCOPE_SCHEMA,
execution_mode => dbms_ilm.ilm_execution_offline,
task_id => v_executionid);
end;
/
Recheck Tablespace Status
------------------------------------
SYSTEM 970 4 966 .37 99.63 32767.98 97.05 2.95
MGMT_AD4J_TS 200 199 1 99.5 .5 32767.98 100 0
TSTADO 400 399 1 99.75 .25 400 99.75 .25
SPACE_PRESS 150 135 15 90 10 150 90 10](https://image.slidesharecdn.com/automaticdataoptimizationwithoracledatabase12c-131010003518-phpapp02/85/Oracle-12c-Automatic-Data-Optimization-ADO-ILM-11-320.jpg)
Oracle 12c Automatic Data Optimization (ADO) - ILM
- 1. Automatic Data Optimization with Oracle Database 12c As of automatic-data-optimization 12c in relate to Information Lifecycle Management (ILM) : In Oracle Database 12c ILM related features Advanced Compression Option. Heat Map automatically tracks modification and query timestamps at the row and segment levels, providing detailed insights into how data is being accessed. Automatic Data Optimization (ADO)automatically moves and compresses data according to user defined policies based on the information collected by Heat Map Heat Map and ADO make it easy to use existing innovations in Oracle Database Compression and Partitioning technologies, which help reduce the cost of managing large amounts of data, while also improving application and database performance. Together these capabilities help to implement first ++++++++++++++++++++++++++++ TESTING: compression policy ++++++++++++++++++++++++++++ Step1: First create tablespace to store SH.SALES data and insert rows: SQL> create tablespace tstado datafile '/BPELAIT2/MONDB01/MONDB01/tstado1.dbf' size 400M reuse autoextend off extent management loc al; Tablespace created. Step2: Then creating another tablespace, space_press where SH.SALES table data may be moved for space pressure SQL> create tablespace space_press datafile '/BPELAIT2/MONDB01/MONDB01/sp1.dbf' size 150M; Tablespace created. SQL> grant dba to sh; Grant succeeded. Step 3: create a procedure which simulates the passage of time in this example so that the table qualifies for ADO action (compression) even though actually 20 days have not elapsed.
- 2. SQL> CREATE OR REPLACE PROCEDURE set_stat (object_id number, 2 data_object_id number, 3 n_days number, 4 p_ts# number, 5 p_segment_access number) 6 as 7 begin 8 insert into sys.heat_map_stat$ 9 (obj#, 10 dataobj#, 11 track_time, 12 segment_access, 13 ts#) 14 values 15 (object_id, 16 data_object_id, 17 sysdate - n_days, 18 p_segment_access, 19 p_ts# ); 20 commit; 21 end; 22 / Procedure created. SQL> grant execute on set_stat to public; Grant succeeded. SQL> grant select any dictionary to SH; Grant succeeded. Step 4: Create a view used based on COMPRESSION_STAT$ table to retrieve segment compression results after ADO action. Grant select on the view to public and create a public synonym for the view. SQL> CREATE OR REPLACE VIEW user_compression_stats 2 (object_name , subobject_name, avgsize_uncomp, avgsize_disk, 3 nblocks_uncmp, nblocks_oltp, nblocks_ehcc, nrows_uncmp, 4 nrows_oltp, nrows_ehcc) 5 as select b.name, 6 b.subname, 7 a.AVGROWSIZE_NC, 8 a.AVGROWSIZE_C, 9 a.NBLK_NC,
- 3. 10 a.NBLK_ADVANCED, 11 a.NBLK_EHCC, 12 a.NROWS_NC, 13 a.NROWS_ADVANCED, 14 a.NROWS_EHCC 15 from sys.compression_stat$ a, 16 sys.obj$ b 17 where a.obj# = b.obj# and 18 b.owner# = userenv('SCHEMAID'); View created. SQL> GRANT select ON user_compression_stats TO PUBLIC; Grant succeeded. SQL> CREATE OR REPLACE PUBLIC SYNONYM user_compression_stats FOR sys.user_compression_stats; Synonym created. Step 6: Enable heat map tracking, and set the heat map tracking start time back 25 days to ensure statistics logged after this time are valid and considered by Automatic Data Optimization (ADO). SQL> alter system set heat_map=on scope=both; exec dbms_ilm_admin.set_heat_map_start(start_date => sysdate - 20); SQL> create table sales_ado as select * from sales; Table created. SQL> alter table sales_ado move tablespace tstado; Table altered. Populte the table with some more data : SQL> declare sql_test clob; begin for i in 1..7 loop sql_test := 'insert /*+ append */ into sh.sales_ado select * from sh.sales where CUST_ID=3548'; execute immediate sql_test;
- 4. commit; end loop; end; / PL/SQL procedure successfully completed. SQL> select count (*) from sh.sales_ado; COUNT(*) ---------- 920376 Step 7: Verify that heat map tracking collected statistics for the sh.sales_ado table. SQL> select OBJECT_NAME,SEGMENT_WRITE_TIME , SEGMENT_READ_TIME, FULL_SCAN FROM dba_heat_map_segment WHERE OBJECT_NAME='SALES_ADO' AND OWNER = 'SH';SQL> 2 3 4 OBJECT_NAME -------------------------------------------------------------------------------- SEGMENT_WRITE_TIME SEGMENT_READ_TIME FULL_SCAN ------------------ ------------------ ------------------ SALES_ADO 10-oct-13 10:36:25 Check Segment level ----------------------------- col "Segment write" format A14 col "Full Scan" format A12 col "Lookup Scan" format a12 select object_name, track_time "Tracking Time", segment_write "Segment write", full_scan "Full Scan", lookup_scan "Lookup Scan" from DBA_HEAT_MAP_SEG_HISTOGRAM WHERE OBJECT_NAME='SALES_ADO' AND OWNER = 'SH';
- 5. OBJECT_NAME Tracking Time Segment write Full Scan Lookup Scan ------------------ -------------- ------------ ------------ SALES_ADO 10-oct-13 10:37:49 NO YES NO CHECK THE COMPRESSION attribute ------------------------------------------------- SQL> select compression, compress_for from dba_tables where table_name ='SALES_ADO'AND OWNER = 'SH'; COMPRESS COMPRESS_FOR -------- ------------------------------ DISABLED Step 8: SQL> analyze table SH.SALES_ADO compute statistics; Table analyzed. SQL> select object_name, nrows_uncmp, nrows_oltp, nrows_ehcc from sys.user_compression_stats;SQL> no rows selected CREATE COMPRESSION POLICY on table SH.SALES_ADO --------------------------------------------------- SQL> alter table SH.SALES_ADO ILM ADD POLICY ROW STORE COMPRESS ADVANCED SEGMENT AFTER 20 DAYS OF NO MODIFICATION; Table altered. SQL> show user USER is "SH" SQL> select policy_name, action_type, scope, compression_level, condition_type, condition_days from user_ilmdatamovementpolicies order by policy_name; POLICY_NAME ACTION_TYPE SCOPE COMPRESSION_LEVEL CONDITION_TYPE CONDITION_DAYS -------------------------------------------------------------------------------------------------------------------------------------------------- P1 COMPRESSION SEGMENT ADVANCED LAST MODIFICATION TIME 20
- 6. SQL> select policy_name, object_name, inherited_from, enabled from user_ilmobjects; POLICY_NAME OBJECT_NAME INHERITED_FROM ENA -------------------- ------------------------------------------------ P1 SALES_ADO POLICY NOT INHERITED YES Step 9: We are simulating a situation where no modification has been done on SH.SALES_ADO table for the last 20 days. Use the procedure sys.set_stat to fake the passage of time so that compression policy qualifies for ILM action. We are setting heat map statistics clock back by 20 days. As SYS: SQL> conn sys/*** as sysdba Connected. SQL> alter session set nls_date_format='dd-mon-yy hh:mi:ss'; Session altered. SQL> declare 2 v_obj# number; 3 v_dataobj# number; 4 v_ts# number; 5 begin 6 select object_id, data_object_id into v_obj#, v_dataobj# 7 from all_objects 8 where object_name = 'SALES_ADO' 9 and owner = 'SH'; 10 select ts# into v_ts# 11 from sys.ts$ a, 12 dba_segments b 13 where a.name = b.tablespace_name 14 and b.segment_name = 'SALES_ADO'; 15 commit; 16 sys.set_stat 17 (object_id => v_obj#, 18 data_object_id => v_dataobj#, 19 n_days => 20, 20 p_ts# => v_ts#, 21 p_segment_access => 1); 22 end; 23 / PL/SQL procedure successfully completed.
- 7. select object_name, segment_write_time from dba_heat_map_segment where object_name='SALES_ADO'; OBJECT_NAME SEGMENT_WRITE_TIME ------------------------------------------------------------ SALES_ADO 20-sep-13 11:03:46 16 rows selected. SQL> !date Thursday, 10 October 2013 11:05:30 AM EST statistics are showing that the table was last accessed more 20 days back. Step 10: To open and trigger the ADO policies jobs [Note: Not waiting for the maintenance window for testing] ================================================================ SQL> conn sh/sh Connected. SQL> declare v_executionid number; begin dbms_ilm.execute_ILM (ILM_SCOPE => dbms_ilm.SCOPE_SCHEMA, execution_mode => dbms_ilm.ilm_execution_offline, task_id => v_executionid); end; / PL/SQL procedure successfully completed. Step 11: View the results of the job that completed the compression operation. ================================================================ SQL> select task_id, start_time as start_time from user_ilmtasks; TASK_ID ---------- START_TIME --------------------------------------------------------------------------- 2 10-OCT-13 11.09.07.952584 AM
- 8. SQL> SQL> select task_id, job_name, job_state, completion_time completion from user_ilmresults; TASK_ID JOB_NAME JOB_STATE COMPLETION --------------------------------------------------------------------------- 2 ILMJOB2 COMPLETED SUCCESSFULLY 10-OCT-13 11.09.14.826402 AM 1 select task_id, policy_name, object_name, 2 selected_for_execution, job_name 3* from user_ilmevaluationdetails SQL> / TASK_ID POLICY_NAME OBJECT_NAME SELECTED_FOR_EXECUTION ------------------------------------------ JOB_NAME ------------------------------------------------------------------------------------------------- 2 P1 SALES_ADO ILMJOB2 Step 12: Check if ADO triggered compression on SH.SALES_ADO segment. SQL> show user USER is "SH" SQL> analyze table SH.SALES_ADO compute statistics; Table analyzed. SQL> select compression, compress_for FROM user_tables where table_name = 'SALES_ADO'; COMPRESS COMPRESS_FOR -------- ------------------------------ ENABLED ADVANCED SQL> select object_name, nrows_uncmp, nrows_oltp, nrows_ehcc from user_compression_stats; OBJECT_NAME NROWS_UNCMP NROWS_OLTP NROWS_EHCC ----------- ---------- ---------- SALES_ADO 0 918843 0 So finish testing compression policy.
- 9. +++++++++++++++++++++++++ TESTING: TIER POLICY +++++++++++++++++++++++++ CLEANUP Existing Policy Check Current ILM policies on the SH.SALES_ADO table. SQL> select * from user_ilmpolicies; SQL> select * from user_ilmdatamovementpolicies; Delete all ILM Policies on the SH.SALES_ADO table. SQL> alter table SH.SALES_ADO ilm delete_all; NOW CHECK the threshold value ------------------------------------------- SQL> col name format A21 SQL> col value format 9999 SQL> select * from dba_ilmparameters; NAME VALUE --------------------- ----- ENABLED 1 JOB LIMIT 10 EXECUTION MODE 3 EXECUTION INTERVAL 15 TBS PERCENT USED 15 TBS PERCENT FREE 85 6 rows selected. Tablespace Usage Check ---------------------------------- SQL> set linesize 300; SQL> SELECT a.tablespace_name TBS_NAME, ROUND (a.bytes_alloc / 1024 / 1024, 0) MEGS_ALLOC, 2 ROUND (NVL (b.bytes_free, 0) / 1024 / 1024, 0) MEGS_FREE, 3 ROUND ((a.bytes_alloc - NVL (b.bytes_free, 0)) / 1024 / 1024,0) MEGS_USED, 4 ROUND ((NVL (b.bytes_free, 0) / a.bytes_alloc) * 100, 2) PCT_FREE, 5 100 - ROUND ((NVL (b.bytes_free, 0) / a.bytes_alloc) * 100, 2) PCT_USED, 6 ROUND (maxbytes / 1048576, 2) MAX_MEGS_ALLOC, 7 100 - ROUND ((a.bytes_alloc - NVL (b.bytes_free, 0)) / maxbytes * 100, 2) MAX_PCT_FREE,
- 10. 8 ROUND ((a.bytes_alloc - NVL (b.bytes_free, 0)) / maxbytes * 100, 2) MAX_PCT_USED 9 FROM (SELECT f.tablespace_name, SUM (f.BYTES) bytes_alloc, 10 SUM (DECODE (f.autoextensible,'YES',f.maxbytes,'NO', f.BYTES)) maxbytes 11 FROM dba_data_files f 12 GROUP BY tablespace_name) a, 13 (SELECT f.tablespace_name, SUM (f.BYTES) bytes_free 14 FROM dba_free_space f 15 GROUP BY tablespace_name) b 16 WHERE a.tablespace_name = b.tablespace_name(+) 17 ; TBS_NAME MEGS_ALLOC MEGS_FREE MEGS_USED PCT_FREE PCT_USED MAX_MEGS_ALLOC MAX_PCT_FREE MAX_PCT_USED ------------------------------ ---------- ---------- ---------- ---------- ---------- -------------- ------------ ------------ SYSTEM 970 4 966 .37 99.63 32767.98 97.05 2.95 MGMT_AD4J_TS 200 199 1 99.5 .5 32767.98 100 0 TSTADO 400 385 15 96.25 3.75 400 96.25 3.75 SPACE_PRESS 150 149 1 99.33 .67 150 99.33 .67 USERS 40 38 2 95.78 4.22 32767.98 99.99 .01 MGMT_ECM_DEPOT_TS 40 20 20 49.38 50.62 32767.98 99.94 .06 MGMT_TABLESPACE 500 43 457 8.51 91.49 32767.98 98.6 1.4 SYSAUX 830 46 784 5.59 94.41 32767.98 97.61 2.39 EXAMPLE 323 0 323 .02 99.98 32767.98 99.01 .99 UNDOTBS1 330 72 258 21.69 78.31 32767.98 99.21 .79 10 rows selected. Create Policy ----------------------- CREATE A STORAGE TIER POLICY ON TABLE SH.SALES_ADO SQL> ALTER TABLE SH.SALES_ADO ilm ADD POLICY TIER TO SPACE_PRESS; Table altered. SQL> col policy_name format a10 SQL> col tier_tbs format a20 SQL> select POLICY_NAME, ACTION_TYPE, SCOPE, TIER_TABLESPACE 2 from user_ilmdatamovementpolicies 3 order by policy_name; POLICY_NAME ACTION_TYPE SCOPE TIER_TABLESPACE --------------------------------------------------------------------------------- P21 STORAGE SEGMENT SPACE_PRESS
- 11. For testing: Now set the threshold value for TBS_PERCENT_FREE to (97%) and TBS_PERCENT_USED to 3% SQL> conn sys/*** as sysdba Connected. SQL> exec dbms_ilm_admin.customize_ilm(dbms_ilm_admin.tbs_percent_free,97) PL/SQL procedure successfully completed. SQL> exec dbms_ilm_admin.customize_ilm(dbms_ilm_admin.tbs_percent_used,3) PL/SQL procedure successfully completed. SQL> col name format A21 SQL> col value format 9999 SQL> select * from dba_ilmparameters; NAME VALUE --------------------- ----- ENABLED 1 JOB LIMIT 10 EXECUTION MODE 3 EXECUTION INTERVAL 15 TBS PERCENT USED 3 TBS PERCENT FREE 97 6 rows selected. maintenance window to open and trigger the ADO policies jobs [Note: Not waiting for the maintenance window for testing] ================================================================ SQL> conn sh/sh Connected. SQL> declare v_executionid number; begin dbms_ilm.execute_ILM (ILM_SCOPE => dbms_ilm.SCOPE_SCHEMA, execution_mode => dbms_ilm.ilm_execution_offline, task_id => v_executionid); end; / Recheck Tablespace Status ------------------------------------ SYSTEM 970 4 966 .37 99.63 32767.98 97.05 2.95 MGMT_AD4J_TS 200 199 1 99.5 .5 32767.98 100 0 TSTADO 400 399 1 99.75 .25 400 99.75 .25 SPACE_PRESS 150 135 15 90 10 150 90 10
- 12. USERS 40 38 2 95.78 4.22 32767.98 99.99 .01 MGMT_ECM_DEPOT_TS 40 20 20 49.38 50.62 32767.98 99.94 .06 MGMT_TABLESPACE 500 43 457 8.51 91.49 32767.98 98.6 1.4 SYSAUX 830 36 794 4.34 95.66 32767.98 97.58 2.42 EXAMPLE 323 0 323 .02 99.98 32767.98 99.01 .99 UNDOTBS1 330 85 245 25.83 74.17 32767.98 99.25 .75 10 rows selected. So free space is more on TSTADO tablespace and SPACE_PRESS is showing value for column %used. Check the ADO task Status -------------------------------------- SQL> select task_id, state , COMPLETION_TIME from user_ilmtasks; 2 COMPLETED 10-OCT-13 11.09.14.826402 AM 14 COMPLETED 10-OCT-13 01.53.09.689013 PM 17 COMPLETED 10-OCT-13 02.23.14.231013 PM SQL> select table_name, tablespace_name from user_tables 2 where table_name='SALES_ADO'; TABLE_NAME TABLESPACE_NAME ------------------------------------------- SALES_ADO SPACE_PRESS CleanUP ---------------- Delete all ILM Policies on the SH.SALES_ADO table. SQL> alter table SH.SALES_ADO ilm delete_all; Table altered. Verify that there are no ILM policies on the SH.SALES_ADO table. SQL> select * from user_ilmdatamovementpolicies; no rows selected SQL> select * from user_ilmpolicies; no rows selected SQL> alter system set heat_map=off scope=both; System altered.
- 13. Clean up Heat Map Statistics ----------------------------------- SQL> exec dbms_ilm_admin.clear_heat_map_all; PL/SQL procedure successfully completed. SQL> select obj#, ts#, track_time from sys.heat_map_stat$; OBJ# TS# TRACK_TIM ---------- ---------- --------- -1 -1 10-OCT-13 Note: The above value is for the dummy stat. So TIER Policy test has been completed for ADO.