Oracle Database features every developer should know about

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Oracle Database Features
Every Developer Should Know About
Gerald Venzl
Senior Principal Product Manager
Oracle Development
August, 2018
@GeraldVenzl

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
2

Join the Conversation
3
Related Videos
• Gerald’s YouTube Channel
• Oracle Database PM Channel
Any Additional Questions
• My emails: gerald.venzl[at]oracle.com
Additional
Resources
https://geraldonit.com
https://twitter.com/GeraldVenzl

Program Agenda
Invisible Columns
Virtual Columns
External Tables
Useful Supplied PL/SQL Packages
How to Find Your Needle in the Haystack
1
2
3
4
4
5

Invisible Columns
5

Invisible columns
Difficulties with the relational model today:
• Different applications may need different columns from a single table
but all applications are impacted by any DDL change on a table
– Dropping a column will break queries still referring to it
– Adding a column breaks “SELECT *” queries
– Dropping a column will delete the data from all the rows, even those that are
already archived
6
Flexible schemas in a relational model
12c

Invisible columns
Difficulties with the relational model today:
• Different applications may need different columns from a single table
but all applications are impacted by any DDL change on a table
– Dropping a column will break queries still referring to it
– Adding a column breaks “SELECT *” queries
– Dropping a column will delete the data from all the rows, even those that are
already archived
• Invisible columns allows you to introduce non-disruptive DDL changes
7
12c

Invisible columns
8
Dropping columns is disruptive
SQL> SELECT first_name, last_name, initials
FROM customers;
FIRST_NAME LAST_NAME IN
---------- ---------- --
George Kolaski GK
Joseph Smith JS
Gerald Teller GT
FROM customers;
---------- ---------- --
George Kolaski GK
Joseph Smith JS
Gerald Teller GT
App A
App B
FIRST_NAME LAST_NAMEINITIALS

Invisible columns
9
FROM customers;
SELECT first_name, last_name, initials
*
ERROR at line 1:
ORA-00904: "INITIALS": invalid identifier
App A
App B
LAST_NAMEFIRST_NAME
ALTER TABLE customers
DROP COLUMN initials;
SQL> SELECT first_name, last_name
FROM customers;
FIRST_NAME LAST_NAME
--------- ----------
George Kolaski
Joseph Smith
Gerald Teller

Invisible columns
10
FROM customers;
--------- ----------
Geroge Kolaski
Joseph Smith
Gerald Teller
App A
App B
FIRST_NAME LAST_NAMEINITIALS
MODIFY initials INVISIBLE;
FROM customers;
---------- ----------
George Kolaski
Joseph Smith
Gerald Teller
FROM customers;
---------- ---------- --
George Kolaski GK
Joseph Smith JS
Gerald Teller GT

Invisible columns
11
Adding columns is disruptive
App A
App C
LAST_NAMEFIRST_NAME
FROM customers;
---------- ----------
George Kolaski
Joseph Smith
Gerald Teller
SQL> SELECT *
FROM customers;
---------- ----------
George Kolaski
Joseph Smith
Gerald Teller

SQL> SELECT *
FROM customers;
FIRST_NAME LAST_NAME DOB
---------- ---------- ---------
George Kolaski 12-SEP-88
Joseph Smith 24-MAR-93
Gerald Teller 02-NOV-69
Invisible columns
12
SQL> SELECT first_name, last_name, dob
FROM customers;
---------- ---------- ---------
App A
App C
LAST_NAMEFIRST_NAME DOB
ADD dob date;

SQL> SELECT *
FROM customers;
---------- ----------
George Kolaski
Joseph Smith
Gerald Teller
Invisible columns
13
App A
App C
ADD dob date INVISIBLE;
FROM customers;
---------- ---------- ---------

Invisible columns
• Invisible columns can be used as partition keys
• Be aware that column order changes when setting columns invisible
• Use SET COLINVISIBLE ON; to see invisible columns in DESC
14
SQL> SET COLINVISIBLE ON;
SQL> DESC CUSTOMERS;
Name Null? Type
----------------- -------- ------------
FIRST_NAME VARCHAR2(10)
LAST_NAME VARCHAR2(10)
DOB (INVISIBLE) DATE

Virtual Columns
15

Virtual columns
• Example: Compute total sales price
• Net = Price + (Price * Tax)
• As expressions get more complex it’s
possible developers will accidentally
define them differently in their code
• This results in inconsistent reporting
• You could change the data model
every time the business determines
a new measure & create a trigger to
populate the data
16
Analytic queries often contain complex expressions
PRODUCT TAXPRICE
Price+Price*Tax
11g

Virtual columns
17
Add column with insert trigger
PRODUCT TAXPRICE
Price+Price*Tax
NET
SQL> ALTER TABLE sales ADD net number
INVISIBLE;
Table Altered.
SQL> UPDATE sales set net =
price+(price*tax);
15 rows updated.
SQL> CREATE OR REPLACE TRIGGER set_net
BEFORE INSERT or UPDATE ON sales
FOR EACH ROW
BEGIN
:new.net := :new.price +
(:new.price * :new.tax);
END;
/
Trigger created.

Virtual columns
18
Add column with insert trigger
PRODUCT TAXPRICE
Price+Price*Tax
NET
SQL> INSERT INTO sales (product, price, tax)
VALUES('Toy Train','5',’0.08');
1 row created.
SQL> COMMIT;
Commit complete.
SQL> SELECT product, price, tax, net
FROM sales;
PRODUCT PRICE TAX NET
---------- ---------- ---------- ----------
Toy Train 5 0.08 5.4

Virtual columns
19
Appear to be normal table columns
PRODUCT TAXPRICE NET
SQL> ALTER TABLE sales ADD net number
INVISIBLE GENERATED ALWAYS AS
(price+(price*tax))VIRTUAL;
Table Altered
SQL> INSERT INTO sales (product, price, tax)
VALUES('Toy Train','5',’0.08');
1 row created.
SQL> COMMIT;
Commit complete.
SQL> SELECT prod, price, tax, net
FROM sales;
---------- ---------- ---------- ----------
Price+Price*Tax

Virtual columns
20
Can be used in where clause
SQL> SELECT product, price, tax, net
FROM sales
WHERE net > 5;
---------- ---------- ---------- ----------
SQL> SELECT * FROM
TABLE(DBMS_XPLAN.DISPLAY_CURSOR());
PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------
Plan hash value: 781590677
-----------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost |
-----------------------------------------------------------------------------
| 0 |SELECT STATEMENT | | 1788 | 228K| 20 |
| 1|TABLE ACCESS FULL | SALES | 1788 | 228K| 20 |
------------------------------------------------------------------------------
Price+Price*Tax

Virtual columns
21
Can be indexed
SQL> SELECT prod, price, tax, net
FROM sales
WHERE net > 5;
PROD PRICE TAX NET
---------- ---------- ---------- ----------
SQL> SELECT * FROM
TABLE(DBMS_XPLAN.DISPLAY_CURSOR());
PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------
Plan hash value: 781590677
------------------------------------------------------------------------------
| Id | Operation | Name | Rows |
-------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1788 |
| 1 | TABLE ACCESS BY INDEX ROWID | SALES | 1788 |
| 2| INDEX RANGE SCAN | IDX_NET | 1788 |
------------------------------------------------------------------------------
CREATE INDEX idx_net
ON sales(net);
Price+Price*Tax

External Tables
22
SALESTABLE(external)
UKPartition GermanyPartition USAPartition
SQL

External tables
• Query flat files as if they were tables in the database
– Provides the full power of SQL over flat files
• Preprocessor capabilities
– Un-compress, pre-format, etc. data before being read by the database
• Enables easy data loading
– Use INSERT INTO … SELECT (incl. DML error logging)
• Query data pump files
– Run queries on last nights backup to confirm it validity
• Query data store on HDFS/Hive
23
The universal out-of-database query interface
9i

External tables
24
Query files as if they were tables
CREATE TABLE customers_ext(
first_name varchar2 (10),
int varchar2 (2),
last_name varchar2 (10),
dob date
)
ORGANIZATION EXTERNAL (
TYPE ORACLE_LOADER
DEFAULT DIRECTORY HOME_DIR
ACCESS PARAMETERS (
RECORDS DELIMITED BY NEWLINE
SKIP 1
FIELDS TERMINATED BY ',' (
first_name char(10),
last_name char(10),
int char(2),
DOB char (10)
DATE_FORMAT DATE 'YYYY-MM-DD')
)
LOCATION ('customers*.csv'))
REJECT LIMIT UNLIMITED;
CREATE DIRECTORY home_dir AS
‘/home/gerald/’;
/home/gerald/customers_1.csv
FName,LName,Initials,DOB
George,Kolaski,GK,1988-09-12
Joseph,Smith,JS,1993-03-24
Gerald,Teller,GT,19691102
:
:

External tables
25
Query files as if they were tables
FROM customers_ext;
---------- ---------- ---------
Geroge Kolaski 12-SEP-88
FROM customers_ext
WHERE dob > TO_DATE('1990','YYYY');
---------- ---------
Joseph Smith
FName,LName,Initials,DOB
George,Kolaski,GK,1988-09-12
Joseph,Smith,JS,1993-03-24
Gerald,Teller,GT,19691102
:
:
/home/gerald/customers_1.csv

External tables
26
Un-compress files on the fly
CREATE TABLE customers_ext_comp(
...
)
TYPE ORACLE_LOADER
ACCESS PARAMETERS (
RECORDS DELIMITED BY NEWLINE
PREPROCESSOR USR_BIN:zcat
SKIP 1
...
)
)
LOCATION ('customers*.csv.gz')
)
/home/gerald/customers_1.csv.gz
�Ycustomers_1.csv�A
�0@�}�2�$�b�R�bz��
��є$��<
�A�Ѧ��r~(/XH�2�尫
��8�~{��t�A�Ѧ��r~
(/XH�2�尫
11g

External tables
27
Un-compress files on the fly
SQL> SELECT int, last_name, dob
FROM customers_ext_comp;
INT LAST_NAME DOB
--- ---------- ---------
GK Kolaski 12-SEP-88
JS Smith 24-MAR-93
GT Teller 02-NOV-69
FROM customers_ext_comp
WHERE first_name COLLATE_BINARY_CI
LIKE 'GE%';
---------- ---------
George Kolaski
Gerald Teller

External tables
28
Easy data loading
SQL> CREATE TABLE customers AS
SELECT * FROM customers_ext_comp;
Table created.
SQL> SELECT first_name, int, last_name, dob
FROM customers;
FIRST_NAME INT LAST_NAME DOB
---------- --- ---------- ---------
Geroge GK Kolaski 12-SEP-88
Joseph JS Smith 24-MAR-93
Gerald GT Teller 02-NOV-69
CUSTOMERS

External tables
29
Easy data export
SQL> CREATE TABLE customers_dump
ORGANIZATION EXTERNAL
(
TYPE ORACLE_DATAPUMP
LOCATION ('customers.dmp')
)
AS SELECT * FROM customers;
Table created.
CUSTOMERS
11g

External tables
30
Easily query your dump files
CREATE TABLE customers_dmp (
first_name varchar2(10),
int varchar2 (2),
last_name varchar2 (10),
dob date
)
ORGANIZATION EXTERNAL
(
TYPE ORACLE_DATAPUMP
LOCATION ('customers.dmp')
);
SQL> SELECT int, dob
FROM customers_dmp
WHERE last_name = 'Teller';
INT DOB
--- ---------
GT 02-NOV-69

External tables
31
Easily query on HDFS/Hive
CREATE TABLE customers_ext(
...
)
TYPE ORACLE_LOADER
DEFAULT DIRECTORY HDFS_DIR
ACCESS PARAMETERS (
RECORDS DELIMITED BY 0X'0A'
PREPROCESSOR
”OSCH_BIN_PATH":'hdfs_stream'
SKIP 1
...
)
)
LOCATION ('osch-20140515719-2786-1')
)
CAT osch-20140515719-2786-1
<?xml version="1.0"?>
<configuration>
<property>
<name>oracle.hadoop.exttab.tableName</name>
<value>SH.SALES_EXT_DIR</value>
</property>
<property>
<name>oracle.hadoop.exttab.dataPaths</name>
<value>/data/s1/*.csv,/data/s2/*.csv</value>
</property>
Requires Oracle SQL Connector for Hadoop
12c

Useful Supplied
PL/SQL Packages
32

DBMS_SESSION.SLEEP()
33
Accessible sleep function within PL/SQL
SQL> set timing on;
SQL> DECLARE
v_start date;
v_end date;
BEGIN
v_start := SYSDATE;
-- Sleep for 10 seconds
dbms_lock.sleep(10);
v_end := SYSDATE;
END;
/
PL/SQL procedure successfully
completed.
Elapsed: 00:00:10.02
• DBMS_LOCK includes other, more
sensitive methods
• Therefore not granted to public
• Requires DBA intervention to get a
accessible sleep function in PL/SQL

DBMS_SESSION.SLEEP()
34
Accessible sleep function within PL/SQL
SQL> set timing on;
SQL> DECLARE
v_start date;
v_end date;
BEGIN
v_start := SYSDATE;
-- Sleep for 10 seconds
DBMS_SESSION.SLEEP(10);
v_end := SYSDATE;
END;
/
PL/SQL procedure successfully
completed.
Elapsed: 00:00:10.02
• Doesn’t require GRANT anymore
– Before required explicit grant on
DBMS_LOCK
– DBMS_SESSION granted to public
• Compatible with DBMS_LOCK.SLEEP
– You can search/replace
• Introduced thanks to the Oracle
community
– https://community.oracle.com/ideas/4852

DBMS_UTILITY.EXPAND_SQL_TEXT
• Views can be a useful way to hide
complexity from developers
• But they can also cause problems
• It's easy to write apparently simple
statements, that result in extremely
complex SQL being sent to the database
• The DBMS_UTILITY.EXPAND_SQL_TEXT
procedure expands references to views,
turning them into subqueries in the
original statement
35
SQL> SELECT * FROM sales_v;
Determining if this really is the right view to use
12c

set serveroutput on
DECLARE
l_clob CLOB;
BEGIN
dbms_utility.expand_sql_text(-
input_sql_text => 'SELECT * FROM sales_v', -
output_sql_text => l_clob);
dbms_output.put_line(l_clob);
END;
/
36

SELECT "A1"."order_id" "ORDER_ID",
"A1"."time_id" "TIME_ID",
"A1"."cust_id" "CUST_ID",
"A1"."prod_id" "PROD_ID"
FROM (SELECT "A3"."order_id" "ORDER_ID",
FROM "SH"."sales" "A3",
"SH"."products" "A2"
WHERE "A3"."prod_id" = "A2"."prod_id") "A1"
37

SELECT "A1"."order_id" "ORDER_ID",
FROM (SELECT "A3"."order_id" "ORDER_ID",
FROM "SH"."sales" "A3",
"SH"."products" "A2"
WHERE "A3"."prod_id" = "A2"."prod_id") "A1"
38

DBMS_ADVANCED_REWRITE
39
Fixing the unfixable SQL statement
• You’ve been told you need to fix a
SQL statement that is no longer
returning results in the correct order
• But you can’t change the application
code directly as that would require
down time
• What do you do?
SQL> SELECT x, y, sum(z) FROM t
GROUP BY x, y;
X Y SUM(Z)
---------- ---------- ----------
1 4 110
2 3 84
Something changes and now you get
GROUP BY x, y;
X Y SUM(Z)
---------- ---------- ----------
2 3 84
1 4 110
10g

40
Step 1
• Check what changed with the
execution plan
ORIGINIAL PLAN
-----------------------------------
| Id | Operation | Name |
-----------------------------------
| 0 | SELECT STATEMENT | |
| 1 | HASH GROUP BY | |
| 2 | TABLE ACCESS FULL| T |
-----------------------------------
NEW PLAN
--------------------------------------
--------------------------------------
| 1 | SORT GROUP BY NOSORT| |
| 2 | INDEX FULL SCAN |T_IDX |
--------------------------------------

41
Step 2
• Check the order of the columns in the
index
SQL> SELECT column_position,
column_name
FROM user_ind_columns
WHERE index_name='T_IDX'
ORDER BY column_position;
COLUMN_POSITION COLUMN
--------------- ------
1 Y
2 X
3 Z
• New index returns the rows in a
different order because its sorted on
column Y rather than X

42
Step 3
• Create a view v that includes an order
by clause so the query results will
always be order based on the values
column x
SQL> CREATE OR REPLACE VIEW v
2 AS
3 SELECT x, y, sum(z) "SUM(Z)"
4 FROM t
5 GROUP BY x, y
6 ORDER BY x, y;
View created.

43
Step 4
• Use the DBMS_ADVANCED_REWRITE procedure to map the original query
to a simple SELECT * FROM v;
SQL> BEGIN
2 sys.dbms_advanced_rewrite.declare_rewrite_equivalence(
3 name => 'DEMO_TIME',
4 source_stmt => ’SELECT x, y, sum(z) FROM t GROUP BY x, y',
5 destination_stmt => 'SELECT * FROM v',
6 validate => FALSE,
7 rewrite_mode => 'TEXT_MATCH');
8 END;
9 /
PL/SQL procedure successfully completed.

44
• Each time our original SQL statement
is issued the
procedure rewrites it to be
SELECT * FROM v;
• Always get the query
output in an order list
GROUP BY x, y;
X Y SUM(Z)
---------- ---------- ----------
1 4 110
2 3 84
-----------------------------------
-----------------------------------
| 1 | VIEW | V |
| 2 | SORT GROUP BY | |
| 3 | INDEX FULL SCAN|T_IDX |
-----------------------------------

Finding YOUR needle in the haystack
• Databases are high concurrency, multi-user data management systems
• A performance issue in your app requires to filter out the noise from
everybody else, but how to do that?
• If only there was a way to clearly identify the different modules and
actions from your application…
46
How to filter the noise from YOUR workload

DBMS_APPLICATION_INFO
• SET_CLIENT_INFO()
– Supply additional information about the client application
• SET_MODULE()
– Set the name of the current application or application module
• SET_ACTION()
– Set the name of the current action within the module
• SET_SESSION_LONGOPS()
– Record the on-going progress of a long running operation in V$SESSION_LONGOPS
47
How to filter the noise from YOUR workload
8i

48
Native APIs - Python
conn = cx_Oracle.connect(
db_user,
db_password,
cx_Oracle.makedsn(db_host, db_port, service_name=db_name))
conn.client_identifier = "Twitter Streamer”
conn.module = "DBLoader”
conn.action = “persist”
cursor = conn.cursor()
cursor.prepare("INSERT INTO TWITTER (tweet) VALUES (:tweet)")

49
Native APIs – Node.js
oracledb.getConnection({
user : db_user,
password : db_password,
connectString : db_url
},
function(err, connection) {
if (err) { console.error(err.message); return; }
connection.clientId = ”Twitter Streamer";
connection.module = ”DBLoader";
connection.action = ”persist";
connection.execute(”INSERT INTO TWITTER (tweet) VALUES (:tweet)",
function(err, result) {
. . .

50
Native APIs – Java
Connection conn = DriverManager.getConnection(dbUrl, dbUser, dbPassword);
conn.setClientInfo("OCSID.CLIENTID", ”Twitter Streamer");
conn.setClientInfo("OCSID.MODULE", ”DBLoader");
conn.setClientInfo("OCSID.ACTION", ”persist");
Statement stmt = conn.createStatement();
stmt.execute("INSERT INTO TWITTER (tweet) VALUES (?)");

51
AWR report

52
Active Monitor

53
Active Monitor – SQL details

Oracle Database features every developer should know about

More Related Content

What's hot (20)

Similar to Oracle Database features every developer should know about (20)

Recently uploaded (20)

Oracle Database features every developer should know about

Editor's Notes