Recipe 14 of Data Warehouse and Business Intelligence - Build a Staging Area for an Oracle Data Warehouse (1)

Single meal
Designing, building, loading and
managing a Staging Area for an Oracle
Data Warehouse.
Meal ready in 20 minutes.
Price: completely free
(without loading products and without writing a line of code)
Recipes of Data Warehouse and Business Intelligence
Chef: Massimo Cenci (part 1)

• The data file to be loaded. As an example, we use the data file that describes the
world's financial markets downloaded from the site iso15022.org.
• A document describing the file structure. The information on its structure are
described in the FAQ on the site.
• The Micro ETL Foundation (MEF), that is, the ecosystem of facilities and processing
modules that will connect all the ingredients to obtain the final plate.
• A text editor
• Command Prompt to access in SQL*Plus to the Oracle Database
Ingredients
Chef: Massimo Cenci

Preparation
1. Decide a code of three characters for the project (for example dwh), open a DOS
prompt and create a folder with the same name as the project (for example d:
projectsdwh).
2. Download MEF (from https://drive.google.com/open?
id=0B2dQ0EtjqAOTQzZSaUlyUmxpT1k) and unpack it in the folder dwh.
3. Go under the folder dwhmef and open the mef_def.sql. In this configuration file you
must set some environment variables needed to the Data Warehouse environment
that will be created: the ones you see below, are just an example for Windows. For
the moment only change the red variables according to your environment. (Part 2 of
this article will provide more details)
Chef: Massimo Cenci

Preparation
define prj = 'dwh' -- Project code
define usr = 'etl' -- ETL user name
define pwd = 'etl' -- ETL user password
define frm = 20120101 -- Start day for day dimension table
define ndy = 3660 -- Num days
define dla = 'ITA' -- Default language for descriptions
define dev = '?' -- Default value for varchar2
define den = 0 -- Default value for number
define ded = '99991231' -- Default value for day
define srv = '10.84.253.61' -- Email server (name or ip)
define fss = '' -- File system separator (Windows)
--define fss = '/' -- File system separator (Linux)
define prt = 'd:projects' -- Physical path of the project root (Windows)
--define prt = '/u01/projects' -- Physical path of the project root (Linux)
Chef: Massimo Cenci

Preparation
4. Open a dos prompt under dwhmef, connect to Oracle as sysdba and run the
script mef_install.sql. This installs the MEF ecosystem and will create the etl
user
E:projectsdwhmef> sqlplus / as sysdba
SQL*Plus: Release 12.1.0.1.0 Production on Mon Feb 8 09:29:16 2016
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit
Production
With the Partitioning, OLAP, Advanced Analytics and Real Application
Testing options
SQL> @mef_install
Chef: Massimo Cenci

Preparation
5. Open with an editor the mef config.sql script and change the email addresses of
Massimo Cenci with yours.
6. Connect to the database with the etl user and install the core environment by
running the script mef_core_install.sql
E:projectsdwhmef> sqlplus etl/etl
Connected to:
Production
Testing options
SQL> @mef_core_install
Chef: Massimo Cenci

Preparation
7. Install the staging environment through the script mef_sta_install.sql.
Connected to:
Production
Testing options
SQL> @mef_sta_install
Chef: Massimo Cenci

Preparation
8. Run the configuration script for the data file that we will code as MIC. The
download of the data file and the information about its structure have already
been setted.
Connected to:
Production
Testing options
SQL> @sta_conf_io.sql MIC
Chef: Massimo Cenci

Cooking
1. Run the job to load the data file
Connected to:
Production
Testing options
SQL> exec mef_job.p_run('sta_iso_mic');
Chef: Massimo Cenci

Tasting
• Verify that the Staging Area table is loaded. We can use, for example, SQLDeveloper
Chef: Massimo Cenci

Tasting
• Check that the loading was successful and to have received the result also via email.
Chef: Massimo Cenci

Conclusion
• With the Micro ETL Foundation, you can design, create and manage a Staging Area
using just a text editor and the database internal language.
• The loading of a simple data file does not require more than 20 minutes. We verify the
times:
• Time to configure the features of the data file: 5 minutes
• Time to configure the structure of the data file: 15 minutes
• Running time: 3 seconds (see log)
• Total time: 20 minutes
Chef: Massimo Cenci

Recipe 14 of Data Warehouse and Business Intelligence - Build a Staging Area for an Oracle Data Warehouse (1)

More Related Content

What's hot (20)

Similar to Recipe 14 of Data Warehouse and Business Intelligence - Build a Staging Area for an Oracle Data Warehouse (1) (20)

More from Massimo Cenci (16)

Recently uploaded (20)

Recipe 14 of Data Warehouse and Business Intelligence - Build a Staging Area for an Oracle Data Warehouse (1)