Teradata utilities-fast export

Teradata Utilities: FastExport
Reprinted for KV Satish Kumar, IBM
kvskumar@in.ibm.com
Reprinted with permission as a subscription benefit of Books24x7,
http://www.books24x7.com/

Table of Contents
Chapter 6: FastExport.....................................................................................................................1
Why it is Called "FAST" Export..............................................................................................1
How FastExport Works....................................................................................................1
FastExport Fundamentals......................................................................................................2
FastExport Supported Operating Systems.......................................................................2
Maximum of 15 Loads............................................................................................................3
FastExport Support and Task Commands.............................................................................3
Support Environment Commands..........................................................................................3
Task Commands....................................................................................................................4
FastExport Supported SQL Commands................................................................................5
SQL Commands...............................................................................................................5
A FastExport in its Simplest Form..........................................................................................5
Sample FastExport Script......................................................................................................6
FastExport Modes and Formats.............................................................................................7
Modes..............................................................................................................................7
Formats............................................................................................................................8
A FastExport Script using Binary Mode.................................................................................8
i

Chapter 6: FastExport
"An invasion of armies can be resisted, but not an idea whose time has come."
- Victor Hugo
Why it is Called "FAST" Export
FastExport is known for its lightning speed when it comes to exporting vast amounts of data from
Teradata and transferring the data into flat files on either a mainframe or network-attached
computer. In addition, FastExport has the ability to use OUTMOD routines, which provide the user
the capability to write, select, validate, and preprocess the exported data. Part of this speed is
achieved because FastExport takes full advantage of Teradata's parallelism.
In this book, we have already discovered how BTEQ can be utilized to export data from Teradata in
a variety of formats. As the demand increases to store data, the ever-growing requirement for tools
to export massive amounts of data also increases.
This is the reason why FastExport (FEXP) is brilliant by design. A good rule of thumb is that if you
have more than half a million rows of data to export to either a flat file format or with NULL
indicators, then FastExport is the best choice to accomplish this task.
Keep in mind that FastExport is designed as a one-way utility — that is, the sole purpose of
FastExport is to move data out of Teradata. It does this by harnessing the parallelism that Teradata
provides.
FastExport is extremely attractive for exporting data because it takes full advantage of multiple
sessions, which leverages Teradata parallelism. By default, Fastexport will take up 4 sessions that
will be running on the local computer. FastExport can also export from multiple tables during a
single operation. In addition, FastExport utilizes the Support Environment, which provides a job
restart capability from a checkpoint if an error occurs during the process of executing an export job.
How FastExport Works
When FastExport is invoked, the utility logs onto the Teradata database and retrieves the rows that
are specified in the SELECT statement and puts them into SPOOL. From there, it must build blocks
to send back to the client. In comparison, BTEQ starts sending rows immediately for storage into a
file.
If the output data is sorted, FastExport may be required to redistribute the selected data two times
across the AMP processors in order to build the blocks in the correct sequence. Remember, a lot of
rows fit into a 64K block and both the rows and the blocks must be sequenced. While all of this
redistribution is occurring, BTEQ continues to send rows. FastExport is getting behind in the
processing. However, when FastExport starts sending the rows back a block at a time, it quickly
overtakes and passes BTEQ's row at time processing.
The other advantage is that if BTEQ terminates abnormally, all of your rows (which are in SPOOL)
are discarded. You must rerun the BTEQ script from the beginning. However, if FastExport
terminates abnormally, all the selected rows are in worktables and it can continue sending them
where it left off in a very smart and very fast manner!
Also, if there is a requirement to manipulate the data before storing it on the computer's hard drive,
an OUTMOD routine can be written to modify the result set after it is sent back to the client on either
the mainframe or LAN. Just like the BASF commercial states, "We don't make the products you buy,
we make the products you buy better". FastExport is designed off the same premise, it does not
make the SQL SELECT statement faster, but it does take the SQL SELECT statement and
processes the request with lighting fast parallel processing!
Reprinted for ibmkvskumar@in.ibm.com, IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

FastExport Fundamentals
#1: FastExport EXPORTS data from Teradata. The reason they call it FastExport is because it
takes data off of Teradata (Exports Data). FastExport does not import data into Teradata.
Additionally, like BTEQ it can output multiple files in a single run.
#2: FastExport only supports the SELECT statement. The only DML statement that FastExport
understands is SELECT. You SELECT the data you want exported and FastExport will take care of
the rest.
#3: Choose FastExport over BTEQ when Exporting Data of more than half a million+ rows.
When a large amount of data is being exported, FastExport is recommended over BTEQ Export.
The only drawback is the total number of FastLoads, FastExports, and MultiLoads that can run at
the same time, which is limited to 15. BTEQ Export does not have this restriction. Of course,
FastExport will work with less data, but the speed may not be much faster than BTEQ.
#4: FastExport supports multiple SELECT statements and multiple tables in a single run. You
can have multiple SELECT statements with FastExport and each SELECT can join information up
to 64 tables.
#5: FastExport supports conditional logic, conditional expressions, arithmetic calculations,
and data conversions. FastExport is flexible and supports the above conditions, calculations, and
conversions.
#6: FastExport does NOT support error files or error limits. FastExport does not record
particular error types in a table. The FastExport utility will terminate after a certain number of errors
have been encountered.
#7: FastExport supports user-written routines INMODs and OUTMODs. FastExport allows you
write INMOD and OUTMOD routines so you can select, validate and preprocess the exported data
FastExport Supported Operating Systems
The FastExport utility is supported on either the mainframe or on LAN. The information below
illustrates which operating systems are supported for each environment:
The LAN environment supports the following Operating Systems:
UNIX MP-RAS•
Windows 2000•
Windows 95/98/XP•
Windows NT/2000•
UNIX HP-UX•
AIX•
Solaris SPARC•
Solaris Intel•
Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 2

The Mainframe (Channel Attached) environment supports the following Operating Systems:
MVS•
VM•
Maximum of 15 Loads
The Teradata RDBMS will only support a maximum of 15 simultaneous FastLoad, MultiLoad, or
FastExport utility jobs. This maximum value is determined and configured in the DBS Control
record. This value can be set from 0 to 15. When Teradata is initially installed, this value is set at 5.
The reason for this limitation is that FastLoad, MultiLoad, and FastExport all use large blocks to
transfer data. If more then 15 simultaneous jobs were supported, a saturation point could be
reached on the availability of resources. In this case, Teradata does an excellent job of protecting
system resources by queuing up additional FastLoad, MultiLoad, and FastExport jobs that are
attempting to connect.
For example, if the maximum number of utilities on the Teradata system is reached and another job
attempts to run, that job will not start. This limitation should be viewed as a safety control feature. A
tip for remembering how the load limit applies is this, "If the name of the load utility contains either
the word "Fast" or the word "Load", then there can be only a total of fifteen of them running at any
one time".
BTEQ does not have this load limitation. FastExport is clearly the better choice when exporting
data. However, if two many load jobs are running. BTEQ is an alternate choice for exporting data.
FastExport Support and Task Commands
FastExport accepts both FastExport commands and a subset of SQL statements. The FastExport
commands can be broken down into support and task activities. The table below highlights the key
FastExport commands and their definitions. These commands provide flexibility and control during
the export process.
Support Environment Commands
(see Support Environment chapter for details)

ACCEPT Allows the value of utility variables to be accepted directly from a file or from
environmental variables.
DATEFORM Specifies the style of the DATE data types for FastExport.
DISPLAY Writes messages to the specific location.
ELSE Used in conjunction with the IF statement. ELSE commands and statements will
execute when a proceeding IF condition is false.
ENDIF Used in conjunction with the IF or ELSE statements. Delimits the commands that
were subject to previous IF or ELSE conditions.
IF Introduces a conditional expression. If true then execution of subsequent
commands will happen.
LOGOFF Disconnects all FastExport active sessions and terminates FastExport.
LOGON LOGON command or string used to connect sessions established through the
FastExport utility.
LOGTABLE FastExport utilizes this to specify a restart log table. The purpose is for FastExport
checkpoint information.
ROUTE
MESSAGES
Will route FastExport messages to an alternate destination.
RUN FILE Used to point to a file that FastExport is to use as standard input. This will Invoke
the specified external file as the current source of utility and Teradata SQL
commands.
SET Assigns a data type and value to a variable.
SYSTEM Suspends the FastExport utility temporarily and executes any valid local operating
system command before returning.
Figure 3-1
Task Commands
BEGIN EXPORT Begins the export task and sets the specifications for the number of sessions
with Teradata.
END EXPORT Ends the export task and initiates processing by Teradata.
EXPORT Provides two things which are:
The client destination and file format specifications for the export data
retrieved from Teradata
•
A generated MultiLoad script file that can be used later to reload the
export data back into Teradata
•
FIELD Constitutes a field in the input record section that provides data values for the
SELECT statement.
FILLER Specifies a field in the input record that will not be sent to Teradata for
processing. It is part of the input record to provide data values for the SELECT
statement.
IMPORT Defines the file that provides the USING data values for the SELECT.
LAYOUT Specifies the data layout for a file. It contains a sequence of FIELD and
FILLER commands. This is used to describe the import file that can optionally
provide data values for the SELECT.
Figure 3-2

FastExport Supported SQL Commands
FastExport accepts the following Teradata SQL statements. Each has been placed in alphabetic
order for your convenience.
SQL Commands
ALTER TABLE Change a column or table options of a table.
CHECKPOINT Add a checkpoint entry in the journal table.
COLLECT STATISTICS Collect statistics for one or more columns or indexes in a
table.
COMMENT Store or retrieve a comment string for a particular object.
CREATE DATABASE Creates a new database.
CREATE TABLE Creates a new table.
CREATE VIEW Creates a new view.
CREATE MACRO Creates a new macro.
DATABASE Specify a default database for the session.
DELETE Delete rows from a table.
DELETE DATABASE Removes all tables, views, macros, and stored procedures
from a database.
DROP DATABASE Drops a database.
GIVE Transfer ownership of a database or user to another user.
GRANT Grant access privileges to an object.
MODIFY DATABASE Change the options for a database.
RENAME Change the name of a table, view, or macro.
REPLACE MACRO Change a macro.
REPLACE VIEW Change a view.
REVOKE Revoke privileges to an object.
SET SESSION
COLLATION
Override the collation specification during the current
session.
UPDATE Change a column value of an existing row or rows in a
table.
Figure 3-3
A FastExport in its Simplest Form
The hobby of racecar driving can be extremely frustrating, challenging, and rewarding all at the
same time. I always remember my driving instructor coaching me during a practice session in a new
car around a road course racetrack. He said to me, "Before you can learn to run, you need to learn
how to walk." This same philosophy can be applied when working with FastExport. If FastExport is
broken into steps, then several things that appear to be complicated are really very simple. With this
being stated, FastExport can be broken into the following steps:
Logging onto Teradata•
Retrieves the rows you specify in your SELECT statement•

Exports the data to the specified file or OUTMOD routine•
Logs off of Teradata•
/* Created by CoffingDW */
/* Setup the Fast Export Parameters */
.LOGTABLE sql01.SWA_Log;
Creates the logtable -Required
.LOGON CDW/sql01,whynot;
Logon to Teradata
.BEGIN EXPORT SESSIONS 12;
Begin the Export and set the
number of sessions on
Teradata
.EXPORT OUTFILE Student.txt
MODE RECORD FORMAT TEXT;
Defines the output file name.
In addition, specifies the
output mode and format (LAN
- ONLY)
/* Get Data From the Student Table */
SELECT
Student_ID (CHAR(11)),
Last_name (CHAR(20)),
First_name (CHAR(14)),
Class_code (CHAR(2)),
Grade_Pt (CHAR(9))
FROM SQL_CLASS.Student_Table;
The SELECT defines the
columns used to create the
exported file.
NOTE: The selected columns
for the export are being
converted to character types.
This will simplify the
importing process into a
different database.
/* Finish the Export Job and Write to File */
.END EXPORT;
.LOGOFF;
End the Export and logoff
Teradata.
Figure 3-4
Sample FastExport Script
Now that the first steps have been taken to understand FastExport, the next step is to journey
forward and review another example that shows builds upon what we have learned. In the script
below, Teradata comment lines have been placed inside the script [/*. . . . */]. In addition, FastExport
and SQL commands are written in upper case in order to highlight them. Another note is that the
column names are listed vertically. The recommendation is to place the comma separator in front of
the following column. Coding this way makes reading or debugging the script easier to accomplish.

/* ---------------------------------------------------*/
/*@(#) FASTEXPORT SCRIPT */
/*@(#) Version 1.1 */
/*@(#) Created by CoffingDW */
/* ---------------------------------------------------*/
ALWAYS
GOOD TO
IDENTIFY THE
SCRIPT AND
AUTHOR IN
COMMENTS
.LOGTABLE SQL01.CDW_Log;
.LOGON CDW/SQL01,whynot;
CREATE
LOGTABLE
AND LOGON;
.BEGIN EXPORT
SESSIONS 12;
BEGIN
EXPORT
STATEMENT.
SESSIONS 12;
.EXPORT OUTFILE Join_Export.txt
MODE RECORD FORMAT TEXT;
DEFINES THE
OUTPUT FILE
NAME. IN
ADDITION,
SPECIFIES
THE OUTPUT
MODE AND
FORMAT(LAN
- ONLY) MODE
RECORD
FORMAT
TEXT;
/* Join Data From the Employee and Department Table */
SELECT EMP.Employee_No (CHAR(11))
,EMP.First_Name (CHAR(14))
,EMP.Last_Name (CHAR(20))
,DEPT.Dept_No (CHAR(6))
,DEPT.Dept_name (CHAR(20))
FROM SQL_CLASS.Employee_table AS EMP
INNER JOIN SQL_CLASS.Department_Table AS DEPT
ON EMP.Dept_No = DEPT.Dept_No ;
THE SELECT
PULLS DATA
FROM TWO
TABLES. IT IS
GOOD TO
QUALILY
WHEN DOING
A TWO-TABLE
JOIN.
.END EXPORT;
.LOGOFF;
END THE JOB
AND LOGOFF
TERADATA;
Figure 3-5
FastExport Modes and Formats
Modes
FastExport has two modes: RECORD or INDICATOR. In the mainframe world, only use RECORD
mode. In the UNIX or LAN environment, INDICATOR mode is the default, but you can use
RECORD mode if desired. The difference between the two modes is INDICATOR mode will set the
indicator bits to 1 for column values containing NULLS.

Both modes return data in a client internal format with variable-length records. Each individual
record has a value for all of the columns specified by the SELECT statement. All variable-length
columns are preceded by a two-byte control value indicating the length of the column data. NULL
columns have a value that is appropriate for the column data type. Remember, INDICATOR mode
will set bit flags that identify the columns that have a null value.
Formats
FastExport has many possible formats in the UNIX or LAN environment. The FORMAT statement
specifies the format for each record being exported which are:
FASTLOAD•
BINARY•
TEXT•
UNFORMAT•
The default FORMAT is FASTLOAD in a UNIX or LAN environment.
FASTLOAD format has a two-byte integer, followed by the data, followed by an end-of-record
marker. It is called FASTLOAD because the data is exported in a format ready for FASTLOAD.
BINARY format is a two-byte integer, followed by data.
TEXT format is an arbitrary number of bytes followed by an end-of-record marker.
UNFORMAT format is exactly as received from CLIv2 without any client modifications.
A FastExport Script using Binary Mode
/*-------------------------------------------------- */
/*@(#)FASTEXPORT SCRIPT - SWA */
/*@(#)Version 1.1 */
/*@(#)Created by CoffingDW */
/*-------------------------------------------------- */
COMMENTS
.LOGTABLE SQL01.SWA_LOG;
.LOGON CDW/sql01,whynot;
CREATE LOGTABLE AND
LOGON TO TERADATA;
.BEGIN EXPORT
SESSIONS 12;
BEGIN EXPORT
STATEMENT;
.EXPORT OUTFILE Cdw_import.txt
FORMAT BINARY;
NAME THE OUTPUT FILE
AND SET THE FORMAT TO
BINARY;
/* Get Data From the Student Table */
SELECT
Student_ID (CHAR(11)),
GET THE DATA FROM THE
STUDENT TABLE;

Last_name (CHAR(20)),
First_name (CHAR(14)),
Class_code (CHAR(2)),
Grade_Pt (CHAR(8))
FROM SQL_CLASS.Student_Table;
.END EXPORT;
.LOGOFF;
END THE JOB
Figure 3-6

Teradata utilities-fast export

More Related Content

What's hot (12)

Similar to Teradata utilities-fast export (7)

Recently uploaded (20)

Teradata utilities-fast export