Data stage scenario design 2 - job1
1 like578 views
This document describes a DataStage job design to solve a scenario problem. The design includes: 1) A job with a seq file input, aggregator stage to count rows by key, and filter stage to output rows by count. 2) The aggregator stage groups data by the "No" column and calculates row counts for each key. 3) The filter stage outputs rows where count equals 1 and where count is greater than 1 to separate files.
Data stage scenario design 2 - job1
- 1. Something about DataStage, DataStage Administration, Job Designing,Developing, DataStage troubleshooting, DataStage Installation & Configuration, ETL, DataWareHousing, DB2, Teradata, Oracle and Scripting. Nuts & Bolts of DataStage Home Interview Questions DataStage Scenarios Series Posts EBooks About Me !! Monday, February 03, 2014 DataStage Scenario Problem > DataStage Scenario Problem2 Solution Design : a) Job Design : Below is the design which can achieve the output as we needed. Here, we are reading seq file as a input, then data is passing through Aggregator and Filter stage to achieve the output. DataStage Scenario Design 2 job1 Total Pageviews 1 4 5 4 6 1 1 Search Try Me DataSet in DataStage Issuing commands to a Queue Manager (runmqsc) Hash Files in DataStage XMeta DB : Datastage Repository InfoSphere DataStage Jobstatus returned Codes from dsjob Conductor Node in Datastage Schema File in Datastage Sort stage to remove duplicate 14 Good design tips in Datastage Datastage Coding Checklist Must Reads 1 More Next Blog» Create Blog Sign In
- 2. b) Aggregator Stage Properties Input data contains only one column "No" , In Aggregator stage, we have group the data on the "No" column and calculate the rows for each Key ( No ). When we have used the "Count Rows" aggregation type, it will generate a new column which contain the count for each Key (No). Here we have given the column name "count" and assigned to output as below. c) Filter Stage Properties In Filter stage, we put 2 where condition count=1 and count>1. and assigned different output files to both conditions. Get daily dose of Tech Food Email address... Submit DataStage4You 111 have us in circles View all Follow tech foodies ▼ 2014 (103) ► October (7) ► September (9) ► August (5) ► July (12) ► June (10) ► May (13) ► April (10) ► March (9) ▼ February (16) Check whether DataStage Job is Multi Instance or n... Get DataStage Job Information without using Direct... How to find the patch History of IBM Infosphere Da... DataWareHouse (ETL) Designing Steps Dimension Table and Its Type in Data WareHouse Datastage Coding Checklist List of strong points for InterView :) DataStage Parallel job: Retrieve sql codes Blog Archive
- 3. Assigned the data ( column No ) to output tab. d) Output File We got two output from the jobs i) Contains where count=1 ( unique values in input ) ii) Contains where count>1 ( dups values in input ) on a fa... Interview Questions : DataWareHouse Part 4 DataStage Error : Loop final value not numeric c... How to find Agents Ports in IBM InfoSphere Server ... DataStage Scenario Design2 job3 How to find the product installed with IBM Infosph... DataStage Scenario Design2 job2 DataStage Scenario Design 2 job1 DataStage Scenario Problem16 ► January (12) ► 2013 (167) ► 2012 (175) ► 2011 (8) Administration application authorities client Code column commands Concept Configuration create Data database DataSet DataStage DataWareHouse DB2 DBMS debug delete design develop difference director Documentation dsenv dsjob DSRPC environment Errors ETL file function Information input install Interview Job keys Link Linux list Logging Logical logs lookup managers message queue Metadata Model MQ names Tags Cloud &PH& 421 advantage Agents aggregator Answers architecture ASB attribute backup basic binary block books Buffer certification change channel checkpoint cleanup clear Column Generator compiler Conceptual conductor container copy counter Crontab deadlock deploy dimension Dimensional DSparam dump duplicate encrypt engine exception execution export fact factless FAQ FileSet filter free ftp fun fundamentals granularity Guest hadoop handling hash head hide horizontal Host huge hyperlink import increase index issue istool Java jdbc join leaders listener load local locks Login macro mail maintenance memory merge modify Monitor MQSC multiple
- 4. Newer Post Older PostHome Subscribe to: Post Comments (Atom) By ETL DataStage at 23:38 0 Comments Labels: aggregator, DataStage, design, develop, filter, Job, problem, scenario, Seq File 0 Comments DataStage4You Login Sort by Best Share ⤤ Start the discussion… Be the first to comment. Subscribe✉ Add Disqus to your sited Privacy Favorite ★ Optimizing Oracle output Parallel parameter partition performance Physical port problem process Project Putty Questions remove routine rows scenario Schema Script Seq File sequence Server Service Setting Shell shell scripting sort source SQL stages Start Stop surrogate table target teradata tips tool transformer Troubleshoot Tutorial Unix User Utility UV variables warnings WAS websphere windows XMETA NLS node notes notification odbc odbc.ini operator orchadmin ORLogging orphan OS osh package Parallelism password peek Perl phantom pivot player Practices profile programming purge read registry reject release report Resource Restart Roles row generator RTLogging run sample SCD scheduler score Scratch section session Share shortcuts show slowly snowflake solution space SSH Standards Star statistics status storage switch system tail temporary time trace transformation trigger tuning type unique uvodbc.config version videos view Vincent McBurney Virtual write Write Range Map xml z/OS The postings on this site are my own and don't necessarily represent IBM's or other companies positions, strategies or opinions. All content provided on this blog is for informational purposes only. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. The owner will not be liable for any errors or omissions in this information nor for the availability of this information. The owner will not be liable for any losses, injuries, or damages from the display or use of his information. // Disclaimer