SAP BO and Teradata best practices
9 likes8,800 views
The document provides best practices for using SAP Business Objects with Teradata databases. It discusses ODBC setup, universe design considerations, and leveraging Teradata features within Business Objects. Specific topics covered include using the appropriate ODBC version, setting connection properties, avoiding character set errors, utilizing Teradata functions, and designing universes for standard versus ad-hoc reporting. The goal is to optimize performance and flexibility when using Business Objects with Teradata.
SAP BO and Teradata best practices
- 1. 1 SAP Business Objects and Teradata best practices Dmitry Anoshin
- 2. 2 Introduction / Scope • Scope and Brief Definitions – E6.5 Centric – Desktop Products (BusinessObjects) – Server Products (InfoView, WebIntelligence) – Administration Products (Designer, Supervisor) • Support Protocol • Resources
- 3. 3 ODBC Setup - General • ODBC Versions: – The PAR* is your friend! – Most often it is best to use latest e-fix available (within the major/minor release qualified by Business Objects) – For example: • PAR says ODBC 3.3 • Use 3.3.x.y e-fix Maintenance Minor Major – Any variance from PAR will have support implications • check with BOBJ *Product Availability Report (PAR) via Business Objects Support Website
- 4. 4 ODBC Setup - Designer • Use appropriate Session Character Set – Use a charset supported in the teradata.crs file • Connection Error or Invalid Charset Error avoidance Teradata ODBC 3.3 Advanced Options Screen Teradata ODBC 3.4 Initial Screen
- 5. 5 ODBC Setup – Designer (cont.) • Controlling the selection of tables presented to the user – No Help Database (check) – Use X Views (check) Teradata ODBC 3.4 Options Screen
- 6. 6 ODBC Setup –WebIntelligence Server • Run in Quiet Mode (check) – for 3 tier – unchecked by default • Max Response Buffer Size – TW 8.0: up to 1048575, pre-TW8.0: up to 65477; default 8192 – Experiment • Use appropriate Session Character Set – Determine if you will be using language specific or Unicode based on your needs – E6.5 (Webi) is UNICODE enabled • use UTF8 or UTF16 for UNICODE support • please see UNICODE feature later for additional UNICODE configuration considerations • Caveat (for UNIX-based servers only) – Watch for conflicting ODBC Driver Manager versions if Webi is UNIX-based and multiple ODBC drivers are used (e.g. Data Direct ODBC Driver for SQL Server for repository and a Teradata ODBC driver for Universe / Report Domains)
- 7. 7 ODBC Setup –WebIntelligence Server (cont.) • Session Pooling (Don’t Pool) – Connection Pooling Tab>>Double Click Teradata Driver to disable connection pooling e-fix
- 8. 8 Designer Considerations • Universe Parameters • Advanced Settings – Connection Properties • Keep Connection Alive X minutes – Connection Mode • Asynchronous – Array Fetch Size • Experiment • Teradata Improvements: • PARSE SQL improvement implemented • ODBC SQLExecute
- 9. 9 Universe Parameters Menus: Advanced Universe Parameters>>Definition Tab Edit... (exposes the Teradata ODBC Driver Dialog Box) Advanced Tab •Connection Properties •Connection Mode •Array Fetch Size
- 10. 10 Advanced Parameter Settings:
- 11. 11 Universe Parameter Settings: • New Tab introduced with E6.5 • Feature Details covered later in the presentation – Group By vs. Distinct (LOV) – ANSI 92 (Full Outer Join) – UNICODE strings
- 12. 12 Username / Password considerations • Issues relate to – User traceability and ability to manage ADW via PSF, etc. – Synchronization • One Option: Advanced Login Strategy – Create Teradata User IDs for every user – Use ALS (@BOUSER) / Password blank –OR-- – External application to manage Teradata password expirations
- 13. 13 Teradata Exploitive FUNCTIONS via Designer • CASE • OLAP Functions – CSUM, MAVG, MDIFF, MSUM, PERCENT_RANK, QUANTILE, RANK • SQL99 Functions (aka Ordered Analytics or “window”) – Some implemented – More coming • Almost ANY Teradata function can be utilized – Object Definition – Modification of Parameter files (.PRM) for drop down
- 14. 14 Derived Tables • A Derived Table is similar to a Temporary Table or View with the following characteristics: – Derived tables are one of the options in the FROM clause – The scope of a derived table is only visible to the level of the SELECT statement calling the subquery. • Derived tables are ANSI SQL-99-compliant. • Derived Tables provide for greater expressivity. • A derived table is defined by an SQL query at the universe level that can be used as a logical table in Designer. • Derived tables are similar to database views, with the advantage that the SQL for a derived table can include BusinessObjects prompts.
- 15. 15 Derived Table in Business Objects Benefits • Query Expressivity – Queries can be developed that previously were not possible • Allows more freedom for Universe Designer – Previously Universe Designer would need to request additional tables or views to be created by the DBA – Now the development cycle can be shortened • Potentially reduces Data Provider proliferation in reports providing for greater performance – serial to parallel execution • Prompt-able via Business Objects
- 16. 16 Creating the Derived Table 02/10/2005
- 17. 17 GROUP BY vs. DISTINCT in LOV • Prior to E6.5, List of Value (LOV) generation was performed via a SELECT DISTINCT clause. • Enterprise 6.5 enables the default LOV syntax generation to be toggled to either GROUP BY or DISTINCT on a Universe by Universe basis. • Universe Parameter Setting for DISTINCT_VALUES – values • GROUPBY (note: no space) • DISTINCT
- 18. 18 ANSI 92 Join Syntax • Enables FULL OUTER Join • Teradata – Friendly SQL
- 19. 19 Other Features – Full Client • Teradata Macro Support – Create a data provider based on a Teradata Macro through the Stored Procedure dialog box. – Single result set (no multi-statement requests) – Available via Full Client only • Complex User Defined Objects (UDO) – BusinessObjects full client supports the notion of user-defined objects that allow users to define their own query objects in addition to the objects exposed in universes. With the BusinessObjects 6.5 full client, power users will be able to create user-defined objects using the new CASE function that will translate to an IF THEN ELSE statement at the database level. – Users can create CASE without requesting Universe changes
- 20. 20 Other Teradata Exploitation Areas • Tip: Ensure PI column(s) are used in LOV – Add Description in Result Objects Pane when constructing LOV – Allows users to scan by Description but choose key (PI) for the where clause • Join Elimination via Soft RI since V2R5.0 and above • Teradata Aggregate Join Index Structures • Connections configuration in conjunction with BOBJ ALS can be utilized with a Priority Scheduler implementation – Tie Account String ODBC Parameter to a “Connection” – Users associated with that connection pick up PSF priority
- 21. 21 BOBJ / ALS / PSF – A possible architecture Connection1: User: @Variable(‘BOUSER’) PW: @Variable(‘BOPASS’) DSN: BO_L ODBC DSN BO_L User: PW: Account String: $L Supervisor User Properties Identification Strategy = No Password Checking BOBJ Client1 User: sg1 PW: sg1 WEBI BOBJ Client2 Universe(s) User: sg2 PW: sg2 Connection3: User: @Variable(‘BOUSER’) PW: @Variable(‘BOPASS’) DSN: BO_H Connection2: User: @Variable(‘BOUSER’) PW: @Variable(‘BOPASS’) DSN: BO_M ODBC DSN BO_M User: PW: Account String: $M ODBC DSN BO_H User: PW: Account String: $H
- 22. 22 Considerations • Multiple Data Providers launch serially – Look for Derived Table opportunities and Analytic Functions • Watch for Date / Time / Timestamp – Cast Objects in Designer – Fractional Precision formats not supported in BO – .SBO DateInput format may need to be modified • “ALL” option in LOV – When the user decides NOT to use the ALL literal, the resultant syntax is like: • WHERE (Retail.EMPLOYEE.Name IN ( ‘Barbara Fish’ ) OR ‘ALL’ IN ( ‘Barbara Fish’ ) • Although user selected ‘Barbara Fish’ a FTS will result – think about a filter instead
- 23. 23 Considerations (cont.) • TDQM – Rejected queries will promulgate message back to end user – There is no message sent back to the user for Deferred queries • The behavior is like any long running query • Ensure proper timeout settings on WebIntelligence Server for your installation Reports created in Full Client but launched in Thin Client – Will spawn BOBJ.exe full client on Server to render – Potential for many instances to launch – Net: Try to use Webi-created reports via WebIntelligence
- 24. 24 Resources • BOBJ User Documentation • BOBJ User Forum – http://www.forumtopics.com/busobj/ • BOBJ Support Site – http://www.techsupport.businessobjects.com/ – PAR Reports • Teradata Documentation and Information – http://www.teradata.com/ • Recorded Virtual Class # 29941 "Globalizing Your Teradata Warehouse" • Teradata Forum (TDATA-L) – http://www.teradataforum.com/ • Business Objects Users Conference • Teradata Partners User Group Conference & Expo • Business Objects – Teradata Support Statement of Direction • Integra Solutions Library – http://www.integrasolutions.net/isi_library.htm
- 25. 25 Implementation Considerations • Briefly setting the scene • This section does not cover the best practices associated with managing and conducting successful engagements • The implementation considerations relate to the stages in a Business Objects / Teradata project once the requirements are gathered from the users and a high level design / reporting requirements are signed off • This section covers designing Universes for Teradata • Standard vs. Ah-Hoc Reporting • Data Model Design • Leveraging Teradata effectively
- 26. 26 Designing Universes for Teradata • The key Universe design considerations that make Teradata and Business Objects Universes successful : – Design the right type of universe for the right type of reporting – On the correctly designed data model – With knowledge of Business Objects and Teradata Performance tuning
- 27. 27 What is Standard Reporting? • What is Standard Reporting? And what are its characteristics? • Well defined reports usually containing or relating to KPIs • High volume usage • Accessible by Large users • Aggressive availability based on specific business service levels • Static design with potential for many complex measures (Sales, Stock, LY YTD) • Prepared by batch or refresh on aggregates • Specific analysis depending on user Drill, slice and dice, UDM • Why would customers need it? • To deliver a formalized MIS (Management Information System) type reporting system • To remove duplication / error from report creation • When we would do this type of reporting • This type of reporting is often a core business requirement from IT and is best serviced by standard reporting built on aggregations
- 28. 28 What is Ad-Hoc Reporting? • What is Ad-Hoc Reporting? And what are its characteristics? • Varying definitions in Business and IT but limited to: – 1. V. Limited: Prompting on standard reporting e.g. full history tables – 2. Ad-Hoc on Standard reporting universe e.g. Report Creation for users – 3. Ad-Hoc on 3NF (3rd Normal Form) / Base data model e.g. True data analysis • The requirements that come from the business can be categorized into one or more of these. Implementations should look to leverage the benefits of each of these. The definition of Ad- Hoc that we refer to in these slide are for 2 and 3 above. • Longer run times for queries • Lower service levels • Smaller user base / higher training for report creators (Especially 3) • Why would customers need it? • To deliver the bridge between the Standard reporting and what they really want • Get into detailed data analysis not covered by Standard reporting • When we would do this type of reporting • We do this reporting for well defined additional reporting requirements. • Ad-Hoc universes focused on specific business areas / top n questions
- 29. 29 Standard vs. Adhoc Reporting Summary • In summary – Standard Reporting • Must be driven from Summary Aggregations if to deliver SLAs (Service Level Agreement) on measure complexity (Pre-Joins of V large tables, Full Outer Join requirements ,complex measures requiring multipass) • Tables should be designed for Partitioned Primary Index, Primary Index or to hold relevant amount of data for performance / requirements • Available to masses and prepared ‘Refresh on open’ or batch e.g. BCA (Broad Cast Agent) = The reports are not run against the base Physical Data Model. Instead the tables and universe are tuned to give the best for standard reporting not Ad-Hoc – Ad-Hoc Reporting • Driven from base 3NF, basic Aggregations and standard report aggregations to give optimized response times and flexibility of reporting • Tables should be design for Primary Index, Secondary Index and Aggregate Join Index for known common business questions • Universe should be designed with Aggregate navigation and Derived tables • User goes to universe to create a report every time information is needed = The reports are run against the relevant table (aggregate, Derived or base PDM). Universe is focused to giving the best performance and flexibility on a specific business area
- 30. 30 Standard vs. Adhoc Reporting Summary • Consider the following in order to get things right from a Business Objects / Teradata Perspective – Get the requirement right (Standard / Ad-Hoc / Both) – Get the Data Model Right – Get the Optimisations right (Indexing etc) – Know what Business Objects and Teradata don’t do well without help? How to help Business Objects / Teradata • If these are not considered its unlikely that the solution with be the most optimised for Business Objects / Teradata
- 31. 31 Data Model Design • What is the purpose of the Data model – The core purpose of the base Data Warehouse data model is to ensure that the data is modeled in such a way to ensure that any question can asked against it. This is why the base PDM is always a physicalisation of a 3NF logical model • How is the model affected by the BI (Business Intelligence) tool and the reporting requirements – In order to meet the requirements of reporting performance that can be expected by the business the Data model there are three levels of additional content required within the data model • In fact there are different data models for different uses – A. 3NF with physicalised optimizations – the base Data Warehouse – B. Supporting base aggregations e.g. Day / week / Period – C. Application Data model ADM. • Introducing BI Key concept: 3NF Base PDM ADM
- 32. 32 Building on the Base Data Model.. • Supporting Base Aggregations – Daily, Weekly, Monthly – Enhance querying on time hierarchy – Simple. Aggregated on time hierarchy only – Multiple uses across the Data Warehouse – Candidates includes Sales, Waste, etc
- 33. 33 Building on the Application data model • The ADM is an extension built from the base PDM • Core Purpose: – Deliver a data model optimized for BI Tool – Deliver a data model that delivers SLAs and content to users satisfaction – Provide optimizations for the reporting performance – Re-useable for Ad-Hoc requirements • What does an ADM contain? – Aggregations – Additional Reference (Exploded History, additional relationship tables for report specific data) • What's an ADM shouldn’t contain? – Multi Dimensional Cubes – Total lines within the data – Data not already found in base PDM
- 34. 34 Building on the Application data model • The ADM can be build from any relational model – Star schema built from 3NF PDM – Snowflake schema from 3NF PDM – 3NF PDM subset • Business Objects does not deal with MOLAP (Multidimensional Online Analytical Processing) and relies on the data being represented in a relational and not cubed way • The Application data model for standard reporting is mainly focused around many single aggregates Snowflake or Star schema • The Application data model for Ad-Hoc standard reporting is mainly focused around 3NF PDM and aggregates where appropriate
- 35. 35 Summarizing Data Model and Reporting requirements • These data model considerations are founded from many years and implementations • Approaches that we have seen succeed at large scale customers time and time again • There are many (data model) ways to achieve results in business objects, but be pragmatic and use aggregations to deliver standard reporting (even implemented Kimball reference hierarchy for Business Objects
- 36. 36 Leveraging Teradata & Business Objects Effectively • Regardless of the final data model used and for what purpose the reporting is to satisfy there are several things that need to be focused on ahead of, and during the design – Know and leverage what Business objects has to offer – Know and leverage what Teradata has to offer – Do it in a mutually inclusive way (inexperience or lack of understanding of either will probably result in a sub optimal solution)
- 37. 37 What Teradata provides? • Indexing – Primary Index (PI) – Secondary Index (SI) – Partitioned Primary Index (PPI) – Aggregate Join Index (AJI) • Collecting Statistics – To help Parser to choose the most optimal query path • Full Outer Joins – This is very powerful and efficient functionality within Teradata • Performance Tuning – Understand and Use Teradata Parser and EXPLAIN functionality
- 38. 38 Performance Tuning • What and how to get the tuning right – Proactive • Ensure right level of Aggregates in universe and available for Business Objects to use in query • Make sure PI are all in place • Collect statistics on PI columns • Get the joins right – Reactive to Business Objects universe / report design • Collect stats on query condition and join columns • Extract SQL generated from Business Objects • Run through Teradata using Queryman • Get Explain and identify join plan opportunities (remember certain SQL statements can not be parsed / recreated by Business Objects) • Implement improvements and repeat as required
- 39. 39 Performance Tuning • 1. During Business Objects design get SQL from SQLViewer • 2. Run Explain in Queryman • 3. Check query plan to ensure Business Objects generating best SQL • 4. Do performance Tuning • 5. Implement through Universe • 6. Repeat… 1. 2. 3.
- 40. 40 Leveraging Teradata Functionality within Business Objects • There are key pieces of functionality that should be used within Business Objects in order to get the best from Teradata and the Data Model – Derived Tables – Set ANSI92 parameter – Generates Teradata ‘friendly’ SQL – FULL OUTER JOINS Option (Set ANSI92 parameter to YES) – Aggregate Navigation • @Aggregate_Aware • Incompatibilities within Aggregates • Contexts – CASE Statements – When appropriate Push things that Teradata is very quick / good at to Teradata • E.g. Joins, CASE, Cumulative SUM – Smart use of LOVs
- 41. 41 Example: Business Objects / Teradata Standard Reporting Universe
- 42. 42 1. Set Business Objects to generate ANSI92 SQL • This can be set in the .prn file or at universe parameters screen • Ensures SQL generated is best suited for Teradata • Enables FULL OUTER JOIN syntax
- 43. 43 2. Business Objects Derived Tables • Consider the following when using this powerful extension – Use when need to pre-aggregate lower level fact table to allow join on same key to second fact – Great when need to do complex SQL / Teradata Functions not possible or supported in Business Objects (e.g. nested derived tables, CSUM, Rank) – Ensure use prompts within Derived table definition to ensure unnecessary Full Table scan / spooling
- 44. 44 3. Using Aggregates (i) Contexts • Create Context for each set of Aggregates (use Aliases)
- 45. 45 3. Using Aggregates (ii)@Aggregate_Aware • Create Measures and Dimensions (as appropriate) using @Aggregate_Aware function based on Contexts • Reliant on setting up Aggregate Navigation in Business Objects • Can be used to ensure the SQL runs against the smallest (least Dimensions) aggregate table required to satisfy the query
- 46. 46 3. Using Aggregates (iii) Aggregate navigation (Incompatibilities) • This tells Business Objects which measure can be used with which tables – E.g. Shows that Supplier class objects (on right) are not compatible with the non supplier aggregate (on left) – The @Aggregate_Aware chooses which column to use based on this function
- 47. 47 Universe Design Summary • Summary points from experience and design best practices – Keep universe focused on subject area – This will make it easier to performance tune and maintain – For standard reporting universes split up into Daily / Weekly / Monthly to limit size and complexity over time – Use Aggregate Awareness, Contexts and Aggregate Navigation functionality to get best performing SQL – Don’t create to many aggregates for the sake of reporting. Plan the aggregates needed based on reporting requirements and potential re-use – Don’t use views to do aggregations or joins – The parser will have to join / aggregate before any conditions can be applied – Ensure ANSI92 parameter is set to enable Full Outer joins if needed and make Business Objects SQL compliant with the Industry standard that Teradata is optimized for
- 48. 48 Universe Design Summary • Summary points from experience and design best practices (Continued) – Remember that a pragmatic approach to extending the Data Model for applications is required in the field – Ad-hoc should try to leverage available aggregates as well as the 3NF base PDM – Speaker Summary Comments
- 49. 49 Environment Setup • Goal – Create a structured, reliable environment to support a repeatable development, testing, and production process – Development – Use by developers to create new universes and reports, test new features, and perform maintenance on existing items. – User Acceptance Testing – Available for end users to perform testing activities on items that are wanted in production. Should have access to an appropriate volume of real data. – Production – Stable environment for end users to perform work on a live system.
- 50. 50 Implementation Considerations • Change Control – Set up rigorous change control process. – This will be critical in the long term to maintain a quality product and the faith of the user community. – The process to manage the approval, signoff, and notification of changes must defined, used, and enforced. – Balance is important, change control should not hinder development. • A source version control process/tool needs to be incorporated into the development and maintenance cycles. This will make it easier to recover from potential problems.
- 51. 51 Repository Configuration Options • One Repository many Domains – Create multiple universe and document domains under one security domain. – Benefits – Universes and Reports can be shared across groups of users. Makes central administration manageable – Costs – Complex security structure if not well planned • Multiple Repositories many domains – Create separate repository structures for each environment. – Benefit – Simpler security structure within each user community – Cost – More complex migration procedure • Have successful Teradata implementations using both • KEY: Keep Repository separate from Warehouse for best performance on Large Implementations
- 52. 52 Network Topology Full Client Modules Web Browser WAN/Intranet/Internet WebI 1 WebI 2 WebI n Repository Teradata Data Center Private Gigabit LAN Firewall/Router ...
- 53. 53 Security Hierarchy • Avoid creating a deep security hierarchy. It will become cumbersome to maintain, and will negatively impact repository performance.
- 54. 54 Security Hierarchy • Organize users by access role to maintain a shallow, easy to maintain hierarchy • Assign command restrictions to groups, make sure users have no more access then they need.
- 55. 55 Connections • Environment Installation – Connections – Use @BOUser to pass BO userid as Teradata user id, use secret password. – Users never know the Teradata password, when it needs to be changed, update the connection, then modify all Teradata user ids with the new password. These activities are simple enough and should be performed by a trusted administrator. – To track database usage by user and universe, use • @BOUSER variable plus extension e.g. ‘_MIS’ in connection setup • Teradata Account String Expansion. – Set up the account string in the ODBC DSN. This may require one DSN per universe. Coordinate this with your DBA staff, they may have some usage data collection standard already in place.