JD Edwards Archiving and Upgrades - a Case Study from DBG
- 1. JD Edwards Archving and Upgrades A Case Study from DBG Joe DiPietro ARCTOOLS
- 2. The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for ARCTOOL’s products remain at the sole discretion of DCSoftware, Inc. © 2015 by DCSoftware, Inc. All Rights Reserved. The material in this presentation may not be redistributed without permission. “Your results may vary…” ARCTOOLS® is a registered trademark of DCSoftware, Inc. ReorgWizard™, JDE Module™ and Archive Integrity™ are trademarks of DCSoftware, Inc. Oracle, Java and JD Edwards are trademarks or registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
- 3. 3 WELCOME! Housekeeping: - Visit us - Pass the hat drawing – biz card or fill out a slip - Show of hands: - JDE? E-Biz? Demantra? DIY? - AS400? Oracle? SQL?
- 4. Agenda • Introductions to DBG and ARCTOOLS • Why purge? • Overview of Archiving in JDE • Why is it soooooooo difficult? • Case studies • Upgrade Case Study - DBG Canada • ARCTOOLS®
- 5. Definitions According to Webster’s • Purge • transitive verb • “to make free of something unwanted” • Archive • noun • “a place where records or historical documents are preserved” • transitive verb • “to file or collect in or as if in an archive” According to ARCTOOLS: Purging is removing records from the production environment. Archiving is saving those records in an archive environment. Purging and archiving together is removing records from production and adding them to the archive. Destruction means permanently removing data from the archive (i.e. “purging from the archive”)
- 6. DBG 6 DBG is a privately owned supplier to the auto and defense industries, manufacturing metal components at 2 plants in Canada and 2 more in Mexico. DBG has been running JDE since 1998.
- 7. DBG Canada • Privately owned manufacturing company • 35 years in business • 2 facilities in Canada, 2 in Mexico, running 6x24 • Customers: truck, auto, military, consumer goods manufacturers • Products: • metal stampings • stainless steel and carbon steel assemblies • design and prototyping
- 8. ARCTOOLS Company History • DCSoftware founded 1996, created ARCTOOLS/400 • David Shea, President • Over 25 years of ERP experience • Six years with a ‘Big Eight’ consultant • Created JDE purge and archive methodology in collaboration with JDE and several customers • World – 2003. E1 on AS400 – 2004 • Created ARCTOOLS/EM (Oracle, SQL Server) 2005 • Barry Yelverton, Chief Product Architect, Oracle/SQL • Locations: Boston, Denver, Paris, Limerick • Over 400 JDE customers on AS400, Oracle, SQL Server 8
- 9. ARCTOOLS and JD Edwards • Early 2001 – JDE customer makes introductions to Denver after a successful custom purge effort • June 2001 – meetings with JDE at Quest Global / Focus • July thru… developed (with JDE) ‘proof of concept’ purges • June 2002 – SRO crowd at Quest Global presentation – over 300 customers • Sept 2002 – development effort in Denver to work with JDE and several large customers • Oct 2002 – beta site – Cargill • Jan 2003 – first “JDE Module” released for World • 2004 – JDE Module for XE on AS/400 • 2005 – Oracle / SQL • OVI for World and EnterpriseOne 9
- 10. ARCTOOLS • Mature, dependable product • Over 400 Oracle JD Edwards customers world-wide • A ‘true database solution’ • Not a UBE. Not ODBC. Processes run on the server, in the native language of the box • Multiple “accelerators” available for out-of-the-box archiving: • JDE, Demantra, Oracle E-Business, “DIY” • Easily tweaked for customized criteria, custom tables • Use “DIY” capabilities for anything else on your server 10
- 11. Why the Purge & Archive Project? • Swamped by data • Ongoing savings and performance enhancement • Upgrade on the horizon – good time to clean house • Legal requirements • Disaster Recovery
- 12. Some Benefits of Archiving • DB Maintenance Savings • Data Storage Savings • Batch processing improvements • Index build times • Quicker Upgrades • Faster backups • DISASTER RECOVERY BENEFITS MULTIPLIED THROUGHOUT OTHER COPIES OF PRODUCTION
- 13. Archiving in JDE • Pre-ARCTOOLS: • JDE built-in purges are few and far between • Cause significant issues – integrity reports, orphaned records • Customer development efforts proved difficult, risky, painful A fairly common issue with most packaged applications. 13
- 14. Why is Purging so Difficult? Data Integrity… • Textbook: • You would think it’s just a matter of headers and details, but you would be wrong… • JDE specific issues: • Cross functional area data relationships (PO, inventory, AP, GL…) • “Batches” in AP, AR, GL • Programs expecting data in other areas (ex: AP inquiry looking for F0911) • JDE integrity reports (ex: AP by Offset) • Complex criteria • Complex scenarios • “Unfortunate realities” in JDE database
- 15. Top-Down Header Child 1 Child 2 Child 2A Child 3 Simple scenario – date, status, etc, criteria from the header. This is what you would think it looks like…. But in reality it’s rarely this easy.
- 16. Bottom-Up Header Child 1 Child 2 Child 2A Child 3 Other Table(s) Complex scenario – multiple criteria across multiple files, cross references with other files… This is far more common, and often far more complex than this.
- 17. Many to Many Very complex scenario – multiple criteria across possibly all files, cross references with other files… many to many relationships… Voucher 1 Payment 1 Payment 2 Voucher 3 Voucher 4 Voucher 6 Voucher 5 Voucher 7 Payment 4 Payment 5 Payment 3 Voided after horizon Voucher 1 F0911
- 18. And Don’t Forget… • All those ‘unfortunate realities’ in the JDE database: • Bad data (converted… self-inflicted… JDE errors) • Duplicate keys (World more so than E1) • Orphan records • Inconsistent field definitions – numeric versus alpha • Inconsistent key sequences – kco/doc/dct?? doc/dct/kco??? • F00165 keys (pipes… truncated leading zeroes… inconsistent sequence, even within the same parent document…..) • Right justified text columns??? • Hooks into F0911… • Doc type? What doc type? Why would you need a doc type? • Moving targets (ex: doc type buried in a random text column… in different positions… for the same document type…) • … etc… etc… etc…
- 19. Case Study – French Opticians Guild • Reduced database maintenance time from 22 to 8 hours per weekend, leading to an average annual saving of > $100k / year • Archived 430 GB of data from production alone • Reduced the production database by 55% • Savings multiplied across environments ARCTOOLS Project Kick Off Regular Scheduling of Purge & Archive Jobs
- 20. Case Study – George P. Johnson During the upgrade to 9.0 data conversion times improved over prior upgrade: • 8.10 Data Conversion: 36 Hours • Purge and archive using ARCTOOLS • 9.0 Data Conversion: Less than 20 Hours “The benefits can be seen not just during the upgrade process, but also on a daily basis by end users and the IT department – with less data in the production environment, end-user queries run more efficiently and database maintenance is simplified.” Brian Bartkowiak Vice President, Information Technology Worldwide at George P. Johnson
- 21. DBG Canada • Privately owned manufacturing company • 35 years in business • 2 facilities in Canada, 2 in Mexico, running 6x24 • Customers: truck, auto, military, consumer goods manufacturers • Products: • metal stampings • stainless steel and carbon steel assemblies • design and prototyping
- 22. JDE at DBG • On JDE since 1998 • Painful tables: • F0911 - > 70 million • F4111 • F3111 • F42199 • … others…
- 23. DBG – Long Running Processes • MRP • Canada: 4 hours • Mexico: 8 hours • Backups • “daily” backup took more than 24 hours • Concerns about restore time and disaster recovery • Restore takes as long (or worse??) • Any backup was at least more than a day old • Long lead time from lightning strike to up and running • What about data updates since backup?
- 24. DBG – 9.1 Upgrade • 8.10 to 9.1 • Also implementing virtualization at the same time • VMWare, server farm, etc. • Virtualized other things, too, not just JDE (domain controllers, etc) • Timeline: • March – new hardware • November – go live • But in July / August, the realization…
- 25. DBG Mid-upgrade Reality Check Data upgrade run time would shut down plants!!! (Four plants, 700 workers…)
- 26. DBG Canada – Upgrade and ARCTOOLS • Without purging, DBG’s upgrade from 8.10 to 9.1 would have shut down plants and idled hundreds of workers • DBG looked at their options and decided to go with archiving. With 15 years of data, they were ready…
- 27. DBG - ARCTOOLS • Archiving decision made in AUGUST • ARCTOOLS installed August 16th. • “Dropped in”, vanilla, with only 10 weeks to go-live upgrade • Decided to just “throw ARCTOOLS at it…”. “followed the script” • Implemented in parallel with on-going upgrade project • Accounts payable, accounts receivable, sales orders, work orders, purchase orders, inventory, general ledger • Significant data reductions • Purge / archive a success • Upgrade successful, on time • Realized performance improvements, DR improvements
- 28. DBG – ARCTOOLS Project Team • Senior analyst – “go figure it out” • Light on the consulting required • More on the project management side of parallel upgrade project • Set a horizon date – ended up at ‘current plus three’
- 29. DBG – Stats… • Driver run times – “minutes” (5…. 30…) • Archive run times – “minutes” (15… 40…) • Throughputs ranging from 2… 10… 40 million rows an hour
- 30. DBG – Benefits of the Purge/Archive • Speed and Run Times – MRP runs each dropped to under an hour (from 4 and 8 hours each previously) • Backup time dropped from more than a day to under two hours • DR improvements • Multiple copies of PD (PY, CRP, other sandboxes) all benefitted • Increased enthusiasm for new version of JDE
- 31. DBG – Upgrade Weekend • Four plants – 700 employees • Shut down JDE on Friday 6pm • Took a backup, restored to new system, ran table conversions • Up and running Monday morning. “We did it all in a weekend, which would have been impossible without ARCTOOLS.” - Julian Wainewright, DBG Canada
- 32. DBG – Going Forward • Nothing formalized yet, but… • Planning to purge every year or two • Have some data issues that were made visible by the purge
- 33. Why Archive Before the Upgrade? “When you’re doing an upgrade project and you unveil something new to people, before they get to ‘hey this is new, this is cool’, they are going to try what they know already. If it’s slower than what they had before, then in the first couple minutes it will kill your project. They haven’t even gotten to the ‘whiz-bang’ stuff. If they try the new system and it’s not faster it can really kill adoption.” - Julian Wainewright, IT Manager, DBG Canada
- 34. Access to the Archive Data • What you do with the archived data is up to you… you can • Destroy it • Back it off to tape • Move it to another platform • Leave it on the system but not configure JDE access to it • Leave it on the system and configure JDE access to “archive only” • Leave it on the system and configure JDE access to “live plus archive”
- 35. JDE Access to Live – E1 Access to Archived data using the Production ‘role’ Live Data PRODUCTION
- 36. JDE Access to Archive – E1 Access to Archived data using the Archive ‘role’ Live Data Archive Data ARCHIVE
- 37. JDE Access to Live + Archive – E1 Access to Archived data using the Union ‘role’ Live Data Archive Data
- 40. Over 400 JD Edwards Customers on 6 Continents
- 41. A Wise Choice!! • JDE Market leader - over ten years, more than 400 JDE customers on six continents • Supports E1 (XE to current), World (a6.2 to current) and co-existence • Runs on IBM i (a.k.a. AS/400), Oracle, SQL • Quick implementation • No expensive additional hardware required, no expensive middleware, no requirement to extract all related master data Advantages of ARCTOOLS®
- 42. Performance!! • A true database solution • NOT a UBE (UBEs could be suh-lowwwwwww….) • Not ODBC • Not a fat client • No JDE services required • Not a proprietary hardware / middleware / compression software solution • ARCTOOLS code generator creates purge/archive programs at run-time in the native language of the platform, and then executes the process on the server, close to the data • Throughputs typically measured in millions of rows per hour Advantages of ARCTOOLS®
- 43. Simplicity!! • No JDE services required • No ASU – we won’t add hundreds of objects to your E1 environment • No ESUs – no hassles with every JDE upgrade • No illogical archive table names – data is archived to original JDE table name – i.e. F4301 is archived to F4301, not FYK57901 • Archive data is not co-mingled with production Advantages of ARCTOOLS®
- 44. Flexibility!!! • Not hard-coded. Completely flexible – add custom tables and custom criteria quickly and easily. • Add F55 tables directly to the related purge process, not as a ‘custom’, ‘one-off’ or ‘orphan’ purge. • Delivered pre-configured for JDE, but can purge any tables on your server inside or outside of JDE. Advantages of ARCTOOLS®
- 45. Easy Access to Archived JDE Data!!! • Data archived to proper table name (i.e. F4301 F4301), not some random custom table name (i.e. F4301 FYK97xxx) • Easy to set up with your existing JDE objects (screens and reports) – not mocked up, look-alike, cloned UBEs (which would require even more mods, ESUs, etc…) Advantages of ARCTOOLS®
- 46. Summarization!!! • Can use JDE summarization for GL Detail F0911 (R09811 / P09811) for the archived data, or; • ARCTOOLS provides custom summarization for F0911 • ARCTOOLS summarization routines are easily customized (versus E1 BSFN, UBE or RPG) • JDE provides no summarization for Cardex F4111… but ARCTOOLS DOES Advantages of ARCTOOLS®
- 47. • Gather control totals, run ‘before’ reports (aging, etc) • Run driver process from JDE Module • Run sanity checks on drivers • Run purge/archive process from ARCTOOLS • Gather control totals, run ‘after’ reports for comparison How to Run a Purge/Archive
- 48. Not Just for JDE… Pre-configured Modules for: • JDE World • JDE E1 • Oracle E-Biz Financials • BPCS • JDA • Demantra • Agile • Maximo • Others in development… ARCTOOLS®
- 49. Questions ?
- 50. For More Information • www.arctools.com • Case Studies • White Papers • Webinars, demo requests, speaking schedules, user groups • www.jdealliance.com • ARCTOOLS and other “best of breed” bolt-ons for JDE • Joe DiPietro • (+1) 508.435.8243 x201 • joe@arctools.com 50
- 51. Go to ARCTOOLS.com for more information @jdearctools facebook.com/ARCTOOLS ARCTOOLS Channel ARCTOOLS Group Joe DiPietro • 508.435.8243 x201 • joe@arctools.com
- 52. Please complete the session evaluation We appreciate your feedback and insight
- 53. Join more than 500 other JD Edwards users at April 12-16, 2015 Mandalay Bay- Las Vegas, NV • Keynote from Lyle Ekdahl, Vice President of JD Edwards • JD Edwards’ Roadmap and Strategy (Including Big Announcements) • Education around JD Edwards with Edge products, macro-level content and high-level strategy • More than 230 Solution Providers across several Oracle Products register now! QuestDirect.org/COLLABORATE