Audit your existing code in Domino - Collabsphere2022_v5.pdf
- 1. Audit your existing code in Domino (just with Lotuscript) Dominique Perarnaud – Data 101
- 3. 3 • Introduction • The customer • The problem to solve > know your code, answer questions • Guidelines from our customer • How to do it • Next steps… • Questions. Agenda
- 4. Dominique Perarnaud • Citizen developer / Domino Administrator / In the Yellow bubble since 2001 • Partner at • . • French in the Basque country, North of Spain, for 20 years • I like to build things 2021-2022
- 5. Javier Sánchez Oliva javiersanchezoliva.com @javdev84 Domino/Notes Lead Developer
- 6. 6 • We are Data 101, HCL Business Partner based in San Sebastian, Spain
- 7. Donostia / San Sebastian: The Bay and the fine food… 3rd industrial region of Spain Biggest concentration of 3* Michelin Restaurants in Europe
- 8. Audit your existing code in Domino (just with Lotuscript) Dominique Perarnaud – Data 101 8
- 9. THE CONTEXT Disclaimer: This is a “Work in Progress” application, we want to show the potential. Third party products are mentionned here, only to add value to the context of this real use case. In any moment, it is a promotion/ads of these tools. The Audit tool in itself could be used without them, but mentionning how it integrate them, add value to the Audit and to this presentation. 9
- 10. • City Council > all the data of the citizens must remain on-Premises • Mantra: “Our citizens are our customers” • Must respect the “Spanish National Framework for Security” (Real Decreto 311/2022) • Domino shop already using third party products The customer 10
- 11. • HCL Domino 12.0.1 FP1 • Installation based on separated Dev / Pre production / Production server • 1 unique server for development (mainly…) • Build Manager partially implemented for pushing changes to pre/production • All type of code: @formula, Lotuscript, Java, Web services, call to external API • Traditional Notes applications co-existing with pure Java applications/server Environment 11
- 12. THE PROBLEM TO SOLVE Blindness… but not only The “Why” 12
- 13. • Accurate and update overview of all database designs / server. • Document the database design. • Know dependencies between applications (getview/getdatabase) • Know use of generic Script Library components trought all applications • Know dependencies between applications for Web Services • Document funcionalities on a per database basis (complexity) • Answer questions when the developer may be gone… • Improve security ➢ This will allow to evaluate cost changes, make right decisions for maintenance. Functional requirements made by the customer 13
- 14. • No license fees. (context: IT Cost per user is rising steadily) • Low cost (only with Lotuscript) & homemade > Rapid Application Developement • UX : Easy to use and understand > Search the data and find • Deliver an accurate and updated Audit > Audit must always contains up-to-date information > Answering urgent requests from Management > Enabling IT for improving/cleaning/decommissioning Notes applications Practical Requirements made by the customer 14
- 15. • Know your code – relationship – dependencies between applications • Security Audit mandate > clean the code > C: references, misuses of tmp folder, hard coding • Detect Bad practices: cartography of inheritance of design elements > Orphans or not – Do not refresh tags • Maintenance & improvements > Migration from Word.application to LibreOffice, Identify / evaluate OLE Calls • Find niddles in my code: > Use case: “errors after v12.0.1 upgrade“ > Where are the calls to specific (deprecated) JAR in my server Solving specific problems 15
- 16. • Scheduled updated (only update design since last scan) • Integration with Teamstudio > continous updates for DBs on Build Manager • Trigger manual update (available with one button) • Update start from last point of failure of the scan if the process ended with error • Size should not be a problem Maintenance of the Audit 16
- 17. • We use the design from the v12 system database • We add some color specific columns • We think in usability first (one view = one answer) DESIGN: v12 UX > Simple (we are not designer) 17
- 18. DESIGN: v12 UX > Menu 18
- 19. DESIGN: v12 UX > Configuration document 19
- 20. DESIGN: v12 UX > Database document 20
- 21. HOW TO AUDIT THE CODE 21
- 22. • Maps all databases in server • Gather useful information • With customized views Dominique Perarnaud Where is the information ? 1.Catalog.nsf 2. WebServices.nsf • Maps all WebServices • Authorize ejecution, and gather database using it (Database path, Project owner) • Clear relation of dependencies: who can call who 22
- 23. Dominique Perarnaud Where is the information ? 3. Central dependencies DB Each Application DB store one “Configuration document” > Dependencies by configuration 6. Each application with his code 4. Build Manager: Who is automated ? 23 5. Script library Component DB
- 24. • First Part: Gather information from external database into “Central Audit database” • Database document from Catalog.nsf -> Audit Db • Doc Configuration from Dependencies DB -> Audit Db • Web Services Dependencies -> Audit Db • Build Manager -> Analysis + update -> Audit Db • Cartography of the central “Script Library Component” DB ressource-> Audit Db ➢ At this point, we have a clear view of dependencies between database, and which one is automated. ➢ No NTF Dbs, No System Dbs in the list DEMO Workflow of analysis 1/2 24
- 25. • Step 0: User enable/ diabled analysis on a per database basis Dominique Perarnaud Workflow of analysis 2/2 25
- 26. • Step 2: Agent extract all the code by design element in Audit Database (DXL) • Step 3: Agent parse the code to extract information • Like if Design elements use LS component • Step 4: Agent validate if Values from Configuration Document are used • Step 5: Agent validate use of Excel/Word, & other specific topics asked by customer Database offer dedicated views to answer some requests. Extra requests are solved by searching the full text index. Dominique Perarnaud Workflow of analysis 2/2 26
- 27. DEMO 27 At this point, we have confirmation about if : • Configuration document are accurate or contains false positive. • Consumer and Provider for Web Services are up-to-date • LS Component are correctly implemented • And a lot of precise informations….
- 28. WHEN THE CODE TALK Results 28
- 29. 1. Hardcoded > bad practices like email, server names, usernames, databases names, old employees, Local Temp Folder 2. Dependencies on external interfaces like DLLs/pdf/Word, ODBC, Fax, Outdated third party applications 3. Inheritance - Do Not refresh 4. Dependencies between Dbs 5. Unused Script library 6. Unused values from Configuration document 7. Compliance with for Nomad web 29 Analysis and Results
- 30. 1. Hardcoded > email, server names, usernames, databases names, old employees (DENY ACCESS GROUP) 30 Results Search Formula of the agent: NAB give us the emails : search and mark then accordingly
- 31. Dependencies on external interfaces DLLs (pdf Word), ODBC, Fax, Outdated third party applications 31 Results Search formula include specific keywords
- 32. Bad management of inheritance – Do not refresh tags 32 Results
- 33. Deployement to production 33 Results Agent identifies key field inside Build Manager and mark then accordingly
- 34. Dependencies between Dbs 34 Results Search by specific fields extracted from the DXL Search and mark then accordingly
- 35. 35 Results
- 36. Unused values from Configuration document 36 Results Search Field name from the Config Document inside the design of all the design element of the same database
- 37. Unused Script libraries 37 Results Search Script Library name present in the DB, in all the design elements of the same database
- 38. DESIGN: v12 UX > Database document 38
- 39. HCL Nomad Web 39 Compliance of Nomad Web is just an agent away… https://help.hcltechsw.com/nomad/1.0_web/nomad_web_limitations.html •Following Java™-based applications are not supported: •JavaAgents •XPages (.xsp) •LotusScript 2 Java Bridge (LS2J – Uselsx “*javacon” ) •Web services design elements are not supported. •Dynamic client extensions via native code are not supported. This includes but is not limited to the following: •The DXJ name picker implemented for Windows via a DLL. •Some LotusScript and Formula commands associated are not supported. •In particular, commands associated with Object Linking & Embedding (OLE) are not supported. •The LotusScript LSXLC and LSXODBC extensions.
- 40. • Regular (and simple) Lotuscript has been used to search, and extract the data, Export DXL and then parse the code… • Most of the work consist in finding how to gather and organize the information in order to present it in an explicit way. • Update the Audit is also an important aspect (“<modified> <date>” ) About the Lotuscript used in this database 40
- 42. • Check the ACL of the user running the Audit, first. Must be Manager (LocalDomainAdmin) • Do not Audit if you have NO design change (already integrated) • Do not Audit mail template (complex, slow) • Do not Audit mail file • Do not Audit System database (useless) • Audit only NSF • Check what database needs more time to be audited -> complex design Some points of interest when running the tool 42
- 43. • Add black list “Do not audit” for Database • Visualize data with Graph (xPages?) • Extend with action in YTRIA Automation API Ytria tools can be launched from the agent to generate more data to add to the Audit. • Integrate with JIRA create ticket to ask improvement, linked with some document of this Audit • Update Audit after each automatic build process launch from Build Manager • Add Usage data / Active users /By database Possible improvements 43
- 44. • We want to know your opinion about our work • Missing features ? Ideas ? • Loophole you see that we missed ? • Would you buy a copy of that work ? At which price ? • Contact: info@data101.es ANSWER OUR QUESTIONS, ASK YOURS 44
- 45. dperarnaud@data101.es Dominique Perarnaud @dperarnaud www.dperarnaud.es www.data101.es 45 Questions ?
- 46. Thank you !! 46