Engage 2018: IBM Notes and Domino Performance Boost - Reloaded
2 likes3,809 views
The document discusses strategies for boosting the performance of IBM Notes and Domino, detailing various optimization techniques such as transaction logging, database maintenance, and configurations for effective data management. It emphasizes the importance of proper hardware, client configurations, and database properties to enhance performance. Additionally, the document provides insights into managing client startup times and improving utilization of IBM collaboration solutions.
Engage 2018: IBM Notes and Domino Performance Boost - Reloaded
- 1. Adm03. IBM Notes and Domino Performance Boost - Reloaded •Christoph Adler (panagenda) •Luis Guirigay (IBM) 1
- 2. Christoph Adler - Senior Consultant at panagenda 17 years of IBM (Lotus) solutions experience Focused on • IBM Notes Client Management • Analysis and Optimization of ICS infrastructures • Cloud onboarding • Application Modernization Good to know • From Germany • Frequent traveler • Geek • Don’t give energy drinks squirrel mode 2#engageug
- 3. Luis Guirigay - WW Executive IT Specialist at IBM Experience 20• + years in the IT industry Published Author• 30• + Technical Certifications Technical Lead for IBM Collaboration• Member of the IBM Academy of Technology• Evangelist, Speaker, Thinker, Problem solver• Good to know Venezuelan born• 🇻🇪 Lives in South Florida• 🇺🇸 🏝 ☀️ 🏖 Frequent traveler• Father of four• 👶🏻👧🏼👦🏻🐶 3#engageug
- 4. Agenda IBM Domino Performance Boost Luis IBM Notes Performance Boost Chris Summary 4#engageug
- 5. Next topic IBM Domino Performance Boost 5#engageug
- 6. #domino2025 Page 6 / © 2018 IBM Corporation Transactional Logging (TXN) • Sequential writing into a log file • Commits transactions during off-peak • Archive TXN Allows for better Performance and Incremental Backup/Restores • Circular TXN allows for better performance • TXN Logs stored on a separate disk controller for best performance (depending on your platform) Disk 1 2 3 4 5 6 7 8 9 … Remove unread mark Remove unread mark vs.
- 7. #domino2025 Page 7 / © 2018 IBM Corporation Smaller Databases = Faster? Or is it just space saver? Some Organizations are not aware of all the space they can save by using: • Design Compression • Data Compression • LZ1 Compression + DAOS Multiply space saving by “X”
- 8. #domino2025 Page 8 / © 2018 IBM Corporation DAOS Requires Transaction Logging• DO NOT play around with the NLO/DAOS Folder unless instructed by IBM. DAOS is not• a toy. Setting DAOS to remove• 4 kb attachments is not a good idea (Use DAOS Estimator) Enable DAOS for the• Mail.box for One vs “many” DAOS/NLO transactions Set AV to not scan/act on DAOS folder and NLO files•
- 9. #domino2025 Page 9 / © 2018 IBM Corporation Database Performance • Should be: • Database.Database.BufferPool.PercentReadsInBuffer > 90 • Database.DbCache.OvercrowdingRejections as close to 0 as possible • Database.DbCache.Hits is high • Database.DbCache.CurrentEntries much less than Database.DbCache.MaxEntries • If Not: • Increase NSF_DbCache_Maxentries (look for Database.DbCache.MaxEntries) • Adjust NSF_DBCACHE_CLEAN_HOLD_TIME • Adjust NSF_Buffer_Pool_Size_MB on partitioned servers • Increase Physical Memory or RAM allocation
- 10. #domino2025 Page 10 / © 2018 IBM Corporation Database Maintenance Tool (DBMT) • Best practice for 9.x servers • Automates Compact, Fixup, Updall and more • Runs as • Program Document • Server Console command (server is running) • Command line (server down) • More resources: • http://www-01.ibm.com/support/docview.wss?uid=swg27039379&aid=1 • http://www-01.ibm.com/support/docview.wss?uid=swg27046303&aid=1 • https://www.ibm.com/support/knowledgecenter/en/SSKTMJ_9.0.1/admin/admn_running_the_database_maintenance_tool_from_a _program_document_t.html • https://www.ibm.com/support/knowledgecenter/en/SSKTMJ_9.0.1/admin/admn_maintaining_user_mail_files_using_the_dbmt_too l_t.html
- 11. #domino2025 Page 11 / © 2018 IBM Corporation More Database Properties Unread marks.• System Databases, Applications, Mail In Databases, etc.• Additional tips here:• https://www.ibm.com/support/knowledgecenter/en/SSKTMJ_9.0.1/admin/t une_databasepropertiesthatoptimizedatabaseperformance_r.html
- 12. #domino2025 Page 12 / © 2018 IBM Corporation The most expensive view in IBM Domino???
- 13. #domino2025 Page 13 / © 2018 IBM Corporation The most expensive view in IBM Domino …..The Inbox!!! • If running from Server and using iNotes or Notes Client…. • Enable Inbox Maintenance • Teach your users inbox management tips • Swift file?? • Best Practice….. • Use Managed Replicas
- 14. #domino2025 Page 14 / © 2018 IBM Corporation Managed/Local Replica vs Server Access
- 15. #domino2025 Page 15 / © 2018 IBM Corporation On Disk Structure CREATE_R• 8_DATABASES=1 CREATE_R• 9_DATABASES=1 Examples of benefits • R5.0 (ODS41) = participate in transaction logging • R6.0 (ODS43) = LZ1 compression and shared templates • R8.0 (ODS48) = design and document compression • R8.5 (ODS51) = DAOS • R9.0.1 (ODS52) = Performance improvements, better handling of huge (2GB+) attachments
- 16. #domino2025 Page 16 / © 2018 IBM Corporation ServerTasks and ServerTasksAt Updates • Keep ServerTasks simple.. Only what you need • Tired of losing your ServerTasksAt customizations when upgrading? SetupLeaveServerTasks to the rescue Add SetupLeaveServerTasks=1 to server’s notes.ini Disables automatic updating of ServerTasksAt#= lines during a Domino Server upgrade
- 17. #domino2025 Page 17 / © 2018 IBM Corporation Unnecessary Replication Tasks
- 18. #domino2025 Page 18 / © 2018 IBM Corporation User vs Server • Do not have users and servers competing for network resources • Have Server to Server and Cluster Replication on their own Private LAN • Separate NIC or IP for Users only
- 19. #domino2025 Page 19 / © 2018 IBM Corporation Clustering and Replication Replica.Cluster.WorkQueueDepth Look for Zero (0)
- 20. #domino2025 Page 20 / © 2018 IBM Corporation Clusters and Replication After increasing Cluster Replicators from 1 to 3
- 21. #domino2025 Page 21 / © 2018 IBM Corporation Health Monitoring • Watch servers on single screen • Monitor servers and/or tasks needing attention • Wait for New Relic integration for even better Health Reports
- 22. #domino2025 Page 22 / © 2018 IBM Corporation Load Balancing • HA = High Availability • DR = Disaster Recovery • Load Balancing and Failover are not the same thing! Look for Load Balancing Best Practices!
- 23. Next topic IBM Notes Performance Boost 23#engageug
- 24. Make Your Data Work for You 2 things before we start ...
- 25. Thank you! Francie Tanner Florian Vogler Daniel Reimann
- 26. NOTES IS NOTDEAD
- 27. Make Your Data Work for You Let‘s start with some numbers
- 28. IBM Notes 9.0.1 Feature Pack 10 Interim Fix 3 ~35.000 files (Notes 6/7 | 8/9 Basic used to have ~550) • 28.600 files in Notes Programframework in 2.700 subdirectories – 11.200 (largely undocumented) .properties files – 1.200 .jar files – 1.300 .html files • 600 files in Notes Dataworkspace in 250 subdirectories – 40 (largely undocumented) XML files – 40 (largely undocumented) .properties files • Which is – 35.000 files in 2.950 subdirectories – 11.280 largely undocumented files – ... and >30 NSFs/NTFs/NDKs – ... and the good old notes.ini file
- 29. IBM Notes 9.0.1 Feature Pack 10 Interim Fix 3 (cont.) Add over 80 preference/dialog boxes Each dialog has ~• 15 settings at average For a total of 1,200+ settings sprayed across address book (very few)• notes.ini (few)• and XML files (lots)• Plus any plugins, widgets, add-ons, etc. “you“ or the user may have added to the Notes client install
- 30. IBM Notes 9.0.1 Feature Pack 10 Interim Fix 3 (cont.)
- 31. IBM Notes 9.0.1 Feature Pack 10 Interim Fix 3 (cont.) Workspace (nsf) Physical nsfs 1, 2, 3, … n
- 32. Make Your Data Work for You What makes the Notes Client START slow
- 33. IBM Notes release If you‘re not on at least 9.0.1 yet 9.0.1 is a significant improvement Software lifecycle of IBM Collaboration Solutions http://www– -01.ibm.com/software/support/collaboration/lifecycle/
- 34. IBM Notes release (cont.) How-to install IBM Notes 9.0.1 Feature Pack 10 1. Uncompress Installation files manually (e.g. using 7-zip) 2. Run the following command to install the Feature Pack: cmd.exe /c””c:TempPackage901FP10setup.exe" /s /v"/qn /lime “c:TempPackageLogs901FP10.log""
- 35. IBM Notes release (cont.) How-to install IBM Notes 9.0.1 Feature Pack 10 Interim Fix 3 1. Uncompress Installation files manually (e.g. using 7-zip) 2. Run the following command to install the Feature Pack: cmd.exe /c””c:TempPackage901FP10IF2ClientUpdate.exe" /s /v"/qn /lime “c:TempPackageLogs901FP10IF2.log""
- 36. Antivirus realtime scanning in Data directory ~55% of the entire data volume in the Data directory changes on a DAILY basis in most setups • 5 - 10% in Dataworkspace (up to 300 files) • DISABLE Antivirus scanning for at least *.ns* *.ndk *.xml *.properties *.jar in Data Directory. 55% 45% Data directory: File changes per day Once or more Less
- 37. What makes the Notes Client START slow (cont.) Client crashes/database fixups Especially bad for network based data directories clients• Outdated Hardware You need >• 2 GB memory if you want to use Notes AND something else Old fixed disks with fragmentation• Fun Exercise: Buy a Solid State Disk and make Notes start off of that ;• -) Tons of startup programs/scripts launching during the “boot storm” If the disk is unable to keep up, it• ‘s not Notes' fault
- 38. What makes the Notes Client START slow (cont.) Outdated Data directories Virtually• 100% of all NTFs (Laptops: 90%) can be removed by Cleaning up old Notes• 4+5+6/7 templates using• ini:SharedDataDirectory= (also with Single User install!) Operating System Windows• 8.1 boots ~20% faster than Windows 7 Windows• 10 ~15% faster than 8.1 – “Meltdown” and “Spectre” and upcoming “Spectre-NextGeneration”patches may have a negative performance impact
- 39. ODS of local databases ODS = On Disk Structure • ODS 16 = Notes 2 • ODS 17 = Notes 3 • ODS 20 = Notes 4 (or Templates) • ODS 41 = Notes 5 • ODS 43 = Notes 6 & 7 • ODS 48 = Notes 8 • ODS 51 = Notes 8.5 & 9 • ODS 52 = Notes 9.0.1 The difference between ODS 43 and 52 = up to 80% LESS FILE I/O This helps with ANY client AND server hardware
- 40. ODS of local databases (cont.) CAUTION: The ODS is not automatically updated when upgrading clients – ODS 51 requires Notes >= 8.5 and CREATE_R85_DATABASES=1 in notes.ini for NEW db's – ODS 52 requires Notes >= 9.0.1 and CREATE_R9_DATABASES=1 in notes.ini for NEW db's Good News for Notes > 8.5 – Use NSF_UpdateODS=1 with CREATE_R85/9_DATABASES=1 for a one-time upgrade of most local databases • NOTE: USE WITH EXTREME CARE IF YOUR DATA DIRECTORIES ARE ON A NETWORK DRIVE! • NOTE: End users can't access databases during compact OR databases will be ignored if in use Manually upgrade ODS of local databases/templates – Run the following command line: ncompact.exe -C -ODS -* -Client -UpdateIndexes
- 41. ODS of local databases (cont.) Impact examples Startup time of a Notes– 9.0.1 client with 3 ODS20 databases in Notes data = 45 seconds After ODS upgrade:• 17 seconds Reduced File I/O of your disks/SAN/NAS after ODS– 43 to 52 upgrade by 60% Removing– 70% of all old files in Data directories on SAN/NAS = 45% less managed storage (backup) AND much faster start The bad news: There is no way to tell what is “out there” – 3rd party tools or random inspections are highly recommended
- 42. IBM helps!
- 43. Make Your Data Work for You What makes the Notes Client START (super) fast
- 44. Eclipse Workspace improvement • Every IBM Notes Client start in a virtual environment (citrix, vmware, etc.) is a first (setup) start and takes at least 23 sec on average • ~65% - 70% of time during first starts is used to build the workspace folder (at least 15 sec) • A typical new workspace folder (IBM Notes 9.0.1 FP10 IF2) consists of – >500 files – >200 folders – >85 MB (including 64MB *xpdplat_.jvm_G11 file NotesShareClasses improvement) Building/Updating the workspace folder causes lots of file I/O
- 45. Eclipse Workspace improvement (cont.) Accelerate the IBM Notes Client start to be >65% faster (with less file I/O) 1. With a non-admin user (Windows + IBM Notes) run an initial setup of IBM Notes 2. Start IBM Notes and wait for 2-3minutes 3. Exit IBM Notes 4. Start IBM Notes again and wait for 2-3minutes 5. Exit IBM Notes 6. Create the following folder %ProgramData%IBMNotesDataCommon 7. Copy the workspace folder from %LocalAppData%IBMNotesData to %ProgramData%IBMNotesDataCommon
- 46. Eclipse Workspace improvement (cont.) Delete the following files & folders in8. %ProgramData%IBMNotesDataCommonworkspace UDMa. Logsb. all but .prov2install .metadatac. .pluginscom.ibm.collaboration.* .metadatad. .pluginscom.ibm.rcp.personality.framework .metadatae. .pluginscom.ibm.rcp.security.auth .metadataf. .pluginscom.ibm.rcp.managedsettings
- 47. Make Your Data Work for You Live
- 48. Make Your Data Work for You What makes the Notes Client PERFORM slow
- 49. What makes the Notes Client PERFORM slow Again • ODS of local databases • Hardware Notes.ini • EXTMGR_ADDINS = (not just) local virus scanners • Disable Notes “plugins“ like virus scanners etc. – They increase Network traffic by up to 200 – 250%! Data directory on the network vs. fixed disk • Disk performance is the #1 bottleneck – think SSD vs. normal HDD ; 7.200rpm vs. 5.400rpm ; network drive vs. local fixed disk
- 50. Port settings Port settings are included in the notes.ini • Type / Active / Encryption / Compression / Timeout Legacy • LAN0 / COMx / DisabledPorts should be removed Good idea • Use TCPIP according to the client type • PORTS = TCPIP • TCPIP = TCP,0,15,0,,12320, Desktops • TCPIP = TCP,0,15,0,,45088, Laptops • TCPIP = TCP,0,15,0,,12288, Citrix / VDI • Attention: Hardware compression (e. g. Riverbed) Don‘t forget the Domino servers!
- 51. Port settings (cont.) Legacy/Default port encryption for Notes/Domino RC• 4 128Bit (Rivest Cipher 4) Best practice settings for port encryption on Domino server >= 9.0.1 Fix Pack 7 AES• -GCM 128Bit (Advanced Encryption Standard) notes.ini• PORT_ENC_ADV=– 84 See the following Technote for details and read before you use the parameter: http://www• -01.ibm.com/support/docview.wss?uid=swg21990283
- 52. What makes the Notes Client PERFORM slow (cont.) Install types: Basic vs. Standard Basic client starts up fast, Standard client, well– … not so fast IBM Roaming Users Try replicating– 80MB each time you logon to Notes before use! Connection documents Pointing to outdated server names or IP addresses– Low priority connection documents, which can cause connectivity failures– Location documents IP addresses for mailserver– breaks Policies Outdated mail and home server information– Outdated mail file path– Catalog server (think db links, doc links, new bookmarks)–
- 53. Why is the Catalog important? Ever wondered what exactly happens when you click on db links / doc links / view links 1. Notes checks whether you already have a workspace icon pointing to the replica ID in the link. If yes, the top most icon on the stack decides on which server the db will open 2. If there's no icon on the workspace, Notes asks the catalog server in your currently selected location document where the db is to be found. So, make sure your catalog is up to date! No catalog server? Notes will ask your mail server! 3. Are you replicating your catalog.nsf across all servers? Bad news: If 2.) applies, Notes will open the database on the server it finds first in the catalog – which is sorted alphabetically. In global infrastructures that might be a server far far away Make sure to use an appropriate catalog server in your location documents!
- 54. Make Your Data Work for You PERFORMANCE Tips and Tricks
- 55. Performance Measuring with Client Clocking NOTE: All of this should only be done on “ONE” client – slows down client, creates huge log files! Use the following to see and measure client performance – CLIENT_CLOCK=1 – CONSOLE_LOG_ENABLED=1 – Debug_Outfile=c:debug.txt – Debug_Console=1 – DEBUG_TCP_ALL=1 (use only if needed) Fileserver or local disk Performance Debugging is virtually impossible – If on >= 8.5 try adding DEBUGGINGWCTENABLED=16386 in Notes
- 56. Performance Measuring with Client Clocking (cont.) Client_Clock=1 leads to a debug file of the following format:
- 57. Performance Measuring with Client Clocking (cont.) OpenNTF – NRPC Parser Link: http://www.openntf.org/internal/home.nsf/project.xsp?action=openDocument&name= Notes%20RPC%20Parser Last Update: v1.0.14 – Nov 10, 2016 Developed by: Jamie Magee, MartinScott Consulting LLC, US Andrew Magerman, Magerman Collaborative Software GmbH, Switzerland
- 58. Performance Measuring with Client Clocking (cont.) OpenNTF – NRPC Parser
- 59. Performance Tuning – Understanding cache.ndk We've all been “advised to”/guilty of deleting the cache.ndk in an effort to cure Notes performance problems/symptoms – Some companies even delete cache.ndk on a scheduled basis HOWEVER, let us explain the impact of that on Notes performance ... – Increases traffic between client and server by 4,000% (forty times more traffic, yes) (10,000+% if the data directory is on a network drive) – The more databases end users use, the worse it gets – There is always a root cause – deleting cache.ndk only fixes symptoms – Let's look at some detailed traffic analysis for just ONE application (the mail file) of ONE user ...
- 60. Performance Tuning – Understanding cache.ndk (cont.) 100 users = 150 MB vs. 3,7 MB 1000 users = 1.5 GB vs. 37 MB Excluding Fileservers! After deleting cache.ndk Keeping / NOT deleting cache.ndk
- 62. Summary: IBM Notes and Domino Performance Boost -Reloaded Update/clean out your ODS, databases and templates and hardware Standardize whenever and however possible • This will save you hugely on support time/cost Keep your users version, fix/feature packs current (if stable) • Each release brings performance improvements but also resource demands Work around your inability to see clients • Write an inventory agent, login script, get a 3rd party tool Any challenge can be overcome with Knowledge and Tools • All client issues are solvable 62
- 64. This presentation is available on slideshare.com You can find the latest version here: https://www.slideshare.net/christophadler50 64