Scaling xtext
- 2. Sigasi ● IDE for Hardware Description Languages ○ VHDL, (System)Verilog ● Using Xtext for 4 years ● Large user base ○ (commercial, free, students)
- 3. Our company goal ● Assist hardware designer ● High quality interactive front-end compiler ○ Instant error reporting ■ parsing, semantic, linting, style checking ○ Visualisations, design exploration, ... ○ Integrate with ecosystem ■ other compilers, simulators, synthesizers
- 4. The challenge ● Large projects ○ > 250 KLOC is not uncommon ○ design + external libraries ○ big files ■ some libraries are distributed as 1 file ■ generated hardware cores ●
- 5. Adopting Xtext ● Started with the early Xtext 2.0 snapshots ● Initial performance analysis ○ Clean build performance of a big project (330k LOC) ■ > 20 minutes ■ > 2 GB ○ Editing big files (> 1 MB) ■ unusable
- 6. Adopting Xtext ● Started with the early Xtext 2.0 snapshots ● Initial performance analysis ○ Clean build performance of a big project (330k LOC) ■ > 20 minutes → < 1 min ■ > 2 GB → < 1 GB memory ○ Editing big files (> 1 MB) ■ unusable → usable with reduced editor
- 7. ● Xtext framework improvements ● Measure → analyze → improve or cheat ○ faster build ○ reduce memory usage ○ UI responsiveness Improving performance
- 8. Overview ● Analysing build performance ○ Analyze the build ■ Macro build measurements ■ Key performance points ○ Reduce workload ○ Parallelize the build ● Analyzing UI issues ○ Monitoring the UI thread ○ Saveguards
- 9. Analyzing builds: builder overview Global indexing Linking Validation Custom Validation Global index Eclipse resources warnings errors resource descriptions Builder Participants resource changes ?
- 10. Analyzing builds: metrics ● For each build ○ # of files being build ○ timing: Global index, Linking, Validation, Individual builder participants ● Instrument by overriding ClusteringBuilderState & XtextBuilder ● Example: Building 134 resources, timing: { global index=1806, linking=378, validation=823, totalLinkingAndValidation=1364 }
- 11. Analyzing builds: resource loads ● Observation: ○ Most time spent in resource loads ○ Certain files are loaded multiple times?! Global indexing Linking Validation Builder Participants resources LOAD POTENTIAL RELOAD POTENTIAL RELOAD
- 12. Analyzing builds: Memory footprint Global index Resource Set ● Global index ○ always loaded →limit size ○ use hashing: name + hash of contents ● Reduce EMF size derived model -> Xcore model http://www.sigasi.com/content/view-complexity-your-xtext-ecore-model
- 13. Optimize loading ● What is resource load? ○ Parse ○ build EMF model ○ build Node model ● Parallelise ○ parse multiple files simultaneously ○ ~3 time faster loads on 4 core machine ○ EMF synchronization issues ● Cache ○ serialize EMF and Node model in a cache ○ 3-4 time faster loads
- 14. Linking Global indexing Linking Validation Builder Participants ● Language specific ○ VHDL vs Verilog ● Avoiding linking ○ expressions that we couldn’t handle (yet) ○ library files, only when used in user-code ● Many iterations ○ lazy linking vs eager linking ○ From 40% of build time to 20%
- 15. Linking Pseudo-algorithm ● In your IScopePovider ○ Find a eContainer that can be linked eagerly ○ Link everything ■ unresolved links become UnlinkedDeclaration ● In your DefaultLinkingService ○ getLinkedObjects(...) ■ already resolved? → skip it ● Disable notifications
- 16. Validation Global indexing Linking Validation Builder Participants ● Combine validations to avoid model traversals ● Local analysis, do global validations moved into builder participant ● Avoid validation ○ disabled validations ○ libraries: errors & warnings are suppressed anyway ● Monitor
- 17. Track performance ● Nightly build ● log build times
- 18. UI responsiveness ● Measuring: detect a blocked UI thread ○ initially Svelto https://github.com/dragos/svelto ○ now our own method & logging ○ Eclipse Mars ● Improvements ○ UI is for drawing only! ○ Make sure everything is cancellable ● Safeguards ○ certain services should never be executed on the UI thread => check & log
- 19. Lightweight Editor (fallback) ● Syntax-highlighting + markers ● For files > 1 MB ● Based on ContentTypes extension point
- 20. Two ContentTypes (based on file size) <extension point="org.eclipse.core.contenttype.contentTypes"> <content-type ... describer="com.sigasi...FullVhdlContentDescriber" name="VHDL editor" <describer class="...FullVhdlContentDescriber" /> </content-type> <content-type ... describer="com.sigasi....LightweightVhdlContentDescriber" name="Lightweight VHDL editor" <describer class="...LightweightVhdlContentDescriber" /> </content-type> </extension>
- 21. Future work ● Continuous process ● Cache global index info per resource? ● Linking without node model? ● Avoid project locking build (Xtext 2.8?)
- 22. Come talk to us about... ● Documentation generation ● Fancy linking algorithms / type systems ● Graphical views ● Cross-language support ● Testing Xtext-plugins ● Lexical macros ● Manage large amount of validations ● ...
- 23. +1 0 -1 Sign in: www.eclipsecon.org Evaluate the sessions