[SiriusCon 2018] AdvoCATE: An Assurance Case Automation Toolset Based on Eclipse and Sirius
1 like863 views
The document discusses Advocate, an assurance case automation toolset developed by NASA to enhance safety management in high-hazard industries through evidence-based assurance and safety cases. It emphasizes the need for tools that support the creation, assessment, and management of safety and assurance artifacts, particularly in the context of unmanned aircraft systems. The toolset incorporates various technologies and methodologies, including barrier modeling and hazard analysis, to provide comprehensive safety assurance throughout the system lifecycle.
[SiriusCon 2018] AdvoCATE: An Assurance Case Automation Toolset Based on Eclipse and Sirius
- 1. AdvoCATE: An Assurance Case Automation Toolset Based on Eclipse and Sirius 4th December, 2018 Ewen Denney & Robbie Henderson (Joint work with Ganesh Pai and Dimo Petroff) NASA Ames Research Center Robust Software Engineering Group ewen.denney@nasa.gov
- 2. Research Motivation • High-hazard industries are moving to active safety management – Safety management system (SMS) in aviation – Need to • Unify reasoning about technical aspects of safety • Support safety-related decision making • Goals-based regulation is attractive for novel applications – When regulations and performance standards are absent • Unmanned aircraft systems (UAS), Autonomous systems, … – Increases flexibility for regulated entity – Evidence-based assurance safety case / assurance case 2
- 3. Safety and Assurance Cases ‘A safety case is a structured argument, supported by a body of evidence, that provides a compelling, comprehensible and valid case that a system is safe for a given application in a given operating environment’ - NASA System Safety Handbook ver. 1 (2014) • Essentially, a safety risk management artifact – Other compatible definitions and guidance on content – Based on application domain, standard, regulatory paradigm, etc. • An assurance case generalizes safety cases to other assurance properties: reliability, security, availability, … 3
- 4. Risk Control Risk Analysis and Assessment Hazard Effect Severity Likelihood Initial Risk Level Hazard Control Residual Risk Level H1 - Airspace encounter with GA aircraft NMAC / MAC 2 (Haz.) 1 (Cat.) Probable Probable 2B 1B Detect & Avoid Flt. Termination ... 2D 2D H2 – Stall CFIT Safety Risk Management 4 System Analysis Concept of Operations, System/change description, Regulations, … HazID Hazards Operational, functional, … Design target Barrier Modeling – Abstract Safety Architecture Safety Requirements Implementation Mitigations Safety requirements Barrier and Control functions Risk scenarios, design targets, risk evaluation Assurance Rationale (Structured Argument) Evidence Artifacts Design, Analysis, Verification Testing, Assurance claims, strategies, context, rationale, … Operational Safety Assurance (Monitoring and Update) Safety performance measures, monitors, … Operational Evidence Verification of safety performance targets Assumption corroboration Hazard tracking, Precursors, …
- 5. Example: UAS Safety 5 Combination of operating modes • Visual line of sight (VLOS) • Beyond visual line of sight (BVLOS) • Beyond radio line of sight (BRLOS) Varying access profiles • Operating range • Terminal airspace • Transit (vertical / lateral) Diverse environment • Populated / urban / built-up areas • Uncontrolled / controlled airspace • Low / high density airspace Varying mission concepts • Package delivery • Surveillance • Aerial inspection • Mapping, … Different configurations • Airborne sensors (Lidar, sonar, FPV camera, Radar) • Ground sensors (Radar) • Multiple GCS, Roaming GCS, … Increasing complexity in mission and operations UAS – Unmanned Aerial System (aka drone)
- 6. UTM: UAS Traffic Management 6
- 7. Tool Needs • Creation and assessment of assurance cases – Support variety of diagrams and for assurance artifacts representations (graphical, tabular, textual) – Views for diverse stakeholders and use cases – Consistency and navigation between assurance artifacts – Automation workflows – Integration with 3rd party tools • Tool technologies – EMF: model-based assurance – Sirius: graphical editing of industry standard safety notations – Xtext: domain specific languages and querying of safety models – NatTable: table editor for hazard/requirements analysis 7
- 8. Barrier Modeling • Collection of barrier models providing a risk basis – Collection of all factors affecting risk – Model for risk qualification/quantification 8 Event chain / accident trajectory Barrier compromise/breach Loss of Control State Threats / Causes / Initiating Events or States Accident / Loss / Harmful States or Events Prevention Barriers Recovery Barriers Hazard
- 9. Bow Tie Diagram (BTD) 9
- 10. Example: Loss of Separation 10
- 11. Rationale Capture via Assurance Arguments 11 Chain of reasoning Safety / Dependability Claims Item of Evidence Developed claims Documentation and Details Goal Structuring Notation (GSN)
- 13. AdvoCATE: Tables • Assurance Case Automation Toolset • Hazard analysis and risk assessment – Conducting hazard identification – Specification of hazard causes and consequences – Assessment of initial and residual risk levels given in terms of probability and severity • Safety and assurance requirements capture 13
- 14. Hazard Log and Tabular Editor 14
- 16. AdvoCATE: Arguments and Patterns • Structured argument development – Pattern specification – Automated pattern instantiation • Integration of formal methods and formal tool-based evidence – Hierarchical and Modular organization – Argument queries and views • Metrics 16
- 18. AdvoCATE: Safety Architectures • Safety architecture development – Composition of multiple bow tie diagrams – Views – Transformations (event and barrier split / merge) • Sequential event split: Loss of safe separation Loss of “well-clear” separation + NMAC • Parallel event split: MAC MAC within OR || MAC outside OR • Barrier split: Ground-based surveillance Radar surveillance + Visual surveillance – Risk computation: event probability along paths 18
- 19. Safety Architectures 19 Bow Tie Modeling Automated View Extraction
- 20. AdvoCATE: Traceability • Navigation • Traceability matrices • Maintaining consistency between related artifacts, e.g., between – Entries in the hazard log and the relevant assurance requirements – Arguments and the corresponding requirements, verification artifacts, etc. 20
- 21. Tracing and Consistency 21 Hazards Safety and Assurance Requirements Assurance Arguments / Rationale Bow Tie Diagrams / Safety Architecture
- 22. Amalgam Activity Explorer • The Amalgam activity explorer is used in the design of our Safety, Mission Assurance, and Risk management (SMART) dashboard 22
- 23. Amalgam Activity Explorer • The (SMART) Dashboard allows us to: – Provide a clear and directional workflow towards a completed safety/assurance case 23
- 24. Amalgam Activity Explorer 24 • For each step we have one EMF model • Dependencies provide some of the workflow, i.e. safety architectures can require “requirements” model components • Necessary components are clearly prompted • Sirius diagrams relevant to the current model are accessible
- 25. Amalgam Activity Explorer • The (SMART) Dashboard allows us to: – Provide a clear and directional workflow towards a completed safety/assurance case – Provide feedback on the status of assurance activities, and areas that need to be developed further – Provide a naive evaluation of the current system safety 25
- 26. Amalgam Activity Explorer 26 • Problems with the safety case development are clearly brought to the users attention, with hyperlinking to the problem source
- 27. Amalgam Activity Explorer • The (SMART) Dashboard allows us to: – Provide a clear and directional workflow towards a completed safety/assurance case – Provide feedback on the status of assurance activities, and areas that need to be developed further – Provide a naive evaluation of the current system safety – In future, provide real-time evaluation of system based on feedback from a live platform 27
- 28. Activity Explorer Issues • We don’t always have a Sirius “session” – Amalgam works very well when provided a Sirius “session” – Some of our models are entirely developed in a DSL, or NAT Table tabular editor – Initially we created viewpoints for all resources….even when it wasn’t useful – We now manually load resources and open editors by id, and only use the Sirius session for the opening/creation of viewpoints • Debugging is hard! – Issues with activity explorer pages often result in no activity explorer at all, with no logging – help! 28
- 29. BX of Safety Models • Sirius viewpoints are used extensively, along side various editors, to avoid complex bi- directional transformations of the safety models – The safety architecture of a system can be viewed as a Controlled Event Structure, a single diagram showing the temporal flow of all events – One event in a CES may have a local bow tie, where we only care about the event, its own causes and effects – Through a combination of Xtend model helpers and multiple viewpoints, we managed to merge most models containing similar information and just provide viewpoints where necessary 29
- 30. BX of Safety Models 30
- 31. Sirius Custom Properties Panel • Many of AdvoCATE’s graphical elements are the product of multiple modelled constructs – To handle this, we made use of Sirius custom property panels – Model elements, such as hazards, are edited from many locations in AdvoCATE, and are viewed in different forms all over the tool – One custom property panel is added, allowing us to define one uniform editing experience for the combined feature, regardless of what is shown – Certain semantic attributes can be shown, but not edited to allow the user context while in a particular viewpoint 31
- 32. Sirius Custom Properties Panel 32 Calculated Values – Mitigation of Risk A Hazard/Event A Hazard in progress – Event Instance
- 33. Property Panel Additions • Some customizations to the custom properties panel we have implemented: – Enum Lists: We have many model features as lists of enumerated values – Xtext editor widgets (more on that later) – Xtext index-query selection boxes – model cross references 33
- 34. Xtext 34 • All models within AdvoCATE make use of Xtext resources and the powerful index they provide • Extensive cross-referencing between models became cumbersome using pure EMF • Integration of Xtext and Sirius has been very smooth – with only minimal customizations to Sirius widgets and some services to take advantage of the Xtext index in diagrams • Most models we use require an Xtext DSL to keep all users happy…so extra effort is minimal
- 35. Xtext - Indexing 35 • With all models being Xtext resources we are able to take advantage of the Xtext index as a one-stop repository of safety elements • Cross-referencing by loading resources becomes quite cumbersome with large projects • We wrap Xtext index querying in services used by our Sirius diagrams, to take advantage of our DSL scope providers • Future plans will involve the DSL Devkit Scope/Export framework, to allow us to fine tune relevant safety artifacts, and export these to an external repository (large scale safety case development)
- 36. Xtext - Indexing 36 • We create an Xtext scope-provider-fed custom property widget • As the DSL is modified, the Sirius properties view is updated automatically – it simply calls our scope provider • Relevant EObjects are resolved and the list of choices is populated
- 37. Xtext – Serialized Models 37 • One important future feature of AdvoCATE is collaborative safety case development – When using pure EMF + Sirius, we found that version control struggled a little…
- 38. Xtext – Serialized Models 38 • One way we thought to combat this problem is a combination of: – Really good auto-layout (if a little ambitious) • We don’t necessarily need to version control the layout if we can do so automatically, and reliably • AIRD merge conflicts become huge, and impossible to merge – we might not need to track them – Serialize the model as a DSL, and parse • The models themselves in XMI format can be hard to merge • New features cause compatibility problems
- 39. Xtext – Serialized Models 39 • By designing a robust Xtext DSL for each model, we can more reliably track changes – Git likes DSLs way more than XMI – New features, or modified metamodels are less likely to also break the parser, but XMI almost always will – We can auto-create appropriate diagrams for our models in Sirius, and auto-layout on first opening • We’re still in the process of finding a solution to our problems – but this fits nicely so far
- 40. Xtext – Direct Edit Xtext Editor 40 • In some contexts, complex syntax had to be embedded in our graphical editors – Argument patterns, are a way to generate a GSN argument based on given data and a “pattern” providing the structure – Parameters are defined, and then embedded in node descriptions to be evaluated at generation time – To do so, we designed a DSL to define the pattern and it’s parameters – Great! We get all the content assist, linking, and that fun stuff
- 41. Xtext – Direct Edit Xtext Editor 41 But wait…what’s the structure?
- 42. Xtext – Direct Edit Xtext Editor 42 • Clearly, a graphical layout gives a much more manageable view of what the generated result might be – We needed a solution that combined the power of the Xtext DSL, for what might become very complex string-building expressions, with the high-level view of a Sirius viewpoint – We created a Sirius Direct Edit widget which wrapped the Xtext Embedded editor – Now we have content assist, syntax highlighting, hyperlinking, and inline validation – all as part of direct edit
- 43. Xtext – Direct Edit Xtext Editor 43
- 44. Perspectives • Ongoing focus on design-time assurance – Artifacts and rationale from development, prior to release-into-service • Outlook towards operational assurance through lifecycle – In-service safety performance monitoring • Autonomy applications – NASA System-wide Safety Project – DARPA Assured Autonomy Program – Expansion in application domain to spaceflight: initially robotic, eventually, human spaceflight • Future tool development – User-customizable dashboards – Query/view language – Collaborative development – Towards the Cloud … 44
- 45. We’re hiring! Contact: ewen.denney@nasa.gov 45 Looking for software engineers with experience in Eclipse, Sirius, Xtext, NatTable, ...
- 46. Please wait a few seconds before we automatically bring you to the next session (First Day Closing Session) If you want to keep talking with the speakers of actual talk, you will have to come back to this session. Thanks for listening to (Ewen Denney|NASA Ames) Any questions?