D3.3 - Executable Scenario Management

In this deliverable, we describe how the ExSce Management enables the storage and querying of provenance data for scenario executions.

1. Introduction. An overview of what provenance is, and the concepts and relations available in the W3C PROV specification used in the ExSce Management. Introduction to metamorphic testing which is used to generate new scenarios and as a test oracle, validating whether a run “passes or fails” the test.
2. Case Study: Navigation Tasks for a Multi-Robot System. A case study based on the PAL use case is used as a running example throughout the deliverable to demonstrate how the ExSce management is used to model and manage the scenarios of a multi-robot system performing navigation tasks. We use the publicly available software of the PAL robots, which is based on ROS1, and uses the default navigation stack.
3. Executable Scenarios
   • How scenario specifications are modelled and conform to the PROV specification.
   • Execution data is obtained from transforming bag files into PROV documents, which can then be queried as new scenarios are generated. This includes scenario meta-information such as derivations from other scenarios, simulation timestamps and various metrics for the entire run and individual path segments.
   • Oracles are modelled as relations between a so-called “base” scenario, resulting in two main types: baseline and metamorphic oracles.
4. Metamorphic Relations. How input transformations can be used to generate new scenarios and the type of output relations that we consider as part of the metamorphic relations. This is enabled by the storage of the PROV data on a property graph using Neo4j, which is allows us to query and compose both input transformations and output relations. The PROV data of these transformations is also stored in the database, enabling stakeholders to trace how scenarios were derived, i.e., which operations generated which artefacts (e.g., task specifications) and which was the resulting scenario.
5. Queriable Scenario Execution. How to query scenarios: Observed outputs from execution data, oracles and metamorphic relations, and scenario derivations and clustering.
6. Tutorial. How to employ the ExSce Management approach and tools using the PAL case study as an example.
7. Generalization of the ExSce Management. How the ExSce Management approach generalizes in the context of other ExSce tools, SESAME deliverables and use cases.