Project overview Introduction
Nowadays, modern systems in the domains of Industry 4.0, health care, autonomously driving cars or smart grids are examples of highly communicating (embedded) systems where software enables increasingly advanced functionality. The growing complexity of these Cyber-Physical Systems (CPSs) and Cyber-Physical Systems of Systems (CPSoS) poses several challenges throughout all system design, development and analysis phases, but also during their deployment, actual usage and future maintenance. Many leading companies have started envisioning the productivity boost of tomorrow to be brought by the application of full-blown AI (Artificial Intelligence) principles and techniques. As Gartner reported in 2019  40% of Infrastructure & Operations teams will use AI-augmented guidance and automation in large enterprises by 2023, thus resulting in higher IT productivity. Nowadays, more and more companies are already deploying AI in some specific parts of their businesses, or at least testing it in the context of proofs of concept or (often limited) internal trials. Notably, several operators are currently experimenting with such technologies on their own or in direct collaboration with particular suppliers. However, to the best of our knowledge, there has not been any general and reusable AI-augmented approach intending to support full continuous software and system engineering processes in the context of different use cases and application domains.
AIDOaRt (AI-augmented automation for efficient DevOps, a model-based framework for continuous development At RunTime in cyber-physical systems) is a 3 years long H2020-ECSEL European project involving 32 organizations, grouped in clusters from 7 different countries, focusing on AI-augmented automation supporting modelling, coding, testing, monitoring and continuous development of CPSs. The project proposes to apply Model-Driven Engineering (MDE) principles and techniques to provide a framework offering proper AI-enhanced methods and related tooling for building trustable CPSs. The framework is intended to work within the DevOps practices combining software development and information technology (IT) operations. In this regard, the project points at enabling AI for IT operations (AIOps) to automate decision making processes and complete system development tasks.
In the first part of this paper, we are going to present the overall AIDOaRt work plan and strategy, and then in the second part we are going to focus on our contribution at Prevision.io as a solution provider. Finally, we will present the future work in Prevision.io’s roadmap.
The AIDOaRt project will follow a continuous integration and continuous delivery (CI/CD) approach inspired by DevOps and depicted in Figure 1.
Figure 1 – The DevOps iteration path.
The main steps of this CI/CD path are the following:
- Technology plan: This step is concerned with the specification of requirements and use cases scenarios as well as the development plan and strategy, according to the global architecture of AIDOaRT;
- Technology development: This step concerns the actual development of the various architectural elements of the AIDOaRt framework;
- Technology integration: Once the architectural elements of the solution are developpent, they are effectively integrated into a common framework at this step.
- UC development: This step concerns the practical application of the AIDOaRT approach and framework in the context of the development of the use cases;
- UC execution and validation: This step includes the execution and validation of the results from steps 1 to 3.
- Evaluation & feedback: This step includes the analysis of the result of all the previous steps (and in particular work item 5) in order to provide evaluation/feedback that will be addressed in the next iteration of the whole AIDOaRT process.
Following this CI/CD development path and the timeline depicted in Figure 2, most deliverables of the project are going to be produced in three version:
- Initial: In the first iteration, we will focus on the prototyping and incremental research on top of the baseline technologies.
- Intermediate: In the second iteration phase, we will develop the intermediate version and deliver more results in terms of research.
- Final: We will consolidate the technology development based on the received feedback and finalize the UC development. We will end the third iteration by obtaining the final integration and evaluation.
Figure 2 – Timeline
In order to achieve the objectives of the project, a structural organisation has been established to break down and have a clear view of the workload. This structural organization is presented in the next section.
To ensure efficient collaborations between the 32 partners, the project has been structured into tasks, work groups, and work packages (see Figure 3).
Figure 3 – Overvue of the AIDOaRt structuration.
A task corresponds to a specific work to be done within a given time period and has one or several deliverables. For each task, a leader is assigned among the partners.
- Work groups
A work group corresponds to a collection of partners working on a specific topic of the project, namely:
- AI-augmented Requirements
- AI-augmented Modeling
- AI-augmented Coding
- AI-augmented Testing
- AI-augmented Monitoring
- AI-augmented Toolkit
- Work packages
A work package is a collection of tasks. The AIDOart project is made up of 6 main work packages:
WP1: Use Cases Analysis and AIDOaRT Solution Architecture sets up the context for the technological development in the project.
WP2: AIDoaRT Data Collection and Representation supports the management of the collected data that represents the initial input required to the AIDOaRT framework.
WP3: AIDOaRT Infrastructure and Framework provides the general infrastructure, framework and core services of the AIDOaRT platform.
WP4: AIDOaRT AI-augmented Toolkit supports the development of the AI-augmented toolchain that will extend the AIDOaRT core framework developed in WP3.
WP5: Integration and Use Case Evaluation coordinates the integration of technical developments from WP2, WP3 and WP4.
WP6: Dissemination and Exploitation deals with defining and executing dissemination and exploitation plans so that the results of the project will be successfully adopted, both for the duration of the project and afterwards.
It should be noticed that all partners work together regardless of the task, work groups and work packages.
Prevision.io is leading the task on explainability and accountability within WP3. It will lead to the intermediate version of the AIDOaRt Core Infrastructure and Framework deliverable. More details on this task are given in the following section.
Prevision.io is among the twenty five technology providers of the AIDOaRt project. We provide the following services through our end-to-end machine learning platform https://prevision.io: modeling, analyzing, deploying and monitoring. Consequently, Prevision.io’s services can be used at any DevOps phase:
- Modeling and Analyzing
Given real time or historical data traces, we provide tools to automatically extract insightful statistical information. The latter are then used to model and build machine learning models to address the following tasks: failure detection and prediction, pattern classification, predictive maintenance to cite a few. The models to be developed for the AIDOaRt framework will in turn be used to support the automated continuous development of CPSs.
Our platform proposes one click deployment of models through either custom web interfaces, or APIs to ease integration within already existing information technology (IT) systems. The added value of this service holds on the fact that many machine learning projects fail not because of the quantitative quality of the models, but because the deployment is not taken into account at the modeling phase.
Once the AI/ML models are deployed, we propose a bunch of tools for their monitoring including performance evaluation through adequate metrics, drift monitoring, resources consumption to cite a few. Similarly to the deployment, the monitoring of models is an important aspect that should be taken into account during the modeling phase. Indeed, it helps to know when a model is obsolete and needs to be retrained.
In addition to the preceding services, Prevision.io is leading the design and development of the AIDOaRt framework services which includes explainability and accountability of the developed AI/ML models.
AIDOaRt framework services: explainability and accountability
Explainability in the context of ML refers to the capacity of understanding the work logic in ML algorithms. Accountability on the other hand refers to the need to explain and justify one’s decisions and actions to its partners, users and others with whom the system interacts . These services will be key features of the framework as there will be a need for users to understand AI/ML models’s outputs. Indeed, depending on the complexity, an AI/ML model can be interpretable/explainable. For instance, while simple models such as linear/logistic regression models are interpretable, black-box models such as deep neural networks may need to be explained. For further discussions on this topic, we refer the readers to Mathurin Ache’s articles on this topic. Along with the explainability, Prevision.io will ensure the development of a responsible framework through the accountability service which aims to justify based on explanations the decisions and actions to be taken by users interacting with the framework.
The Gantt chart in Figure 5 shows the overall efforts of prevision.io in terms of Person/Months (PM) disseminated among the work packages.
Figure 4 – Prevision.io’s efforts
As the leader of the explainability and accountability services of the AIDOaRt framework, we plan to conduct a systematic literature review on explainable AI/ML methods similar to the one done on the AIDOaRt dimensions (not discussed in this article). This study will help us to kown the recent literature on the topic and the techniques used to explain black-box models. Once this study is completed, we are planning with the other partners to develop and integrate an explainable module in the framework.
In this article, we presented the AIDOaRt project which aims to develop a framework for supporting CI/CD of CPSs by using MDE, DevOps and AI/ML techniques. We first presented the context and overview of the project. After that, we discussed Prevision.io’s contributions as a solution provider with its end-to-end machine learning platform on one hand and as the leader of the development of a responsible AI/ML system that will provide explanations and accountability of the models. Finally, we presented some of the upcoming work with a focus on Prevision.io’s.
Special thanks to all AIDOaRt consortium members that have worked on the proposal on which this paper is based. The AIDOaRt project has received funding from the ECSEL Joint Undertaking (JU) under grant agreement No. 101007350.
 Adadi, A., & Berrada, M. (2018). Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE access, 6, 52138-52160.