Shielding

Project Overview

This project focuses on the development of shielding techniques to ensure the safety and reliability of AI systems, specifically in the context of neural network controllers, such as those derived from deep reinforcement learning (DRL). Shielding is a method used to protect systems under black-box controllers by relying on simpler, verified controllers that can intervene when the neural network controller might make unsafe decisions. We aim to develop robust shielding mechanisms that ensure the safety of AI systems in critical applications while preserving the advantages of neural network control.

Objectives

• Shielding techniques development: create and refine methods for shielding neural network controllers, ensuring their safe operation by providing fallback mechanisms through simpler, verified controllers.
• Safety and reliability: investigate and implement strategies that allow the neural network to perform complex tasks while ensuring system safety, especially in high-stakes applications such as autonomous driving.
• Verification and testing: conduct verification of the shielding mechanisms to ensure that the safety requirements are consistently met across a range of scenarios and system dynamics.

Deliverables

• Shielding framework: a developed and tested framework for integrating shielding techniques into neural network controllers.
• Safety verifications: documentation and results from extensive safety verification testing, demonstrating the shielding technique’s effectiveness.
• AI system integration: working prototypes of AI systems (such as autonomous vehicles or other safety-critical systems) that utilize the developed shielding mechanisms.
• Codebase: well-documented code for implementing and testing shielding techniques, along with setup instructions and usage guidelines.