SoTeRiA Laboratory researchers in NPRE combine computational modeling and simulator experiments to support decisions about the risks, benefits, and deployment of AI-enabled technologies.
What evidence do regulators need to evaluate AI-enabled technologies for nuclear power plants? How can their risks be assessed in the licensing of advanced reactors? How can plant managers weigh their benefits against their risks—and how should operators use AI recommendations?
Researchers in the Socio-Technical Risk Analysis (SoTeRiA) Research Laboratory at the University of Illinois Urbana–Champaign are developing risk analysis methods to address these questions. Directed by Zahra Mohaghegh, professor and Donald Biggar Willett Faculty Scholar in the Department of Nuclear, Plasma, and Radiological Engineering (NPRE), the laboratory evaluates AI within its broader socio-technical context, accounting for interactions among AI, human performance, and plant systems.
Across projects supported by the U.S. Department of Energy (DOE) and the National Science Foundation (NSF), the team has developed methods for evaluating AI trustworthiness and transparency and is generating experimental evidence about human–AI interaction.
The research brings together lead Ph.D. student Muhammad Hammad Khalid, research scientists Ha Bui and Seyed Reihani, Ph.D. student Md. Istiaque Ahmed, and Professor Zahra Mohaghegh, with Idaho National Laboratory (INL) collaborators Drs. Ahmad Al Rashdan and Ronald L. Boring.
Connecting AI Trustworthiness to Plant Risk
An AI system’s accuracy provides useful information, but its consequences depend on the application. A missed warning or false alarm may have different implications depending on plant conditions and human responses.
To examine these relationships, SoTeRiA researchers developed the Socio-Technical Risk-Informed AI-Driven Automation Trustworthiness Evaluation framework, known as ST-RATE, through a DOE Nuclear Energy University Program project in collaboration with INL.
ST-RATE Part 1: Conceptualization, published in Risk Analysis, defines trustworthiness in terms of confidence that an AI-driven automation system will function as expected across its conditions of use. It establishes three foundational pillars: application-specific performance expectations, socio-technical conditions of use, and uncertainty-based evaluation of confidence.
The companion paper, “Socio-Technical Risk-informed AI-Driven Automation Trustworthiness Evaluation (ST-RATE): Part 2, Methodological and computational framework, and application to an AI-driven Automated Firewatch system in nuclear power plants,” accepted for publication in Risk Analysis, translates this foundation into a methodological and computational framework. It integrates models of AI, human performance, and physical phenomena with plant probabilistic risk assessment (PRA), which examines what can go wrong, how likely it is, and what the consequences could be.
An AI-driven automated firewatch case study demonstrates the framework’s feasibility. The research incorporates virtual reality to generate synthetic, physics-informed data and draws on SoTeRiA’s Probabilistic Validation methodology to evaluate confidence when empirical evidence is limited.
Understanding AI Transparency, Human Trust, and Performance
AI trustworthiness and human trust in AI are related but distinct. Trustworthiness concerns evidence supporting confidence in system performance; human trust influences whether, and how, people rely on it. Evaluating human–AI interaction therefore requires examining what information an AI system conveys and how that information affects people’s decisions.
The team addresses automation transparency in Automation transparency: A literature review, methodology development, and application to an AI-driven anomaly detection system in nuclear power plants, published in Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability. The paper proposes a definition of automation transparency and develops three evaluation approaches, examining their feasibility and limitations through a hypothetical anomaly detection case study.
Building on this foundation, an ongoing project supported by an NSF Mid-Career Advancement award investigates how AI transparency and human trust influence human performance through controlled experiments in a full-scope Generic Pressurized Water Reactor (G-PWR) simulator.
Participants act as control room operators working with an AI predictive-maintenance assistant. They detect abnormal reactor coolant pump behavior, assess the situation with AI assistance, and recommend maintenance actions. The experiments examine human error probabilities, engagement with AI predictions, and reliance on recommendations under different transparency and trust conditions.
The broader research connects human decisions and AI predictions with component degradation models to examine implications for plant safety and financial performance. This work aims to strengthen human reliability analysis by explicitly accounting for human–AI interaction.
Together, these efforts span conceptual development, computational methods, application demonstrations, and human-performance experiments. They can help plant managers evaluate risks and benefits, developers prioritize testing and improvements, and regulators assess technical evidence for proposed applications.
Explore the research: SoTeRiA’s Automation Trustworthiness and Transparency research page brings together project details, methodological frameworks, case studies, and additional publications.