Options
Organising AI for safety : Identifying structural vulnerabilities to guide the design of AI-enhanced socio-technical systems
Gazos, Alexandros; Kahn, James; Kusche, Isabel; u. a. (2025): Organising AI for safety : Identifying structural vulnerabilities to guide the design of AI-enhanced socio-technical systems, in: Safety science, Amsterdam [u.a.]: Elsevier, Jg. 184, Nr. 106731, S. 1–13, doi: 10.1016/j.ssci.2024.106731.
Faculty/Chair:
Author:
Title of the Journal:
Safety science
ISSN:
0925-7535
1879-1042
Publisher Information:
Year of publication:
2025
Volume:
184
Issue:
106731
Pages:
Language:
English
Abstract:
Artificial intelligence is increasingly being integrated into socio-technical systems. Existing design principles for ethical, safe and trustworthy AI tend to be highly abstract and focus on AI systems in isolation. They have rarely considered the adverse effects on safety that may emerge from interactions between AI and other technical components. Organisational theories of safety take such emergent outcomes of interactions between entities in socio-technical systems into account. They offer guidance on how to identify structural vulnerabilities in socio-technical systems enhanced by AI, and how to organise the design and operation of such systems for safety. In this paper, which is the result of a collaboration between sociologists and computer scientists (AI consultants), we conduct an analysis that can support the process of designing AI-enhanced autonomous systems in order to avoid structural vulnerabilities. It builds on organisational theories of safety and derives five key descriptors from them, the examination of which can guide the design of AI-enhanced systems. We demonstrate the utility of the descriptors by applying them to proposals for AI-enhanced critical functions in advanced microgrids. We discuss these proposals from the research literature on microgrids and review their effects on structural vulnerabilities. We then explore the implications that go beyond the example of advanced microgrids and propose steps for reviewing and reflecting on structural vulnerabilities that AI controllers may introduce into socio-technical systems.
GND Keywords: ; ;
Künstliche Intelligenz
Microgrid <Energietechnik>
Systementwurf
Keywords: ; ; ; ; ;
Socio-technical system
Organisation
Safety
Microgrids
Artificial intelligence
Design
DDC Classification:
RVK Classification:
Peer Reviewed:
Yes:
International Distribution:
Yes:
Type:
Article
Activation date:
December 19, 2024
Versioning
Question on publication
Permalink
https://fis.uni-bamberg.de/handle/uniba/105485