The challenges for artificial intelligence and systems engineering

ABSTRACT

Artificial Intelligence (AI) has the potential to offer huge improvements both to the performance and the efficiency of systems. When engineering any system, it is important to ensure that the user of the system can be confident that the behaviour and performance of that system is as needed. AI presents challenges to achieveing this confidence when introduced into a system. This paper broadly considers these challenges, including the new failure modes exhibited by AI, the unpredictable and evolving behaviour of AI systems and the inability to explicitly validate due the level of trust in the systems and its robustness to future environments. Traditional system assurance techniques have proved ineffective to address these challenges. The paper then looks at potential solutions that are currently being developed that have the potential to address this need and assist in the assurance of the performance of AI enabled systems throughout their lifecycle.

KEYWORDS:

• Artificial intelligence
• system assurance
• Autonomous Systems
• System Engineering

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Jake Vanderlinde

Jake Vanderlinde is an analytically driven engineer with qualifications and experience in aerospace, automotive and systems engineering. He specialises in solving complex problems including implementing new and novel technologies to meet the needs of his clients. Jake is a practicing senior systems engineer with Shoal Group based out of Melbourne, Australia.

Kevin Robinson

Kevin Robinson is a Systems Engineer with a distinguished career in both the UK’s Ministry of Defence and Australia’s Department of Defence. He has made significant contributions to the development of advanced and novel technologies through modelling and analysis, research, and leadership of large cross discipline teams. Kevin is Shoal’s engineering lead providing technical leadership and guidance across the business.

Benjamin Mashford

Dr. Benjamin Mashford is a research scientist with qualifications and experience in physics, data science and machine learning. He is a graduate of University of Melbourne and has worked as a research scientist in several academic and commercial research environments, including IBM Research and Australian National University. He is now the Machine Learning Lead at Shoal where he provides expertise to a range of internal and client projects.