Background

The aviation industry faces massive challenges. It is still struggling with the aftermath of the pandemic and it belongs to those industrial sectors that have to put a great deal of effort into achieving climate targets. However, climate neutrality and sustainability should not come at the expense of high safety standards. Handling such a situation, where different demands need to be met, requires extensive testing. This is particularly true in software development. Because software tests in the aviation industry are often difficult to organize and as a result cost-intensive, in the two projects VAST and EMMSA we already tried to simplify procedures and find ways to reduce costs. The Cross-Level-User Evaluation (CLUE) project builds on the knowledge already acquired and collaborates with Fraunhofer Austria Research and 4D Aerospace Research and Simulation on developing new concepts for software testing in the aerospace industry.

Project Content

Software tests in the aviation industry require complex designs and are expensive. The main reason for this is that test persons are qualified personnel in nearly all of the cases (pilots and air traffic controllers). The number of tests often remains low, which undermines their validity and makes it more likely that aspects such as user-friendliness are neglected. In the precursor project EMMSA we tried to overcome these problems by using students as test subjects for prototype development/evaluation. The CLUE project uses the insights already gained and intends to further optimize software testing of human-computer interfaces in the aviation industry. Methodically sound software tests considering a wide range of different aspects are a prerequisite for easy-to-use and visually appealing user interfaces, which have a decisive influence on whether a new software is well accepted.

Goals

Currently, the airline industry mainly uses expert personnel for software testing. This is expensive and cumbersome. The project team of CLUE aims to remedy this situation by establishing new test-procedures that make use of non-professionals as test persons without undermining the quality of tests. The main aims are:

• Investigating to what extent people outside the aviation industry (non-experts such as students) can be used for tests of air traffic control software systems and to what extent it is possible to save costs and resources on the basis of such an approach.
• Increasing the validity of the tests because larger samples and more tests are possible when using non-experts as testers.
• Advancing digitization in aviation in a way that attaches great importance to the needs of users (human-centered design).
• Contributing to emission reduction in aviation through new and better software systems.

Methodology

The CLUE project reduces the workload for expert personnel with regard to software testing, because tests are mainly carried out with laypersons. Despite relying on non-experts, the validity of the tests is increased. This is made possible by HCI tests (Human Computer Interaction) that make use of statistical correction measures coming from sociology, political science, and meteorology. The test procedures are also set up in a way that takes into account possible future automation processes through machine learning and artificial intelligence (AI). This is a crucial step, since AI technologies enable new forms of cooperation between humans and machines that not only concern technical but also ethical aspects. For this reason, the test procedures are designed along ethics guidelines developed by the EU, which stipulate that assessment lists on ethical issues in the development, introduction and use of AI systems have to be compiled. Although non-experts are used in the tests, we see to it that they have as much domain affinity as possible (e.g., students of gaming design who have experience with MS Flight Simulator, X-Plane or similar).

Result

The CLUE project places software testing in the aviation industry on a new footing. Non-experts are used as test persons and this reduces the workload for technical personnel and as a consequence lowers the costs. To obtain high quality test results, the project team employs methods they borrow from the social sciences and meteorology. CLUE also sets great store by Human Centred Design, thereby ensuring that the resulting software products are as user-friendly as possible. Overall, CLUE helps the aviation industry to arrive at easier-to-use software products, which are urgently needed on the way to a more climate-friendly future.