D.05.06. Techniques and Tools for Empirical Analysis Vs2.0 incl. Handbooks

The main objective of WP5 within HoliDes was to develop methods, techniques and tools (MTTs) for the empirical analysis of Adaptive Cooperative Human Machine Systems (AdCoS). This last deliverable thus presents the final state (version 2.0) of the developed MTTs and shows their integration into the AdCoS development process and the Human Factors Reference Technology Platform (HF-RTP). WP5 was designed to receive requirements from the application work packages 6-9 and from the HF-RTP definition in WP1. WP5 partners worked in close cooperation with the experts from WP2 (Modelling), WP3 (Adaptation) and WP4 (model-based Human Factor Analysis).
In total, 14 human factors MTTs with different applications in the design, development and operation of AdCoS’ have been engineered, adapted or extended in this work package. Of these, four can be considered as nonlife cycle MTTs and will be integrated into cars or aircrafts to detect and determine operator states. One innovative example of these is the cognitive distraction classifier developed by TWT that has been integrated into the IAS AdCoS in the automated overtaking use case. In addition, WP5 produced ten life-cycle MTTs that aid the design and development process of AdCoS at certain stages. These include quantitative and qualitative empirical methods (e.g. Theatre Technique by DLR or SNV’s focus groups), task analysis techniques and tools (e.g. the HF-TA by HFC) or tools for capturing and structuring the human-centred design process (e.g. HF Filer by AWI). Use cases from each of the four domains (Healthcare, WP6; Aeronautics, WP7; Control Room, WP8; Automotive, WP9) have been served by WP5, ensuring that the developed Human Factors MTTs will be applied and exploited in a wide range of applications.
In the course of the project, WP5 had many contacts and interactions with the other scientific WPs 2-4. In particular, the task and human models created in WP2 were informed by the task analysis techniques and empirical methods created in WP5. Workshops between the work packages helped sharpening the concepts commonly used, and create a common vision for task and human modelling. Empirical data from WP5 also helped to create algorithms and methods developed in WP3. In turn the non-life-cycle methods of WP5 can be utilized to create adaptive, context and user aware functionalities. Finally, for verification and validation the model-based evaluation MTTs of WP4 were complemented by the empirical evaluation techniques created in WP5. Examples for this collaboration effort can be found in the workflows and accompanying sections in 5.3 and 5.7.
The main challenges of WP5 were the integration of the MTTs into the HFRTP and the demonstration of the collaboration with the partners from WP2-4. The former challenge comes from the fact that many of the MTTs of WP5, especially the empirical methods and task analysis approaches need manual and cognitive input from a human factors expert to exploit their full potential, which renders an OLSC compliance at all processing steps demanding. Thus, it was decided to show the usage of the MTTs within workflows for AdCoS design and development (see section 5).
These workflows comprehensively show at which point of the process the MTTs should be utilized, thus aiding the exact tailoring of the HF-RTP.
A second measure to tackle the OLSC compliance was taken by AWI in designing the HF Filer (see sections 5.1 and 6.2), a tool that allows to document information from the complete human-centred design process and save these pieces of information in an OLSC-compliant format. The latter challenge arose as, in the beginning, the interplay of the developments of WP2-4 with WP5’s MTTs was only apparent when assuming an application- or use-cased based approach. However, this was overcome as the workflows also make interactions between developments of different WPs apparent and therefore demonstrate how the results of WP2-4 and WP5 act together.