___________________________________________________________________
PAUSA Partage d’Autorité dans le système aéronautique
PAUSA for the future A synthesis of Phase 1 ___________________________________________________________________ Principal Authors:
Sonja Straussberger (Eurisco International) Guy Boy (Eurisco International) Sebastien Barjou (DSNA-DTI) Sylvie Figarol (DSNA-DTI) Franck Salis (Dassault Aviation) Serge Debernard (LAMIH) Patrick Le Blaye (Onera)
Contributors: The PAUSA-Team
___________________________________________________________________ Issued: June 2008 Final Report
Report documentation page Project Description Project: Reference: Document Description Document reference: Document classification:
PAUSA Convention n°: 06.2.93.0583 PAUSA-TR-E4.F TR
Document status:
Final report
Key words:
Authority sharing, authority distribution, delegation, 4D contract, scenario, future ATM Eurisco international (Project coordination) Prof. Dr. Guy Boy EURISCO International 23, avenue Edouard Belin BP. 44013 31028 Toulouse Cedex 4 tel: +33 5 62 17 38 30 fax: +33 5 62 17 38 39 email:
[email protected] www: http://www.eurisco.org
Originator: Contact:
Approval Approver #1:
N.A.
Approver #2:
N.A.
Straussberger, Boy, Barjou et al., 30-06-08
The PAUSA Team Airbus France
Florence Reuzeau (Aeronautics Engineer) Philippe Pellerin (Test pilot)
Dassault Aviation
Franck Salis (Aeronautics Engineer)
DSNA-DTI
Sylvie Figarol (Ergonomist) Sebastien Barjou (Ergonomist) Alain Serres (ATC Operational Expert) Claude Chamayou (ATC Operational Expert) Christelle Pianetti (Simulation Engineer)
Eurisco International
Guy Boy (Coordinator) Sonja Straussberger (HF Expert, Psychologist) Benoit Guiost (Automation Engineer) Beatrice Feuerberg (HF Expert, Psychologist) Jean-Yves Lantes (Computer Scientist) Gudela Grote (Associate Senior Researcher cooperation with ETH Zuerich)
Oktal
Christophe MEDARD (Simulation Engineer) Alain Narbonne (Simulation Engineer) Franck Perret
ONERA
Jean-Loup Farges (Research Engineer) Patrick Le Blaye (Research Engineer) Laurent Chaudron (Research Engineer)
LAMIH
Serge Debernard (Automation Researcher)
INRIA-LORIA
Didier Fass (Researcher) Francois Charpillet (Computer Scientist) Vincent Chevrier (Computer Scientist) Amine Boumaza (Computer Scientist) Guillaume Muller (Computer Scientist)
Thales Avionics
Oanh LE (Aeronautics Engineer) Philippe Salmon (Aeronautics Engineer)
PAUSA-TR-E4.F – Page 3 / 58
in
Straussberger, Boy, Barjou et al., 30-06-08
Acknowledgements As a long-term project, setting up and conducting the PAUSA project required intense efforts during several years. The PAUSA team would like to thank numerous contributors willing to share their knowledge and experience to enable a better global understanding of how air and ground activities interact: •
Christine Savy representing our main sponsor Direction Générale de l’Aviation Civile (DGAC);
•
Dominique Colin de Verdière (DGAC) and Patrick Lelièvre (Airbus France) for providing in their SESAR-oriented perspective;
•
Enroute Controllers from Bordeaux ACC and operational, instructor, retired, and test pilots (Daniel Action, Pierre Lebrun) as operation and training engineer (JeanBernard Boura) from Dassault Aviation, ENAC, Air France and others (Alain Favresse, Jean-Philippe Ramu) for providing in their operational perspective;
•
The team of Patrick Gilbert and Geraldine Schmidt from IAE Paris and Alexandra Moes and Carine Sonntag from ICN Graduate Business School;
•
Representatives from Eurocontrol and DGAC who joined discussions at the Summer Workshop in St. Lary (wherefrom Manfred Barbarino, Stephane Deharvengt, Bernd Lorenz, Herman Nijhuis);
•
Representatives from various European research institutions and industries for discussing current PAUSA outputs at the Final PAUSA Workshop in Paris;
•
All the friendly people taking care of the PAUSA-Team during various visits (CDG approach and tower, DSNA-DTI simulators, etc.).
Finally, we would like to thank the staff at the partner organizations who provided administrative support and in particular Cheikh Sow and Helen Wilson.
PAUSA-TR-E4.F – Page 4 / 58
Straussberger, Boy, Barjou et al., 30-06-08
PAUSA for the future A synthesis of Phase 1
Abstract In the first phase of the PAUSA project, nine French organisations representing aeronautical operations, industry and research were unified to come up with an approach for dealing with authority sharing and distribution in air traffic management (ATM). A human-centered design approach is essential to understand the impact of automation in the context of increased air traffic and requires taking both a global organisational as well as a local agent perspective. A systematic approach consists of using a scenario development method for obtaining consensus- and experiencebased scenarios, a model-based air and ground Human Factors (HF) framework, functional analysis and allocation methods enabling cooperation, and analytical and simulation-based evaluation integrating air and ground actors. The scope is to enable the elicitation of emerging cognitie functions during task execution and allow identifying countermeasures for achieving a safe and performing ATM system. A continuation of the PAUSA project should naturally follow to accompany the implementation of the future concepts of operation within SESAR along all implementation packages. The PAUSA team is open to support such type of activities and enable systematic exchanges within the ATM community.
PAUSA-TR-E4.F – Page 5 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Extended Summary In the first phase of the PAUSA project, nine French organisations from airborne and groundside, representing operations (DSNA-DTI), industry (Airbus, Dassault Aviation, Oktal, Thales) and research (Eurisco, Inria-Loria, LAMIH, Onera) were unified to come up with an approach on how to deal with authority sharing and distribution in air traffic management (ATM). Within the implementation of automation solutions to deal with air traffic increases, a human-centered design approach is essential to identify black holes in the expected system. For example, such a black hole was found in the past, where independently designed safety nets, for example the airborne Traffic Collisoin Avoidance System (TCAS) and groundside Short Term Conflict Alert (STCA) were allowing for overlapping time spans and consequently contradictory decisions. In such a setting within separation and collision avoidance, the problem of authority, which can be briefly summarized as having the power to act, needs to be tackled from a human-machine multi-agent perspective in a global organisational as well as a local agent perspective. A framework distingushing control and accountability elements in the notion of authority supports the classification of problems along two dimensions in relation to design and in relation to operations. Authority sharing and authority distribution lead to the identification of the appropriate common frame of reference and static function allocation during the design process. Within the operational context, authority delegation and authority trading represent settings that allow to put in place contracts between airborne and groundside human and machine agents and negotiating compliance conditions. As a means to address a future ATM system, consensus-and experience-based scenarios were identified as a signifcant approach to accompany the design and development process with iterative analytical and simulation-based evaluation means. Through understanding emerging cognitive functions when executing tasks, solutions and countermeasures can be reflected sufficiently early to direct the development process towards a safe and performing ATM system. A systematic approach consists of the following elements summarized in the PAUSA Toolkit: •
A proposed scenario development method;
•
A socio-cognitive model integrated in a first prototype support tool;
•
An itegrative Human Factors (HF) Framework combining frameworks used by ATC organizations and aircraft manufacturers detailed with model-based criteria and metrics for the evaluation of integrated air and ground segments;
•
A functional analysis and function allocation method using collaborative workspace principles to design cooperation between a certain number of human and machine agents; PAUSA-TR-E4.F – Page 6 / 58
Straussberger, Boy, Barjou et al., 30-06-08
•
Guidelines for iterative evaluation between analytical studies and operational simulations.
Naturally, similar to technological systems designed for the future ATM, also the PAUSA method requires iteration and maturation. A continuation of the PAUSA project should accompany the implementation of the future concepts of operation within the SESAR program. Depending on the advancement of defined implementation packages (IP), PAUSA suggests setting a focus on testing existing cognitive functions (IP1), eliciting emerging cognitive functions (IP2), or refining new cognitive functions (IP3). Also in the future, the multi-domain interdisciplinary PAUSA team is open to support such type of activities and puts in place specific means to enable systematic exchanges and support the advancement within the ATM community.
PAUSA-TR-E4.F – Page 7 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Table of Content EXTENDED SUMMARY............................................................................................. 6 LIST OF TABLES..................................................................................................... 10 ABBREVIATIONS .................................................................................................... 11 INTRODUCTION ...................................................................................................... 12 A THE PAUSA PROJECT...................................................................................... 13 B A GENERAL PERSPECTIVE ON PAUSA ......................................................... 15 B.1 B.2 B.3 B.4 B.5 B.6
PAUSA BETWEEN CONCEPT DEVELOPMENT AND VALIDATION............................... 15 NEED FOR UNDERSTANDING AUTHORITY ISSUES .................................................. 17 OUTPUTS OF PAUSA....................................................................................... 23 PAUSA VISION ................................................................................................ 25 COSTS AND BENEFITS OF WORKING WITH PAUSA .............................................. 26 OPEN QUESTIONS AROUND PAUSA................................................................... 27
C THE PAUSA TOOLKIT FOR DESIGN AND EVALUATION............................... 28 C.1 THE DEVELOPMENT OF HUMAN-CENTRED SCENARIOS .......................................... 28 C.1.1 Why scenarios ......................................................................................... 28 C.1.2 The PAUSA scenario development method............................................. 28 C.1.3 Examples of possible scenarios............................................................... 31 C.2 THE PAUSA SUPPORT TOOL ............................................................................ 33 C.2.1 Why support PAUSA?.............................................................................. 33 C.2.2 How the support tool works: a descriptive socio-cognitive model representation.................................................................................................... 34 C.3 DESIGNING COOPERATIVE HUMAN-MACHINE SYSTEMS.......................................... 37 C.3.1 Functional analysis .................................................................................. 37 C.3.2 Function allocation and task delegation in a collaborative system........... 40 C.4 CRITERIA AND METRICS .................................................................................... 46 C.4.1 HF Issues integrating needs of air and ground actors.............................. 46 C.4.2 Frames and models supporting the iterative evaluation process ............. 47 C.5 AN ITERATIVE EVALUATION APPROACH BETWEEN ANALYSIS AND HUMAN IN THE LOOP SIMULATIONS ........................................................................................................... 51 D CONCLUSION .................................................................................................... 54 REFERENCES ......................................................................................................... 55 PROJECT-RELATED DOCUMENTATION AND DISSEMINATION ........................................... 55 EXTERNAL REFERENCES .......................................................................................... 56
PAUSA-TR-E4.F – Page 8 / 58
Straussberger, Boy, Barjou et al., 30-06-08
List of Figures Figure 1. Iterative Design/Evaluation Model.............................................................. 15 Figure 2. The PAUSA toolkit ..................................................................................... 16 Figure 3. Illustrating cognitive functions .................................................................... 17 Figure 4. Processes characterizing authority ............................................................ 19 Figure 5. The PAUSA-Methodology.......................................................................... 23 Figure 6. The PAUSA Methodology for SESAR Implementation packages .............. 24 Figure 7. Scenario Development Method.................................................................. 29 Figure 8. Scenario context ........................................................................................ 32 Figure 9. Concepts integrated in PAUSA-Support Tool ............................................ 34 Figure 10. Interaction block illustration...................................................................... 36 Figure 11. PAUSA-Tool screenshot with an example of the interaction block representation for air and ground activity in an ASAS S&M scenario sequence.37 Figure 12. Agents' activities in delegation ................................................................. 48 Figure 13. The evaluation of HF issues in a multi-agent setting................................ 48 Figure 14. The description of perception (P), decision (D), action (A) and control (C) processes at the agent level .............................................................................. 49 Figure 15. The human agent in context..................................................................... 50 Figure 16. Analytical and HITL simulations............................................................... 52
PAUSA-TR-E4.F – Page 9 / 58
Straussberger, Boy, Barjou et al., 30-06-08
List of Tables Table 1. Functional analysis grid and definition of elements..................................... 38 Table 2. Information with regard to activities during cooperation .............................. 45 Table 3. HF Issues.................................................................................................... 46
PAUSA-TR-E4.F – Page 10 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Abbreviations ATM
Air Traffic Management
ANSP
Air Navigation Service Provider
ASAS
Airborne Separation Assurance System
ATC
Air Traffic Control
ATCO
Air Traffic Controller
ATSAW
Airborne Traffic Situational Awareness
E-OCVM
European-Concept Validation Methodology
HF
Human Factors
HITL
Human-In-The-Loop
HMI
Human-Machine Interface
IP
Implementation Package
KPA
Key Performance Area
MAS
Multi-agent System
MTCD
Medium-Term Conflict Detection
SCS
Socio-cognitive stability
SESAR
Single European Sky Aviation Research
STCA
Short Term Conflict Alert
TCAS
Traffic Collision Avoidance System
PAUSA-TR-E4.F – Page 11 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Introduction The issue of authority sharing and distribution has become a major point for the development of the future air traffic management (ATM) system. As it turned out in previous incidents and accidents - with the Ueberlingen accident as one of the most frequently cited – in the current system these issues are not well considered and described. In the context of ongoing air traffic increases and automation trends, it might however represent an essential concept. To better understand the underlying problems, nine French organisations from aeronautical industries, institutional and research organisations, started a project that specifically addressed the meaning and consequences of this concept. At the same time, within the European setting, the SESAR program defined task delegation and authority sharing as core concepts to be considered for designing operational concepts and associated technologies and procedures. As a major initiative to address this subject, the combined work and research effort enables PAUSA to provide first answers in relation to problems regarding role and task distribution, an appropriate methodology, and issues regarding human performance. PAUSA does not only provide answers, but also demonstrates the ways to overcome past constraints in system design and development processes. A description of what PAUSA is and what it wants to achieve is thus presented in three parts of this report: Part A provides a brief introduction of the key objectives of PAUSA. Part B describes the concrete meaning of authority problems in ATM and introduces a taxonomy that supports its development. A suggestion for a methodology supporting the human-centred design and evaluation process will provide help to solve upcoming ATM issues and relates PAUSA to the European context within the SESAR programme. Part C describes the content and application of the PAUSA toolkit, which can be implemented at the different stages. The essential methods are the definition of scenarios, underlying socio-cognitive models, and criteria identifying implied human performance and evaluation strategies. Part D presents conclusions on how to continue PAUSA in the future.
PAUSA-TR-E4.F – Page 12 / 58
Straussberger, Boy, Barjou et al., 30-06-08
A The PAUSA project Authority sharing and delegation is recognized as a major issue in ATM. To tackle this subject, the PAUSA (Partage d’AUtorité dans le Système Aéronautique – Authority Sharing in the Aeronautical System) project started as a French national endeavour during the 18 months of Phase 1 from October 2006 to Mars 2008 and federated a set of actors as well as competences in aeronautics, computer science, automation, and human factors. By providing aeronautical industry and governmental agencies a global perception of performance and ATM of the future, the range of possibilities related to interaction between different actors - whether humans or technological -, is explored and innovation and investment strategies proposed. PAUSA is a human factors project and focuses on the definition and consistent articulation of various elements and actors of the global system combining airborne and groundside. It poses the problem of responsibility distribution between actors, as well as the nature and automation levels of the aeronautical system with respect to roles and activities of human operators (controllers and pilots for example). Specification and certification of new technologies to be designed should take into account human factors to optimize the global system integrating various humanmachine components on ground side and airborne. Taking into account authority sharing at either the local level or the global level of the system is likely to optimize airspace use, and even increase its capacity and safety. The objective of PAUSA is to develop a theoretical approach of authority sharing. Models are incrementally validated from the experimental implementation of the various task allocation scenarios that enable the coverage of a set of possible configurations and chronologies. These scenarios have been developed by both ground and airborne specialists proposing modes of automation for three periods, i.e., 2009, 2015 and 2020+. Due to the high number of parameters to be taken into account, the first phase of the PAUSA project consisted in: •
Developing a taxonomy/ontology of the domain centred on authority sharing;
•
Developing socio-cognitive models that support the overall problem of authority sharing in the ATM;
•
Defining different target scenarios that will be used to support human-in-the-loop (HITL) simulations in the next phases;
•
Elaborating and proposing a set of socio-technical principles leading to human factors and organizational issues;
•
Applying the resulting approach to separation and collision avoidance problems (airborne and/or ground) within two specific situations: en-route and approach zones.
PAUSA-TR-E4.F – Page 13 / 58
Straussberger, Boy, Barjou et al., 30-06-08
The PAUSA Phase 1 project associated complementary French national competences to reach these objectives: aeronautics (Airbus, DSNA/DTI, Dassault Aviation, Thales Avionics), including HF experts and operational experts (pilots and air traffic controllers); automation, computer science and human-machine systems (CNRS/LAMIH, ONERA, Oktal); cognitive sciences and cognitive engineering (EURISCO International, INRIA/LORIA). PAUSA was managed by EURISCO International whose expertise is on the interface of all these domains. After 18 months of research and development, creativity and analytical assessment of scenarios and socio-cognitive models of the ATM, PAUSA puts the participants on the forefront of both a European and international level to propose a human-centred framework for the ATM of the future. This human-centred framework is the subject of the current report.
PAUSA-TR-E4.F – Page 14 / 58
Straussberger, Boy, Barjou et al., 30-06-08
B A general perspective on PAUSA B.1 PAUSA between concept development and validation Among the different tasks carried out in PAUSA 1, the PAUSA partners felt it was necessary to provide an overview of these tasks around a single and synthetic view. The objective is to make the PAUSA process easily understandable by the reader through describing how each PAUSA task and deliverable intervenes and helps during the design process of new ATM components. For that purpose, it was agreed that the synthetic view around which the different PAUSA tasks would be integrated, should be the model1 used for the design/evaluation of human/machine systems: i.e. the model of iterative design, described below.
y ilit b i as Fe
ns tio a ific ec p S
v De
elo
pm
en
t
on ati l l ta Ins
In
to
r Se
vic
e
Design process
Iterative evaluations : Literature, expert judgment, Fast time
sim., Real time sim.
Evaluation/Validation Process
Figure 1. Iterative Design/Evaluation Model
The design of systems with human in the loop differs from the design of pure technical systems on some points: •
Technical systems generally offer a good stability in terms of input/process/output. However, they are not able to adapt to non-anticipated events.
1
Several terms used in PAUSA implicitly evoke different and partly contradictory interpretations by the interdisciplinary users that the team needed to aware of. For example, the term “model” is used in an open sense representing a simplified representation of the world. Thus, the models used by PAUSA are of a descriptive, predictive, prescriptive, and analytical nature. A similar problem appeared with the terms of simulation, which is defined as running a model. For example, a simulation can be numerical (human-out-of-the-loop) or real-time or operational (generally human-in-the-loop).
PAUSA-TR-E4.F – Page 15 / 58
Straussberger, Boy, Barjou et al., 30-06-08
•
On the contrary, systems with human in the loop may partly vary in terms of entries/process/output. Yet, they are able, most of the time, to invent solutions and adapt to non-anticipated events.
As a consequence of these differences, designing a system with human in the loop – because of its intrinsic variations – must be done with an iterative approach between design and evaluation: each time the design of the system has reached a new level of maturity, it must be tested, thus evaluated against a reality with a new level of fidelity. This is the model presented in Figure 1. This model will be the one around which the different PAUSA 1 tasks will be integrated, as proposed in Figure 2.
Figure 2. The PAUSA toolkit
Based on the representation introduced in Figure 2, PAUSA can be seen as an “Air/Ground – Human/Machine” toolkit needed during the design and evaluation of new ATM components. The PAUSA toolkit helps in different ways: 1. A mixed expert team from Air/Ground supports the capitalization of task sharing and the collection of regulatory issues. 2. A specific task allocation method supports the design process. 3. Scenarios, socio-cognitive modelling, criteria and metrics for HF issues, and Guidelines for operational simulations support the evaluation and validation process. We can conclude that PAUSA Phase 1 was dedicated to the elaboration of the “Air/Ground – Human/Machine toolkit”. However, even if this toolkit seems promising and was partly tested on some small parts, it has just reached a first level of maturity.
PAUSA-TR-E4.F – Page 16 / 58
Straussberger, Boy, Barjou et al., 30-06-08
The PAUSA toolkit needs to follow up, as any system with human in the loop (Figure 1), an iterative design/evaluation process. Consequently it must be now tested on a larger basis, to be able to reach a new level of maturity, more suitable for wider use in a European context. This may be the goal of the next phase of the PAUSA project: iterating the PAUSA toolkit and delivering an improved and validated version of it. After this period, the PAUSA toolkit could be envisaged to be used on a wider range in Europe.
B.2 Need for understanding authority issues Thinking ATM in terms of multi-agent system with emerging functions The characterization of ATM as a multi-agent system (MAS) paves the way to viewing problems from different perspectives. Agents as acting entities composing interacting board and ground segments can be characterized according to three dimensions: human operators, machine agents, and organizations. Prescribed tasks and effected activities are located between these dimensions. Each agent can be described in terms of cognitive functions that are used to execute tasks. The process of automation can be characterized as transferring cognitive functions from human to machine agents. This relates to the problem of emerging cognitive functions. As shown in Figure 3, the transfer of a function from a human agent on ground to an airborne machine agent not only results in the desired cognitive function to execute the task, but also in emerging human functions that might be of a compensating nature to optimize task execution.
Figure 3. Illustrating cognitive functions
If not sufficiently anticipated and understood, emerging cognitive functions may evoke non-appropriate task behaviour and thus directly or indirectly affect safety. Therefore, to make explicit such cognitive functions, PAUSA has developed a method applying an iterative analytical and simulated evaluation process supported by a socio-cognitive model to derive design recommendations.
PAUSA-TR-E4.F – Page 17 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Main authority definitions Emerging cognitive functions have a direct effect on authorities assigned to agents. Someone has authority when he or she has the power or right to give orders, make decisions and enforce obedience, which is when someone is in charge. The term authority comes from the Latin “auctoritas” that expresses adjunction, complement, or more directly augmentation. We often think of the right to act in a specific way, delegated from one person or organization to another, we get the official permission or authorization. Someone may also be recognized as an authority because of his or her expertise on a given subject. Again, an expert has something more than anybody else, i.e., validated knowledge and know-how. The concept of authority concerns people, organizations and technology. An organization always works according to a structure and function paradigm. The structure may be pyramidal such as an army where there is a general at the top, several layers of officers and sub-officers, and soldiers at the bottom. In such a structure, prescribed authority is expected to be incrementally transferred top-down. Prescribed authority may be different from emerging authority created from competence, expertise and knowledge. This is where it is important to define functions, because each agent in the structure has a function that requires appropriate competence, expertise and knowledge, i.e., the right function (and therefore agent) at the right place. Consequently, an agent will have authority when he or she will have and use appropriately his or her competence, expertise and knowledge related to his or her position in the structure. In this report, we will concentrate on two meanings of the concept of authority: control and accountability. Control is understood in terms of control theory used in engineering, where manipulating input parameters of a system leads to a desired effect of the output of the system, when for example controlling a machine or a moving object. When someone is mandated or empowered to control an entity then he or she has the authority over this entity. An entity could include human(s) and/or system(s). At the same time, he or she is accountable before another authority for the performance of his or her control. We see here the recursive nature of the concept of authority. Accountability includes responsibility. We say that someone is accountable when he or she is required or expected to justify actions or decisions that he or she is responsible for. Authority processes Fitting a function to a structure is interpreted as defining an agent job or role according to his or her competence, expertise and knowledge, as well as his or her impact on the organization, i.e., the agency that the agent belongs to. This is a matter of mutual adaptation of individual agents and agency (organization), where agents may be humans or machines. There are several kinds of processes that must be defined to shape the authority relationships between agents in an organization. Figure 4 illustrates these processes that are distinguished within a design and an operational framework.
PAUSA-TR-E4.F – Page 18 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Figure 4. Processes characterizing authority
The first process is called authority distribution where functions are distributed among a set of agents. Authority is divided in the sense that an agent is in charge of a specific task, another agent of another task, and so on. We often talk about allocation of functions. This allocation is typically done statically, i.e., a priori for highlevel tasks and functions of the various agents, and dynamically, i.e., in real-time for context-sensitive tasks and functions. As far as automation is concerned, it cannot be thought locally, but has to be coordinated among the various agents and therefore allocated functions. The most important factor in the distribution process is the account of relations between the various functions as well as the functions themselves. Consequently, functions are co-defined. The second process is authority sharing, i.e., when two agents make separate decisions toward the same goal, and both of them have control over their part of the action and are responsible for the consequences of their decisions. This assumes that agents sharing authority act cooperatively. As a general principle, agents working cooperatively should use a common frame of reference in order to share appropriate situation awareness. For example, when two planes are getting close to each other, each traffic collision avoidance system (TCAS) provides an alert: one requests to climb and the other requests to descend. These orders are coordinated and they could have authority over the pilot if TCAS would be coupled with the autopilot. Designers and certifiers decided to allocate this authority to each TCAS. In this case, authority sharing is a matter of synchronization between both TCAS agents. There is no confusion possible when they are working correctly. As pilots have to switch to manual mode in order to follow TCAS, authority is shared between both pilots and TCAS: as time constraint does not allow to share situation
PAUSA-TR-E4.F – Page 19 / 58
Straussberger, Boy, Barjou et al., 30-06-08
representation between the two human agents, an interface (traffic display on the navigation display) was designed to propose a common frame of reference. In addition, anytime there is a change, people do not learn functions and attitudes at the same speed and may not have the same understanding and/or interpretation of these functions and attitudes according to their initial cultures. This, of course, deals with the maturity of emerging practice, and therefore appropriate cognitive functions. In the Ueberlingen accident for example, there was a confusion coming from the Euro-American culture that is different from the Russian culture concerning the TCAS. In Europe like in the USA, the TCAS provides orders that pilots are required to follow (i.e., climb or descent!), even if they are aware that TCAS has shortcomings, such as how intruding traffic is detected. Russian pilots relied on air traffic control (ATC) because under time pressure and stress they might give more trust to ATCOs than to a machine. ATM is more redundant and therefore more reliable for them. This kind of confusion is partly due to the fact that these two systems, ATC and TCAS, are not interconnected (Weyer, 2006). There is currently no feedback from the TCAS to the ground, i.e., there is no common frame of reference and information sharing, which an obvious lack of redundancy. In the Ueberlingen accident, the crew of the DHL cargo aircraft followed the American standard operating procedures, i.e., obey the TCAS, and the crew of the Russian Tupolew followed what the ATC said. Both information items were contradictory and time pressure did not allow any recovery mode. In a multi-agent system such as ATM, redundancy is crucial as a socio-cognitive support to the various agents. In addition to a design-induced lack of redundancy coming from the TCAS, there was also only one controller on duty in Zürich on July 1, 2002 at 23:35:32. He took care of both airplanes for a few minutes, and realized the conflict only 43 seconds before the collision. He reacted after both TCAS reactions. He did not have the authority over these artificial agents, i.e., TCASs. Both TCASs reacted perfectly, the Russian one asking to climb, the US one asking to descend. The problem is that the controller asked the Russian aircraft to descend, i.e., the opposite order to the TCAS. An accident does not happen with only one cause; in this case, the Short Term Conflict Alert (STCA) system was partly operational, but the controller did not know this. The STCA is the counterpart of the TCAS for the controller. In addition, the telephone line was out of order for a few minutes. Finally, the controller had to take care of another airplane, leaving the first two airplanes autonomous for a while. This is the reason why he realized too late the occurrence of the conflict. However, the main issue here is that both safety nets act independently on time spans that may overlap and with outputs that may lead to contradictory decisions. Thus, his kind of failure relies on complexity management, tight coupling of independently designed systems (TCAS onboard and STCA on ground), different cultures, lack of awareness of non-operational systems and interruption management. The third process is delegation where an agent gives the authority to another agent to perform a given task for example. This assumes that this other agent has
PAUSA-TR-E4.F – Page 20 / 58
Straussberger, Boy, Barjou et al., 30-06-08
appropriate competence, expertise and knowledge for executing the delegated task. It is also important that the delegated task is clearly defined, and that both agents have a shared representation of task inputs and outputs and, in some cases, of the process involved by the execution of the task. In addition, the initial agent that delegates the task may or may not keep responsibility on the delegated task. Delegation involves the notion of contract that must be managed by the delegating agent and assume competence of the delegate. Cognitive functions emerge from such delegation whether to a machine, i.e., automation, or to a human. They need to be identified. The fourth process refers to the negotiation between agents to establish and conduct contracts. This process involves work on information flows between agents. In the air space system, information flows relate to trajectories of aircraft. Trading involves planning and therefore reduction of uncertainty. It also involves real-time problem solving and therefore flexibility. Authority trading may be handled globally at the level of the entire organization to reduce uncertainty by solid planning, or locally within a subset of the organization to insure flexibility for rapid adjustments. Global trading and local trading may be conflicting processes. Both are typically performed according to principles and criteria such as safety and performance. Sharing and distribution processes belong to the static function allocation class. These processes are implemented at design time. Delegation and negotiation processes belong to the dynamic function allocation class. These processes are performed during operations. They assume a multi-agent framework where functions are allocated to agents. Authority assessment Authority processes should be linked to human factors and organizational issues for assessment purposes. Indeed, such issues should be measured in order to better identify the relevance of authority processes and more generally function allocation. The used approaches do however need to go beyond traditional HF assessment methods. The difficulty in this investigation is that there are cognitive functions that are not known in advance, i.e., they emerge from experience. Analytical approaches are therefore not sufficient and require the implementation of HITL simulations to enable the emergence of such cognitive functions. Obviously, it is never guarantied that we will get a complete picture of these emerging cognitive functions, which actually may “wait” for a long time to emerge, sometimes from real-world practice. Nevertheless, a good mix of human factors and cognitive engineering knowledge and know-how are very likely to foster the set-up of scenarios and simulations toward successful cognitive function emergence. This is where human factors expertise is key, and human-centered design becomes a reality. Boy and Grote (2008) introduced several concepts that support the investigation of authority sharing in socio-technical situations such as the ATM. Stability and social order, according to actor network theory, are continually negotiated as a social process of aligning interests (Latour, 1987). Agents, or actors in Latour’s terminology,
PAUSA-TR-E4.F – Page 21 / 58
Straussberger, Boy, Barjou et al., 30-06-08
have a variety of specific interests that they need to reformulate, reinterpret, represent and appropriate together in order to stabilize their interrelations. This is based on an incrementally updated common frame of reference, common rules of coordination and so on. What Latour calls socio-technical stability should have a specific meaning in the ATM context. Since Latour’s findings, automation and information technology developed exponentially and technical systems cannot be considered without taking into account their own cognitive functions (Boy, 1998) and the emerging distributed cognition among humans and machines (Hutchins, 1995). For that matter, we will prefer the terminology of socio-cognitive stability assuming that there are human and machine agents interacting. Socio-cognitive denotes the integration of cognitive and social attributes of human and/or machine agents and lead to the definition of socio-cognitive research and engineering (Sharples et al., 2002). The growing number of interdependencies among ATM agents stresses the need for a measure of socio-cognitive stability. We propose a first contribution to such a measure. We make a distinction between local and global socio-cognitive stability (SCS). Local SCS is related to agent’s workload, situation awareness, ability to make appropriate decisions and, finally, correct action execution. It can be supported by appropriate redundancies and various kinds of cognitive support such as trends, relevant situational information and possible actions. Nevertheless, its interpretation is to be undertaken beyond local descriptions, through comparing individual patterns between agents. Global SCS is concerned with the appropriateness of functions allocated to agents, pace of information flows and related coordination. It is very similar to the level of synchronization of rhythms in a symphony. Globally, sociocognitive support could be found in a safety net that would take into account the evolution of interacting agents and propose a constraining safety envelope in real time. There are unexpected or unplanned events that oblige the revision of contracts in order to insure an acceptable level of safety. Socio-cognitive stability is then related to the resilience of a socio-cognitive system like ATM. Passive SCS refers to an ATM system that returns by itself to a stable state after a disturbance. Active SCS refers to an ATM system that requires external intervention to make it return to a stable state. Obviously, there is a continuum between passive and active socio-cognitive stability where several levels of difficulty can be defined to stabilize an ATM system, as well as several levels of resilience of that system. We propose that this difficulty can be assessed using three kinds of metrics that will need to be further validated: time pressure criticality; perceived versus internal complexity; and flexibility. Time pressure criticality is the amount of workload that an agent (or a group of agents) require(s) to stabilize an ATM system after a disturbance. In ATM, complexity is frequently expressed as the number of relevant aircraft to be managed per appropriate volumetric zone with respect to the type of flow pattern (e.g., Cummings & Tsonis, 2006; Hilburn, 2004; Laudeman et al., 1998; Mogford et al., 1995). From an air/ground integrated perspective, this operational definition still PAUSA-TR-E4.F – Page 22 / 58
Straussberger, Boy, Barjou et al., 30-06-08
needs to be extended. In the ATM framework, the need for operational flexibility should guide human-centered automation and organizational setting. Therefore, flexibility is an attribute of the control side of ATM-agent’s authority. It is clear that methods and tools that will decrease uncertainty through precise planning and execution accordingly will also decrease tactical flexibility. The ATM of the future is aiming for more certain flight plans, potential loss of flexibility is an important concept to assess. Flexibility as it is understood here concerns the ease of modification of an air-ground contract in real-time.
B.3 Outputs of PAUSA The future ATM is developed through undertaking operational improvements along three time periods. The period from 2008 to 2013 addresses immediate operational improvements, the 2013 - 2020 period requires developments and the timeframe from 2020 on is addressing research and development initiatives. In line with the validation process suggested in SESAR, this process is executed until a sufficient level of accomplishment with reference to the addressed key performance areas (safety, efficiency, etc.) is achieved. As introduced, one of the major outputs of PAUSA Phase 1 is a suggested methodology to put in place a human-centred design and evaluation process for assessing authority issues throughout this process. For this iterative design and evaluation process, PAUSA proposes a collection of tools for various stages. The relationship between these tools is demonstrated in Figure 5.
Figure 5. The PAUSA-Methodology
The core inputs required by PAUSA are the expertise of a multidisciplinary team. PAUSA disposes of such a team that combines single competences, but that also has learned to work together. Such single competences are between human factors experts, operators, research and design engineers, etc. It is however the constant interaction between these competences that is resulting in maximum outputs to cover the field of authority sharing. How these competences are shared or integrated depends on the phase the PAUSA methodology is used at. In addition, a PAUSA library introduced in Part C supports this process through established guidelines. PAUSA-TR-E4.F – Page 23 / 58
Straussberger, Boy, Barjou et al., 30-06-08
The first step of this process is developing scenarios. A systematic approach is required to integrate various levels of experience and demands to achieve consensus-based scenarios (cf. C.1). The evaluation of the impact of different forms of authority sharing on the multi-agent system is executed with reference to a set of criteria and principles. Such indicators need to comprise needs of air and ground actors on a local as well as on a global level and are part of a suggested HF taxonomy. A socio-cognitive model is required to link the different structural (agents, technologies, etc.) and functional aspects (such as procedures) through adding a chronological component for different levels of granularity (cf. C.2). For a criteria-based evaluation of scenarios, two main processes can be used. An analytical analysis allows a first evaluation of the impact of scenarios from the point of view of criteria and principles. Air/ground integrated HITL simulations help to observe behaviours in the use of new technologies. These criteria and principles address local levels through individual and segment-based consideration as well as global organizational principles (cf. C.4). Results from this combination of studies shall be expressed in terms of recommendations of possible solutions for discovered problems and result in an upgrade to continue the evaluation loop.
Figure 6. The PAUSA Methodology for SESAR Implementation packages
This methodology is applicable to all implementation packages (IP) emerging from SESAR. However, as shown in Figure 6, it has different implications for these processes. Concepts close to implementation defined by the IP 1 activities require a
PAUSA-TR-E4.F – Page 24 / 58
Straussberger, Boy, Barjou et al., 30-06-08
test of existing functions regarding the defined HF issues to show emerging functions not made explicit. More advanced concepts as defined in IP 2 require a focus on the elicitation of emerging cognitive functions. Concepts submitted to further research in the context of IP 3 require a refinement of new emerging functions. Throughout its implementation, expertise, scenarios, models and evaluation methods accompany this function allocation process. However, because of a diverse advancement of the different IPs, the focus on the issues of interest in relation to emerging functions is adapted. Finally, a constant air/ground expert team supports optimizing outputs.
B.4 PAUSA vision The PAUSA toolkit provides a good starting point for a human-centred design of the future ATM system, focusing on the optimal way of allocating functions between human and machine and air and ground in order to reach SESAR performance objectives. As recalled in the introduction, this human-centred design occurs through an iterative approach, which requires either paperwork (HF Case, modelling) or human-in the loop simulation, according to the level of maturity of the design. PAUSA 1 was expressly restricted to paperwork, centred on scenario-based design. Moreover, PAUSA 1 identified, through these scenarios, some “risk areas” concerning dynamic task allocation between air and ground that would need to be carefully studied: these areas are related to the use of several operational improvements (OI) in one single airspace which may dynamically combine and raise problems for the human who will be responsible for safety-related tasks (e.g., insuring separations). Therefore, the authority sharing issue seems relevant to be addressed as soon as possible in future ATM design, starting at IP 1. Seeing the anticipated long-term perspectives in relation to the PAUSA 1 outputs, the follow-up of PAUSA could take 3 different forms: •
Refinement of the PAUSA toolkit through a specific human-in-the loop simulation which could be a reference case for present authority sharing (2009 period): this work could allow new partners from other Air Navigation Service Providers (ANSP) and Research centres in Europe to help consolidating the PAUSA approach and could be based on part-task simulations, integrating air/ground activities (e.g., ASAS-ATSAW on board, and MTCD, electronically-based environment on ground). The objective would be to prepare this simulation through the appropriate level of socio-cognitive modelling in order to address the relevant human performance issues. Another objective could be to extend the theoretical basis for addressing authority sharing with new partners. This work could be performed in the WP16 framework of SESAR.
•
Integration of the PAUSA approach into specific projects related to SESAR Operational Improvements (OI): the objective here would be to participate in the iterative design process of either En-Route Threads (e.g., based on the ERASMUS approach, concerning new modes of separation), safety net issues, or TMA and Airport Threads (ASAS Sequencing and Merging). The PAUSA toolkit
PAUSA-TR-E4.F – Page 25 / 58
Straussberger, Boy, Barjou et al., 30-06-08
could be used to consolidate the operational requirements and the function allocation process (e.g., which procedure for the allocation function) and HMI design (which feedback is required for the pilots and controllers, which information to ensure sharing Situational Awareness between air and ground,…) and to participate in the HITL validations (HF metrics definition and evaluation methods). This work could be performed in the WP related to each OI. The PAUSA approach should be applied to early stages of Implementation Packages (IP), as authority sharing issues are already raised in present ATM system (e.g., through TCAS and STCA). •
Human-oriented specific analysis of areas of the SESAR concept of operations, as identified in PAUSA 1 scenarios, as for example transition phases between free-flight airspace and controlled airspace, or transition between 4D contracts and Sequencing and Merging in dense TMA: this would allow to pre-define operational needs for pilots and controllers while reducing the number of iterations. The PAUSA approach could be fully applied in this case, from task allocation design and HMI design to HITL simulations. This work could be performed within the WP 16 framework.
Through an interdisciplinary multi-domain approach PAUSA made sure that frequently contradicting interests, perspectives, and experiences of different stakeholders are integrated, discussed, and harmonized from an early stage during the design and development process. Extending the experience obtained on a national level to a European community requires harmonizing applied approaches with alternative forms of understanding problems and new perspectives on possible solutions. To facilitate such a process and obtain commitment, sufficient opportunities to interact on a subject should be provided to multiple actors to support the development of a mixed working team. Also in the future, the PAUSA team is open to support such types of activities and will put in place specific means to enable systematic exchanges and support the advancement on PAUSA issues within the ATM community, integrating representatives on HF, operational, and design perspectives from air and ground.
B.5 Costs and Benefits of working with PAUSA Working with PAUSA requires putting in place an interdisciplinary team with expertise in human factors, operations, and automation engineering from an early stage on. Even though such a team structure may slow down the work progress at the beginning of a PAUSA task, as discussion processes require eliminating misunderstandings, on a long-term perspective consensus-based decisions can be obtained more efficiently. This requires however a systematic accompaniment and guidance of this development process. In the first phase of the project, PAUSA was undertaken on a national level. This composition of a team needs to be extended for future work in the SESAR context.
PAUSA-TR-E4.F – Page 26 / 58
Straussberger, Boy, Barjou et al., 30-06-08
The SESAR environment can however benefit from the collaboration of agents that have already advanced in learning to understand new worlds. PAUSA put in place a toolkit, which has been developed putting together an interdisciplinary team in a first phase. Through a systematic combination of analytical and HITL simulation methods, interdisciplinary expertise in research and operations, risks are reduced to overlook important problems. Through running analytical studies on emerging human behaviour in a future scenario, problems can be uncovered and solved before running simulations. A first cycle of analytical approaches supports choosing the right scenarios for simulations and supports cost reduction through focusing on the pertinent conditions. Certainly, as operational concepts require time to mature, also the PAUSA toolkit is getting more and more advanced. Its continuous application in an iterative setting will contribute to such a maturity increase.
B.6 Open questions around PAUSA PAUSA is an initiative dedicated to understand authority issues in the current and future ATM. Even if this is not its primary focus, other domains may profit from its experience. However, the proposed toolkit does not cover the completeness of requirements formulated in the E-OCVM concept validation methodology and understands itself as complementary. Efforts need to be undertaken to mature the process on one hand and to familiarize the ATM stakeholder community with its content, on the other hand.
PAUSA-TR-E4.F – Page 27 / 58
Straussberger, Boy, Barjou et al., 30-06-08
C The PAUSA toolkit for design and evaluation C.1 The development of human-centred scenarios C.1.1 Why scenarios Consensus- and experience-based scenarios help us to describe and test the use of a future ATM environment. We can point out problems linked to the use of possible new environments. A consensus-based approach relating authority issues between the air and ground segment in not yet existing situations requires today’s experience to anticipate the impact of new technologies. Scenarios allow the integration of diverging user needs on a more complete basis: they include the work place as well as the social situation, embody information about resource constraints, explain why users do what they do, take users’ goals and context explicitly, and imagine what-if situations. Thus, to anticipate the behaviour of the actors of the future ATM, the selection of appropriate scenarios is essential. The execution of this definition work occurs in a multi-disciplinary environment, with a certain number of representatives of mixed operational backgrounds and various experience levels. These representatives may have different, even contradictory, objectives. As an interdisciplinary exchange medium, scenarios allow approaching a problem from different perspectives and with different levels of granularity at the different stages in the process and thus bring user’s perspectives into the design process. Consequently, scenarios enable the analysis and evaluation of “soft” human-factor based performance criteria from different agent perspectives.
C.1.2 The PAUSA scenario development method For defining scenarios, a multi-step procedure proposes the process summarized in Figure 7. The challenge of such a procedure is to profit from the experience of users that are not necessarily used to interact with each other beyond procedures required by the operational context.
PAUSA-TR-E4.F – Page 28 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Figure 7. Scenario Development Method
Under consideration of previously defined constraints and limitations, the following steps are proposed and exemplified:
Step 1: Definition of basic scenario structure. The first step summarizes which form of characterization is required with regard to the overall objectives and selects the appropriate form of scenario views and elements. To start the description of authority sharing scenarios with domain experts from different professional backgrounds, a separate description of structural and functional scenarios was envisaged to approach the complex setting by focusing on single elements. The notions of declarative and procedural expressed these static and dynamic elements. Declarative scenarios describe the necessary elements involved to achieve the mission’s goals. The essential components are agents, tasks or functions, procedures, and context. Such descriptions necessarily lead to the way objects and agents interact with each other, and consequently to application use cases. Procedural scenarios describe chronologies of events and interactions among objects and agents and thus add the temporal component to this interaction. The implication in stories and episodes demonstrates how agents are instantiated.
Step 2: Elicitation of scenario concepts. This process applies a consensus-based method with multi-disciplinary user and stakeholder groups. To obtain declarative elements, the group elicitation method (GEM; Boy, 1996) is used. The advantage of this method is that it allows participants to reach a PAUSA-TR-E4.F – Page 29 / 58
Straussberger, Boy, Barjou et al., 30-06-08
consensus by considering individual statements and emerging discussion points at the same time. This method is composed of several phases where participants are asked to address an open question. In a first round, viewpoints are collected individually in a computer-based setting and alternately completed by all other participant’s agreements and disagreements. The subsequent open discussion leads to an integration of single viewpoints into elaborated concepts. Finally, these concepts are exposed to a rating and re-evaluated in a final critical discussion. This technique of defining concepts helps to complete the list of necessary scenario elements based on the experience of involved users. It should be mentioned that the representation of the links between the elicited issues in the form of a domain ontology (e.g., Reiss, Barnard, Moal, et al., 2006) supported the characterization process. Defining basic terms and relations using the vocabulary of a specific area, this type of ontology describes physical and functional resources and allows the expression of semantic complexity.
Step 3: Categorization of concepts. This process summarizes relevant concepts with regard to required characteristics and prioritizes them in a hierarchy. Within the current process, these elements were assigned to categories characterizing manipulated and measurable experimental variables such as air space, agents and roles, normal and abnormal conditions, technologies, HF issues, etc.
Step 4: Integration of declarative scenario concepts with procedural component. This process links the elements by adding the temporal components and sets them in action. In this setting, the elements were incrementally merged towards synthesized generic scenarios integrating declarative and procedural components. For this purpose, the procedural part was developed in guided focus group meetings by adding the time component to the different elements developed in the specific context. To ease the process of interdisciplinary work, textual tabular and graphical process representations were used applying a domain-oriented language to describing the content.
Step 5: Iteration on scenarios. A first iteration reviews the scenarios with the objective to complete or correct the representations with regard to the original objectives. Step 6: External Validation and Re-iteration of scenarios. This step conducts an evaluation of scenarios by assessing their relevance by domain experts external to the project. External experts independently evaluate the relevance of the scenarios with regard to a French and European application. For this purpose, the choice of a representative sample and the construction of appropriate instruments for quantifiable results are required. For such an instrument, criteria with regard to the representations of authority problems as well as the domain’s characteristics are proposed. A kind of criteria checklist is obtained for applying scenario validation instruments that PAUSA-TR-E4.F – Page 30 / 58
Straussberger, Boy, Barjou et al., 30-06-08
guarantee the integration of core authority sharing/distribution issues in each scenario throughout the evaluation process.
C.1.3 Examples of possible scenarios PAUSA recognized the need to dispose of scenarios for different time periods depending on the advancement of ATM automation. The need to dispose of reference settings and testing one condition against other conditions of potential solutions is used to evaluate the impact of changes. The 2009 scenario represents the state of the art to define relevant HF Issues and enable the methodological validation of conditions to be compared. The 2015 and 2020+ scenarios are linked with the introduction of specific technological environments and traffic density. Additional assumptions are required to address solved and unsolved settings (e.g., airspace structure and ground organization). At the same time, each scenario has to be run in nominal situations, and with unexpected events. However, considerating that scenarios in the far future are more difficult to imagine because of many undefined factors and open questions, different levels of granularity are used. To obtain better efficiency, adequate level of analysis for fine-grained view of operator behaviour and appropriate occurrence of disruptive events express the necessity to zoom on specific parts. At the same time, different levels of granularity also answer the difficulty to imagine the far future. In its first phase, PAUSA used the specific context of the French environment of a generic flight from Toulouse to Paris CDG (Figure 8) to provide a common basis for work on scenarios. This context allows the description of specific human and machine agents and their interaction in enroute/cruise and approach sectors. We have characterized one specific setting where we have played all the different types of scenarios. This was necessary to help participating operators better imagine the situation. The following situations were identified as being of interest: •
ASAS S&M (2015)
•
ASAS-Self-Separation (2020+)
•
Mixed Space and 4D (2020+)
PAUSA-TR-E4.F – Page 31 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Figure 8. Scenario context
To address scenarios for the 2015 time frame, PAUSA illustrated the ASAS Sequencing and Merging Procedure. For the 2020 time frame, PAUSA analyzed the following points using the •
Transfer of Free Flight to controlled sector
•
The impact of 4-dimensional (4D) contract
•
Alternative forms of delegation in mixed space
These settings are presented in detail in the PAUSA Technical Report of Task 7 on emerging human-machine systems (Debernard, Salis, Serres, Chamayou, Le, Straussberger, & Guiost, 2008) and analyzed for the following aspects: •
Objectives of defined concepts regarding human factors (HF) and expected performances;
•
Function allocation and information to be shared/transmitted between various agents and vectors of communication under consideration for these transfers;
PAUSA-TR-E4.F – Page 32 / 58
Straussberger, Boy, Barjou et al., 30-06-08
•
Emergent tasks, mode of co-operation between human and machine and between segments (ground and aircrew), the definition of the agent holding the authority, the possibilities of negotiation between agents, the procedures, before, during and after the referred function;
•
Definition of human-machine interface (HMI) supports for ground and air segments and necessary cognitive supports;
•
Checking technical feasibility of each technical agent and propose required specifications;
•
Definition of the possible technical and human “failures”, possible means of detecting them, and possible answers (tools, modification of the authority, etc);
•
Definition of the events interacting with the concept/task/activity (e.g., TCAS, weather), the means of detecting them and the possible answers (e.g., tools, modification of the authority) and critical points of the concept to be evaluated.
C.2 The PAUSA support tool C.2.1 Why support PAUSA? Depending on the goals for using scenarios, a continuous refinement of scenarios is necessary. A certain form of representation combining both approaches for air and ground is required that allows: •
The integration of operational experts’ experience;
•
The communication between different users at various stages of the design process;
•
The demonstration of the impact of selected design solutions of scenarios;
•
The representation of scenarios iteratively defined;
•
The support of the analytical scenario analysis before and after HITL simulations.
Accessing the required level of granularity is enabled with the PAUSA support tool for socio-cognitive modelling. This tool is available in the form of an interface-based prototype with a low level of dynamic interaction in the first prototype version. Sociocognitive modelling describes systems in terms of cognitions and social behaviours and thus links organizations, technologies, individuals, and context in terms of structures and functions. The support tool integrates both the organizational and functional perspectives instantiated at the described scenario sequence. For representing the individual agent perspective the interaction block representation was used to characterize the
PAUSA-TR-E4.F – Page 33 / 58
Straussberger, Boy, Barjou et al., 30-06-08
cognitive functions, the related resources, as well as the interfaces involved at an individual level. The organizational perspective illustrates the implicated structures using the AGIR model. Finally, HF issues are used to evaluate specific conditions. The PAUSA support tool uses a common database formed throughout the development of scenarios for the definition of both the organizational and functional model and contains declarative and procedural components. To build a sociocognitive model, the integration of operator experience is essential. The support tool can help to execute a pre-evaluation or guide certain hypotheses in order to set measurements during subsequent simulations optimally. Figure 9 provides an overview of its elements illustrated with some examples.
Figure 9. Concepts integrated in PAUSA-Support Tool
C.2.2 How the support tool works: a descriptive socio-cognitive model representation The organizational approach to characterize declarative and structural scenario perspectives uses the Agent-Group-Role of Ferber (1997) and Ferber et al. (2003) framework to address the relationships between the different agents. As reactive, proactive, social and autonomous entities agents play roles and interact within groups. Groups constitute an interaction context, which gathers a set of agents sharing the same characteristics. Roles are an abstract representation of a service or
PAUSA-TR-E4.F – Page 34 / 58
Straussberger, Boy, Barjou et al., 30-06-08
a function of an agent. For instance, a human agent can play the role of “executive controller” within the air traffic controller (ATCO) group. In order to link the abstract notion of role to the concrete tasks of the ATM, these tasks are described in terms of goal, sub-goal, service, and operation on different vertical hierarchical levels. For instance, one goal of ATM is “safety” and a sub-goal of this one is “collision avoidance”. Furthermore, the transfer of an aircraft between sectors is a service and monitoring the execution of this service is an operation. The interaction block representation of Boy (1998) describes the local and cognitive level from an agent perspective. The concept of cognitive functions was introduced to describe how functions transform prescribed tasks in observable activities with underlying resources. The resource concept was extended to illustrate the impact of HF issues at the concept design stage in collaboration with interdisciplinary user groups. Thus, the PAUSA interaction blocks as shown in Figure 10 describe resources (internal and external), a situation pattern including agents with their general context and activity type (action/dialogue). Goals and detailed internal and external resources are added as a textual box. We represent an abnormal situation as an evolution in terms of contextual allocated resources. The context is detailed in each phase of the scenario, for this reason it was not included in the standard characterization of interaction blocks. The representation of changes in context is considered through an element changing in the characterization of resources, e.g., the information entering from a system or from ATC. Activities may be internal cognitive functions, such as cognitive processes, or external observable processes, such as pushing buttons. Similarly, examples for internal resources may be cognitive structures acquired through experience (e.g., mental models of conflict situations), or external resources such as emergency procedures or alerts. Groups of interaction blocks are also presented in Figure 10. In normal situations, interaction blocks are organized and processed in a tree sequence. The resulting process is linear (strategy). Abnormal situations interrupt this linear sequence to branch into other blocks.
PAUSA-TR-E4.F – Page 35 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Figure 10. Single (upper graph) und Group (lower graph) Interaction block illustration
To characterize air and ground actors’ functions, basic generic activity models were built for the tasks of ATCOs and pilots with data obtained from task and activity analyses executed in this domain (e.g., Kallus et al., 1997; Dittmann et al., 2000). Comparable forms of representation were selected for both air and ground segments. These activities were described on three hierarchically oriented levels that decompose tasks, activities, and cognitive functions. This decomposition helps to access the appropriate level of information at a certain stage during the design and development cycle. Within these settings, authority and responsibility are characterized as properties on both levels, the roles assigned with authority as well as the functions evoking authority. Therefore, to assess the impact of function allocation solutions on authority, a complete description of a socio-cognitive model is required. To facilitate the communication between different potential users of scenarios, a preliminary version of a prototype for a support tool is available that integrates the organizational and functional perspective and illustrates the assessment of organizational and individual human factors metrics. Figure 11 illustrates the content of this tool that contains the scenario context (left upper section), procedural scenarios (left lower section), cognitive functions of the implied agents (right lower section) and detailed descriptions of resources, content and scenario functions (right upper section).
PAUSA-TR-E4.F – Page 36 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Figure 11. PAUSA-Tool screenshot with an example of the interaction block representation for air and ground activity in an ASAS S&M scenario sequence.
C.3 Designing cooperative human-machine systems Designing cooperative human-machine systems is a complex process. It starts with the accomplishment of a functional analysis, which is consequently continued with a function allocation process. Within the context of PAUSA, this function allocation occurs under specific consideration of authority and cooperation issues.
C.3.1 Functional analysis The scope of the functional analysis is to elicit the possible combination of function allocation of the main ATM functionalities, between ground - board and human machine. These allocations are considered as a part of the scenario writing process and ATM modelling. The main object of these allocation proposals is to be further analyzed to suggest the final allocation, taking into account human factors. The functional analysis requires the knowledge of the technology supports envisaged for the future. The first step of the analysis is to identify which systems could be assessed as agents, and their potential degree of autonomy. According to the Technical Report of PAUSA Task 1 on the State of the Art (Figarol, Straussberger, Boy, & Debernard, 2006) an agent is defined by its ability to perceive its environment through sensors, and subsequently act upon this environment through effectors. It is assumed, as part of this analysis, that giving information to other agents, which in turn may physically act upon the environment, is also acting on the environment. The autonomy of the agents is assessed using the following six classes of functions: Data
PAUSA-TR-E4.F – Page 37 / 58
Straussberger, Boy, Barjou et al., 30-06-08
acquisition, Analysis-diagnosis, Implementation, and Monitoring.
Plan
elaboration,
Solution
elaboration,
In the framework of the functional analysis, definition for functionality and function are fixed. Functionality is approached from a dual complementary point of view: •
A "set of attributes that bears on the existence of a set of functions and their specified properties. The functions are those that satisfy stated or implied needs" (ISO 9126).
•
A high level function, that is the one which informs on the "what it does", while functions (or subfunctions), which constitute the functionality, give the "how or when it does it".
The functions underpinning the functionality are assigned in the same six classes, already used to assess technology agents’ autonomy. Hence, the methodology of the functional analysis consists of filling a grid with items demonstrated in Table 1. For legibility and understanding, results are summarized in the form of a descriptive text, associated to main findings regarding agents, emerging cognitive functions, HF, emerging technology, and expected impact on performances of the ATM.
PAUSA-TR-E4.F – Page 38 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Table 1. Functional analysis grid and definition of elements Functionality name Class
Function 1st rank
Function 2nd rank
Function 3rd rank
Class Function 1st rank Function 2nd rank Function 3rd rank Segment Current allocation HF Current Allocation Possible allocation Expected performance
Desirable functions HF Possible Allocation
Segment
Current allocation
HF Curr. Alloc.
Possible allocation
Expected perform.
Desirable functions
HF Poss. Alloc.
Class of function as defined in previous paragraph Functions of 1st rank; that composes the functionality Functions of 2nd rank; that composes the functionality Functions of 3rd rank; that composes the functionality Ground or board Current agent in charge of the function HF relevant for the current allocation Allocation envisaged for the future Expected advantages from the proposed possible allocation: Key Performances Areas or attributes as described in the ISO functionality definition could be used Associated new functions desirable as derived by the new allocation; emerging cognitive functions HF relevant for the envisaged allocation
As part of PAUSA Phase 1, eight functionalities of ATM were identified: •
Flow management
•
Planning
•
Cluster
•
Sequencing
•
Merging
•
Spacing
•
Separation
•
Collision avoidance
Of course, in order to keep flexibility regarding the issue under review, the statement or definition of these functionalities may be tailored. In complement, some guidance is supplied based on the following issues to support the analysis: •
Identified ATM tenets (Safety, Regulation, Procedures, Cooperation (perfectible), Responsibility, Proficiency, Divide and Composure) to better approach human behaviour;
PAUSA-TR-E4.F – Page 39 / 58
Straussberger, Boy, Barjou et al., 30-06-08
•
A HF Convergence method, derived from the Eurocontrol Human Factors Case, offers a standardized HF approach;
•
SESAR KPAs provide a metric to understand the gains of the proposed allocation.
C.3.2 Function allocation and task delegation in a collaborative system Through dynamic and static allocation of functions, the levels of autonomy and authority of agents are defined as load (and responsibility) and transferred to one or several technical or human agents. Several criteria are identified to be relevant: •
A transfer of a task is linked to costs (cognitive cost, delay, transfer of information, coordination, explanation), therefore, it should be avoided that there is no use of such a transferred task;
•
A « Passive monitoring » situation, which might relate to a loss of situation awareness and competencies, shall be avoided (Metzger & Parasuman, 2001);
•
Sharing the representation of a situation and a goal needs to avoid interference and decisional conflicts;
•
The form of information transfer (low level vs. synthesised) is linked with questions on resulting costs, amount and availability;
•
Providing feedback avoids loss of control.
C.3.2.1 The need for delegation In the opinion of the PAUSA team, the interest of the task delegation relates to two aspects: the controllers’ workload decrease, and future ATM performance. By 2020, it is expected that the traffic load will increase in a very important way during certain periods of the day, which implies a modification of the controllers’ activity affecting the maintenance of an acceptable workload to face issues that are difficult to evaluate. This workload is due to various elements such as: •
Quantity of information to be integrated: the traffic increase will increase this quantity, even with part of the traffic in 4D contract or self-separation where the ATCO plays a secondary part with a less direct involvement;
•
Number of tasks to be managed in parallel: the traffic increase will impact the tasks to be managed, even if the implication of the ATCO is reduced;
•
Management of uncertainty: nowadays, the uncertainty management by ATCOs is an important factor of load (wait, compromising, ...). The implementation of RBT (Reference Business Trajectory) regularly resynchronized and the installation of tools for mid-term conflict detection (MTCD), will allow a decrease in this uncertainty. It will nevertheless be necessary to provide these new trajectories to ATCOs;
PAUSA-TR-E4.F – Page 40 / 58
Straussberger, Boy, Barjou et al., 30-06-08
•
Activity synchronization related to traffic events and command availability or the availability of other agents: the delegation would partly make it possible to desynchronize the activity of the controller, the flight crew ensuring the implementation of decisions partly resulting from the ground;
•
Monitoring carried out by the controllers: upstream of the decision to determine the situation, or downstream of the decision to check the engaged activities, should be reduced by the delegation on board.
Concerning the performance, one can hope that the decisions and the implementations realized by the flight crew will be more effective than those that are taken to date by the ground. But this performance in terms of distance covered, time of transit or kerosene consumption will be quite relative. On the other hand, if the separations ensured by airborne systems (ASAS type) are finer and require shorter duration in their implementation, (- a point to be checked -), resources in terms of used space will be released, forcing a reduction of other tasks to be executed. Our opinion is that the main interest of the tasks delegation, apart from the ATCO’s load transfer towards the F/C, lies primarily in the fact that the controller will remain the traffic “leader”, i.e. will always be responsible for the definition of critical situations and of corresponding resolution strategies. Consequently, the task allocation must make it possible for the controller to more easily maintain the situation awareness. Thus, we should not seek to use a technical agent because it exits, but to design this technical agent in order to meet the needs of an ATCO. Nevertheless, in order to get actual benefits from the delegation, it will be necessary: •
To minimize the costs (temporal and cognitive) of implementation of these delegations; if it takes more time to explain to another agent than it is necessary to do by oneself, these delegations will not be used;
•
To preserve the stability of these delegations. First, by avoiding that the delegation solutions become too important over time. Indeed, a non-stable solution could surprise controllers for their activity planning. Second, by avoiding ruptures of delegation implementation by the flight crew. In that case, the effort of delegation implementation would be unnecessary;
•
To build and maintain a shared presentation of the situations and objectives of each actor, therefore between the ground and the flight crew. Being given the traffic increase envisaged and the quantity of requested information to be managed, it will be necessary to take care to ensure the decisional authority;
•
To provide sufficient feedback to ensure the situation awareness of the ATCO during the delegation. This feedback has to present sufficiently synthesized information to avoid an informational overload with low-level elements. Elements of a low level correspond to raw data, such as flight level, flight position, flight destination, etc. and require a lot of effort to interpret and integrate.
PAUSA-TR-E4.F – Page 41 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Within the framework of delegation, the aeronautical system will become a multiagent system with a division of the decision and the sharing of the authority. Our approach is based on the work of Castelfranchi (1998). For this author, an agent X delegates a goal (more or less constrained) to an agent Y when X has a need for an action (or function) of Y and integrates it in its plan. This requires a trust of X towards Y because: •
X believes that Y can do it and will do it;
•
X has a goal for Y;
•
X relies on Y to do it and coordinates its actions related to those of Y.
The motivation of the delegation is due primarily to dependence: X wants to achieve a goal but cannot do it (competence, workload, etc.) and delegates a goal to Y (under constraints). The author defines two levels of delegation: •
The “Executive delegation”, in which the plan is completely specified and just requires to act.
The “Open delegation”, where the plan or the decision is specified and delegated to the another agent who has a certain autonomy for performing the rest of the task. The first type of delegation is more costly for the agent who is in charge of the delegation than the second, but in the end more predictive.
C.3.2.2 The need for cooperation In consequence, if the operation is relatively effective to achieve the defined goals, the various agents will have to cooperate when trying to achieve their own goals by facilitating the goals of others (Hoc, 2001). Basic agents have the advantage of a better predictability of their behaviour, whereas « Intelligent agents » require more interaction. The problem is however not the definition of the levels of autonomy and authority of the various agents, but that of the coordination degree to be set up between the agents (Christoffersen & Woods, 2002). Therefore, two aspects must be respected: the system must be observable and controllable. Concerning observability, it is necessary to set up a shared representation of the situation/problem to be managed. For a pilot, the aircraft, which he/she has to manage, is in the centre of his/her problems, whereas the controllers have to manage several situations. This shared representation of the problem state concerns: •
The representation of the activities of the other agent and the access to the information used by the other;
PAUSA-TR-E4.F – Page 42 / 58
Straussberger, Boy, Barjou et al., 30-06-08
•
Goals, strategies and solutions followed with in particular the understanding of the reasons why one strategy is used rather than another;
•
Intentions of the other and if possible their projections in the future.
Because of the increase in the complexity and autonomy of the agents, we consider that: •
A simple representation of low-level information is not enough any more because it would require expensive re-building of the goals of the other for example. The synthesis of the intermediate “results” of the activity of the other one (goal, situation, strategy, partial solution) would thus be an advantage.
•
An increase in the feedback seems necessary. Indeed, an insufficient level of feedback generates problems of understanding, increases the communication, generates at the same time tactical and strategic decisional conflicts. An integrated representation that “amplifies” the mental image of the current situation and the activities of the various agents, and their possible evolutions in the future appears interesting. Various representations are then possible: event-based (highlight exchanges), future-oriented, or pattern-based (displays abnormally or unexpected condition).
To cooperate/communicate/negotiate, however, the controllers and the pilots need to have common objects. Through the operational concepts defined in SESAR around the concept of trajectories (e.g., Reference Business Trajectory, RBT) and the notion of sharing data (System-Wide Information Management, SWIM), it is possible to be optimistic. On the aspect of the controllability, two concepts will have to be tackled: •
The concept of authority and control of the tasks/sub-tasks/functions allocated to an agent (technical or human). The level of autonomy in the decision, the degrees of freedom to achieve a goal or an instruction, and the expression of the constraints regarding a goal have to be defined. It will also be necessary to take care to avoid the interference between goals.
•
The notion of authority and control for the assignment of tasks/subtasks/functions. A certain number of answers to the following questions will have to be in particular defined according to the progress of the operational concepts of SESAR, the technical possibilities of the various agents (ground or aircrew), of the means of communication available between these agents, and finally of the availability of the human agents: - Who is the allocator of a certain function (one vs. several / ATCO vs. A/C)? - Is it possible to create a certain “negotiability” of the allocation? - Will it be possible to build a certain “predictability” of the acceptance of an allocation based on the model of the other agent’s activity? PAUSA-TR-E4.F – Page 43 / 58
Straussberger, Boy, Barjou et al., 30-06-08
- How to ensure the stability of the allocation related to the structure of control (division into airspace sector for example), or related to the use (transferring, taking back,...). - Will one have to set up a monitoring of the activity of the other agents by a specific agent? •
All these questions will be considered in the case of the delegation.
C.3.2.3 A cooperative workspace for supporting delegation in ATM From the functional analysis reported in the Technical Report of PAUSA Task 4 (Salis & Straussberger, 2007), various delegation forms can be proposed by extending the work of Castelfranchi. Therein, the delegated object is in fact the starting point of the activity of the agent to which one delegates. This object contains in an implicit way a certain objective to be reached. Activities selected to deal with problems are the following: monitoring, implementation, solution definition, strategy definition, problem definition, and information acquisition (cf. the model of problem resolution of Rasmussen, 1980). On this basis, it is possible to define the concepts of partial or total delegation depending on the fact that all of the functions necessary to the resolution of a task starting from the delegated object is carried out completely or not by an agent (Figure 12).
Figure 12. Agents’ activities in delegation
PAUSA-TR-E4.F – Page 44 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Partial delegation has the advantage of fixing hard “check-points” for the agent who delegates, but provides additional work if he/she wants to delegate the whole of the activities/functions to the other agent. For the total delegation, if one wants to maintain common situation awareness, it is necessary that the agent who delegates can reach the representations or the intermediate information generated (by the other agent) during the execution of the specific functions to a delegated object. A problem that appears important in a context of high traffic relates to the definition of constraints associated with a goal. These constraints allow the agent who delegates not to be surprised by the decisions of another agent. These constraints are in general implicit and not expressed. If the traffic increases, it is reasonable to think that the complexity of the situations will also increase. It will perhaps be necessary, at the time of the delegation of a goal to be reached, to specify additional constraints, in particular if the conflict integrates several interfering aircraft. This will induce a problem of communication between the aircrew and the ground when it is necessary to express these constraints. The delegation of tasks results in new forms of cooperation and can be supported by a collaborative workspace. Cooperation occurs when a number of agents is involved in the completion of a task. To a certain extent the involved parties need to be aware of what other agents are doing. Cooperation implies that a human and machine system work jointly to achieve an operational goal. A Common Frame Of References (COFOR: Carlier & Hoc, 1999) is a set of shared information linked to the management of the process and the management of cooperative activities. A part of the COFOR can be implemented in order to support the cooperation between technical and human agents. This process is illustrated with AMANDA (Riera & Debernard, 2003). For AMANDA, the 3 classes of cooperative activities were defined: •
Building and updating the COFOR: “Simple Supply” and “Simple Request” of information;
•
Supervision of the COFOR: “Interference Detection” on information;
•
Management of the interferences: “Interference Solution”: negotiate, accept, impose.
Table 2 summarizes the exchanged information during cooperative activities and presents a general support for analyzing information and activity types also in airground integrated settings. Table 2. Information with regard to activities during cooperation Information type Basic Information (eg. aircraft level) Situation/Problem (2 or more A/C) Strategy (eg. Turn behind) Solutions Commands
Activity type information elaboration diagnosis schematic decision-making precise decision-making implementation
PAUSA-TR-E4.F – Page 45 / 58
Straussberger, Boy, Barjou et al., 30-06-08
C.4 Criteria and Metrics C.4.1 HF Issues integrating needs of air and ground actors According to the specific demands of PAUSA, a first taxonomy was developed that contains the essential HF issues to be addressed. Its main objective is to propose a shared framework to elicit the HF issues in a common language usable by ground and air staff. The taxonomy is based on the description of EUROCONTROL (Human Factors Case) and Airbus HF design processes (integrating human factors in the design cycle), certification initiatives (CS 25-1302) and a merge of all the HF issues already sorted in PAUSA’s previous tasks. The approaches developed by EUROCONTROL and Airbus have been driven by the need to establish a HF framework understandable by the multi-disciplinary teams and also to shape the HF Development Plan that define the HF issues and the roadmap to put in place the HF studies and simulations all along the design process. This taxonomy serves as a basis for further refinement in human factors indicators and will be further developed and prioritized in the ongoing task process. For example, indicators describing the issues of maturity as well as stability on a system level should be addressed. These HF issues are thus linked to the assessment on a global and a local ATM system level. A list of relevant factors is summarized in Table 3. Table 3. HF Issues Category Human-Machine Interface and working environment
Organisation and Staffing Training and Development Procedures/Roles/Responsibilities
HF Issue Appropriateness of work environment: Appropriate context in terms of work organization and equipment Utility of information: Appropriate information to be transmitted between the agents for the purpose of goal achievement Not defined as the core activity of PAUSA, but may be useful Not defined as the core activity of PAUSA, but may be useful Workload: Appropriate level of strain evoked when dealing with the work system; Appropriate availability of resources to address prescribed tasks Trust/confidence: Appropriate level of trust evoked by system reliability to elicit appropriate use behaviour Performance: Means available to achieve task goals; Matching between activities; outputs and task goals Difficulty: Difficulty/Easiness: Appropriate effort required by a task; problem or task identification, plan and safe execution shall require resources compatible with the integration of the task in subject within the whole tasks achievement Responsibility: Appropriate legal and moral responsibility assigned to roles Procedure format/ positioning/ structure/ content/ realism; Availability of appropriate procedures; Appropriateness of procedures Situation awareness: Appropriate information available to know what is going on to; perform the appropriate activities to remain in that state
PAUSA-TR-E4.F – Page 46 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Multi-agent teams and communication
Management of the variability of operational and technical situations, in normal conditions or after a failure
Stability: Appropriate active and passive equilibrium of the global and local integrated ATM system; how long does it take to return to a steady state following a disturbance in the organization due to an unscheduled event; local stability in terms of human machine interaction is reflected in indicators such as workload or situation awareness; global stability in terms of function allocation in a multi agent system is reflected in indicators for cooperation and coordination Cooperation: Joint operation or action of agents for reaching a common goal, or different goals with interferences (on procedures, means, resources, etc.) Communication: Explicit and implicit exchange of signals or information between two or more agents Coordination: Appropriate process of informing each agent on of planned behaviours of other agents, or assignment of a plan to another agent Team structures/dynamics/relations: Appropriate roles and relations in a group of agents that have a common goal Error tolerance: Management of abnormal situations possible Management of expected events possible Management of unexpected events possible Error resistance: Can errors be avoided? Uncertainty Management: Management of unsure conditions or unsure information (trajectories, a/c performance, etc.) Collaboration: Mechanisms to work towards a common goal are provided
C.4.2 Frames and models supporting the iterative evaluation process Based on the introduced and defined notions around HF issues, a first approach was developed that can be used for an early analytical evaluation or pre-evaluation of the impact of proposed authority distribution between agents on an individual level. This method uses different framework models to support air-ground HF assessment and represents an integrated approach to describing common problems of the air and the ground segment. The idea is to get a synthezised vision between air and ground representatives of what should be a « Human Factor Analysis » in PAUSA, in relationship to SESAR. Based on the Concept of Operations, scenarios are designed to illustrate specific points that might raise HF issues. These scenarios will be implemented through HITL simulations. We need « operational » HF Models to describe what behaviour is expected and also to interpret what will be observed (which functions did not run) or which key performance criteria (KPA) has been affected. Figure 13 describes how to address collective work in a multi-agent environment, which is the main issue of cooperation, authority, and task sharing in PAUSA. The situations that will be observed are multi-agent situations.
PAUSA-TR-E4.F – Page 47 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Figure 13. The evaluation of HF issues in a multi-agent setting
In a space of tasks, within a certain environment (both technical and organizational) the agents are assumed to « produce » their activities, based on pre-determined interaction procedures. In this setting, individual HF metrics based on the PerceiveDecide-Act-Control (PDAC) model will be gathered. At the same time it will be possible to set performance metrics (e.g., capacity, safety, efficiency) and organisational metrics (e.g., number of vocal communications). Qualitative data will be extracted concerning organizational aspects. The individual HF metrics will then be temporarily and conceptually correlated with the more global measures (organizational, performance and qualitative data). In addition to HF metrics, the basic PDAC is summarized in Figure 14 and helps to analyze basic cognitive processes. To provide an example on how PDAC can be used: if a perception problem is identified as a cause for a poor decision, then we can question the data input channel. However, this model is a simplified representation of the most important cognitive issues. The analysis needs to be related to the context, which is an operational context (traffic situation), and the role as played by the agent. The essential components of the agent’s behaviour have been identified as PDAC. As this represents a simplified illustration, it is noted that well-known relevant cognitive processes such as anticipation are considered on a different level through goalversus event-driven data cognitions as well as individual “conditioners of action” in Figure 15.
PAUSA-TR-E4.F – Page 48 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Figure 14. The description of perception (P), decision (D), action (A) and control (C) processes on an agent level
Figure 15 also contains additional aspects to be observed in a HITL simulation. From the world input, we can deduce an expected behaviour related to the prescribed task. What will be observed is a result of human activities through observable « output actions » to understand the variation to the prescribed task. When there is a difference between expected behaviour and output actions, we need to assess the HF issue that might have produced such a result: the PDAC model allows to question the human basic functions and to locate the elements (intrinsic trigger and organizational context) that had an impact. At this level, this can be applied to a single human agent.
PAUSA-TR-E4.F – Page 49 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Figure 15. The human agent in context
HF issues as listed in Table 3 are defined by elements in relation to operational indicators and related to objectively measurable behaviours. An assessment matrix serves to determine the following issues: •
How to recognize and mark critical situations/information?
•
Which diversity of action will really happen?
•
Which HF issue plays a role or when is it useful to measure a certain sequence in the scenario flow to detect critical moments?
Moreover, it can provide hints for reinvestigation in a second simulation. As a result, recommendations on how to solve the issues can be concluded. The PDAC (Perceive, Decision, Action, Control) will be considered as a generic evaluation element. In addition, we assume that certain basic mental processes are part of this evaluation in form of the elements implemented in interaction blocks. For example, anticipation (of perception, action, event) is considered as an element in relation to Perceive. Finally, it is noted that certain core activities are linked to different HF issues, for example identifying perception may be relevant to workload as well as situation awareness.
PAUSA-TR-E4.F – Page 50 / 58
Straussberger, Boy, Barjou et al., 30-06-08
C.5 An iterative evaluation approach between analysis and human in the loop simulations An iterative approach was developed to support the evaluation process with the objective to: •
Enable collaboration between multiple users from different work domains and disciplines;
•
Demonstrate human factors in authority sharing between agents/organizations on an agent and an organizational level;
•
Support concept analysis, prototyping, and evaluation before and after HITL simulations.
multiple
This approach allows the comparison of different design solutions on a functional or an organizational level by integrating an analytical approach applying the support tool and operational real-time HITL simulations (Figure 16). The analytical method uses the PAUSA support tool that integrates scenarios of interest, the underlying socio-cognitive models, and HF criteria defined for preevaluation. Applying this approach before a HITL simulation, critical or incomplete situations of interest can be identified and possible impacts on HF issues demonstrated. Subsequent to real-time simulations, by determining emerging cognitive functions and organizational patterns, these solutions can lead to an upgrade in the design process.
PAUSA-TR-E4.F – Page 51 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Figure 16. Analytical and HITL simulations
Once a first evaluation cycle based on analytical methods has been passed, operational simulations follow. Focused on the observation of human behaviour and real time function delegation and authority trading, they are aimed to understand emerging functions for completing the analytical analysis steps. By measuring joint human and machine performance (safety and capacity) in integrated air and ground segments, local and global system outputs can be evaluated and fed back to a second version of the socio-cognitive model. In version 2, this model helps to argue for different design solutions. Operational or human in the loop simulations can be run on platforms addressing different levels of realism. Adequate simulation means may be selected between mainly two options: •
Part-Task oriented platforms: This type of platform is adequate for HF studies (e.g., used for the TCAS and ASAS/ATSAW studies at DSNA) requiring high operational realism and has the advantages of being easy to launch, flexible, enabling incremental development, and adapted to early design.
•
Air-Ground Integrated full-scale simulator: This simulator type has to be as close as possible to real technical functions and a real work environment concerning the number of ATC positions. (e.g., used for the PALOMA study on ASAS S&M). Its advantage is a high level of maturity for validation phases.
PAUSA-TR-E4.F – Page 52 / 58
Straussberger, Boy, Barjou et al., 30-06-08
In all the cases, the type of protocol required defines all technical AND operational questions as suggested in the E-OCVM. Specific characteristics to address are: •
Air/ground joint evaluations => pilots AND controllers;
•
Data gathering on individual and collective behaviour as well as system performance;
•
Joint pilots/controllers debriefings and self-assessment based on records.
Finally, it should be said that such an iterative approach may enter in endless loops if not applied in a careful manner. Only a constantly involved expert team and a clear definition as well as update of objectives can counteract an inefficient use of the PAUSA toolbox.
PAUSA-TR-E4.F – Page 53 / 58
Straussberger, Boy, Barjou et al., 30-06-08
D Conclusion The first phase of the PAUSA project allowed to build the foundation for an approach to the issue of authority sharing/distribution in the ATM system. Most important it provided a unique opportunity to put together different representatives of the aviation community, both on the ground and the airborne sides, from research and industry up to the operational experts. As a result, the proposed scenario-based approach addresses the viewpoints of the two end-user populations. It also combines the existing frameworks regarding HF issues as used by the ATC organizations and by the aircraft manufacturers. Finally, socio-cognitive models help to put these materials together for an iterative evaluation with analytical and HITL simulation means. The scope is to evaluate emerging human-machine systems and suggest solutions in case HF problems are identified. Within this context, operator experience represents an essential precondition to choose appropriate scenarios. Understanding the reasons and limits for behaviour that is put in place when executing tasks can provide a basis for building hypotheses on the type of activities to be expected in future systems. At the same time, such an experience has to be analyzed in terms of constraints, seeing that future user cultures may require different needs. Using the known to explore the unknown is the first step to determine the possible impact of new technologies. By iterating on systems we build over time, an advancement of the ATM system can be achieved with regard to identified performance and HF criteria. A continuation of the PAUSA project should naturally follow to accompany the implementation of the future concepts of operation within SESAR and requires putting in place the necessary activities to support theoretical and practical discussions around a network of excellence. Only by informing all stakeholder groups, especially industry and regulatory bodies, on the gains of considering PAUSA from the beginning, a safe and efficient, thus, mature ATM system can be achieved.
PAUSA-TR-E4.F – Page 54 / 58
Straussberger, Boy, Barjou et al., 30-06-08
References Project-related documentation and dissemination Technical reports2 Figarol, S., Straussberger, S., Boy, G., & Debernard, S. (2006). TASK 1: State of the Art (Tech. Rep. PAUSA-TR-S1.F). Toulouse, France: Eurisco. Le, O., Farges, F.L., & Chamayou, C. (2007). TASK 2: Authority Sharing based on Technologies (Tech. Rep. PAUSA-TR-T1.F). Toulouse, France: Eurisco. Straussberger, S., Boy, G., Feuerberg, B., Fass, D., Reuzeau, F., Figarol, S. & Le Blaye, P. (2008). TASK 3: Authority distribution from a human and organizational perspective (PAUSA-TR-E3.F). Toulouse, France: Eurisco. Salis, F. & Straussberger, S. (2007). TASK 4: Functional Analysis (Tech. Rep. PAUSA-TR-D1.F) Toulouse, France: Eurisco. Straussberger, S., Lantes, J.Y., Mueller, G., Boumaza, A., & Salis, F. (2008). TASK 5: Socio-cognitive Modeling (Tech. Rep. PAUSA-TR-E1.F-SS). Toulouse, France: Eurisco. Medard, C. (2008). TASK 6: Proposal for Experimental Protocols (Tech. Rep. PAUSA-TR-K2.F). Toulouse, France: Eurisco. Debernard, S., Salis, F. Serres, A., Chamayou, C., Le, O., Straussberger, S., & Guiost, B. (2008). TASK 7: Emerging Human-Machine Systems (PAUSA-TRV1.F). Toulouse, France: Eurisco. Boy, G. (2008). TASK 8: Prospective of PAUSA (Tech. Rep. PAUSA-TR-E5.F). Toulouse, France: Eurisco. Straussberger, S. Chamayou, C. Pellerin, P., Serres, A., Salis, F., & Feuerberg, B. (2008). Scenarios In PAUSA (Tech. Rep. PAUSA-TR-E2.F). Toulouse, France: Eurisco. Publications Boy, G. & Grote, G. (2008). Authority in increasingly complex human and machine collaborative systems: Application to the air traffic management evolution. Paper prepared for HCI Aero, September 2008, Toulouse, France.
2
All Technical Reports were edited with the help of numerous contributors.
PAUSA-TR-E4.F – Page 55 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Feuerberg, B. & Straussberger, S. (2008). Perceptions of roles in Air Traffic Management – A comparing interview study. Paper prepared for European Association of Aviation Psychology, September 2008, Valencia, Spain. Straussberger, S. & Boy, G. (2008, in press). Designing scenarios: the challenge of a multi-agent context for the investigation of authority distribution in aviation. D. de Waard, F.O. Flemisch, B. Lorenz, H. Oberheid, and K.A. Brookhuis (Eds.) (2008), Human Factors for assistance and automation (pp. 1 - 13). Maastricht, the Netherlands: Shaker Publishing. Straussberger, S., Lantes, J.-Y., Boumaza, A., Muller, G. & Salis, F. (2008). A method for analyzing authority sharing in ATM. Paper prepared for the 7th INO Workshop, December 2008, France.
External References Boy (1996). The Group Elicitation Method: An Introduction. In N. Shadbolt et al., Advances in Knowledge Acquisition, Springer Verlag, Berlin. Boy, G. (1998). Cognitive function analysis. Westport, CT: Ablex, Greenwood Publishing Group. Carlier, X., & Hoc, J. M. (1999). Role of a common frame of reference in cognitive cooperation: sharing tasks in air-traffic control. In J. M. Hoc, P. Millot, E. Hollnagel, & P. C. Cacciabue (Eds.), Proceedings of CSAPC'99 (pp. 67--72). Valenciennes, F: Presses Universitaires de Valenciennes. Castelfranchi, C. (1998). Modelling social action for AI agents. Artificial Intelligence, 103(1-2), 157-182. Christoffersen, K., & Woods D. (2002). How to make automated systems team players. Advances in Human Performance and Cognitive Engineering Research. 2, 1-12. Cummings, M.L. & Tsonis, C. G. (2006). Partitioning Complexity in Air Traffic Management Tasks. International Journal of Aviation Psychology, 16 (3), 277 -295. LATOUR, B. (1987). Science in Action: How to Follow Scientists and Engineers through Society. Milton Keynes: Open University Press. Dittmann A. , K W. Kallus , K W., Dittmann A. & D. Van Damme (2000). Integrated Task and Job Analysis of Air Traffic Controllers – Phase 3: Baseline Reference of Air Traffic Controller Tasks and Cognitive Processes in the ECAC Aera. (HUM.ET1.1.ST01.1000-Rep-05). Brussels: Eurocontrol. Ferber J. (1997). AALAADIN: a meta-model for the analysis and design of organizations in multi-agent systems. Rapport technique, LIRMM, Université Montpellier II.
PAUSA-TR-E4.F – Page 56 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Ferber J., Gutknecht, O. & Michel, F. (2003). From Agents to Organizations: an Organisational View of MultiAgent Systems. In Agent-Oriented Software Engineering (AOSE’03), volume LNCS 2935, Springer-Verlag). Hutchins, E. (1995). Cognition in the Wild. MIT Press. Kallus, K. W. Barbarino, M. & Van Damme, D. (1997). Model of the Cognitive Aspects of Air Traffic Control (HUM.ET1.ST01.1000-REP-02). Brussels: Eurocontrol. Hoc, J.M. (2001). Towards a cognitive approach to human–machine cooperation in dynamic situations. International Journal of Human-Computer Studies, 54(4), 509-540. Metzger, U., & Parasuraman, R. (2001). The role of the air traffic controller in future air traffic management: An empirical study of active control versus passive monitoring. Human Factors, 43, 519-528. Reiss, M., Moal, M., Barnard, Y., Ramu, J.-Ph., Froger, A. (2006). Using Ontologies to Conceptualize the Aeronautical Domain. In F. Reuzeau, Korker, K. & Boy, G. Proceedings of the International Conference on Human-Computer Interaction in Aeronautics (pp. 56-63). Cépaduès-Editions, Toulouse, France. Rasmussen, J. (1998). Notes on human error analysis and prediction. In G. Apostolakis, S.Garriba & G. Volta (Eds.), Proceedings of the NATO advanced study institute on synthesis and analysis methods for safety and reliability studies (pp. 357–389), New York. Riera, B. & Debernard, S. (2003). Basic Cognitive Principles Applied to the Design of Advanced Supervisory Systems for Process Control. In Erik Hollnagel (ed.) (Ed.), Handbook of Cognitive Task Design (pp. 255-281), London : Lawrence Erlbaum Associates. Weyer, J., (2006). Modes of Governance of Hybrid Systems. The Mid-Air Collision at Ueberlingen and the Impact of Smart Technology. In STI-Studies 2 (2006), (pp. 127-149); http://www.sti-studies.de/articles/ 2006-02/Weyer-011206.pdf. Mogford, R. H., Guttman, J. A., Morrow, S. L., & Kopardekar, P. (1995). The complexity Construct in Air Traffic Control: A review and synthesis of the Literature (DOT/FAA/CT-TN95/22). Washington, DC: Department of Transportation/Federal Aviation Administration, Office of Aviation Research. Laudeman, I. V., Shelden, S. G., Branstrom, R. & Brasil, C.L. (1998). Dynamic Density. An Air Traffic Management Metric. California: National Aeronautics and Space Administration, Ames Research Center, NASA/TM-1998-112226. Hilburn, B. (2005). Modeling Cognitive Complexity in Air Traffic Control: Eurocontrol’s Coca Approach. Digital Avionics Systems Conference (DASC), The 24th Volume, 30 Oct.-3 Nov. 2005 Page(s): 5.C.5-1- 5.C.5-1. Digital Object Identifier 10.1109/DASC.2005.1563384.
PAUSA-TR-E4.F – Page 57 / 58
Straussberger, Boy, Barjou et al., 30-06-08
Sharples, M., Jeffery, N., du Boulay, J.B.H., Teather, D., Teather, B., & du Boulay, G.H. (2002) Socio-cognitive engineering: a methodology for the design of human-centred technology. European Journal of Operational Research, 136, 2, 310-323. Org.:
Reviewed by:
Date:
SN:
Description of amendment:
EUR
SS
20-04-08
1.0
Initial draft
EUR
SS
29/04/08
1.1
Contributions from SF, SB, FS, GB
EUR
SS
19/05/08
1.2
Accept all prior modifications as no comments returned
EUR
SS
24/05/08
1.3
Feedback PB, JL, SF, SD, PS, finalized to 1.F, corrected by HW, confirmed by GB;
PAUSA-TR-E4.F – Page 58 / 58
