Simulation-Based Acquisition
ISAE - Jan. 2010
Systems of Systems Engineering :
Simulation-Based Acquisition
Pascal CANTOT M&S and SE domain manager DGA, Centre for Defence Analysis
[email protected]
MINISTÈRE DE LA DÉFENSE
Featured in this course Basics of modelling & simulation What it is
History Definitions, basic taxonomy Some use cases
System modelling How it works
Modelling process Specific models : stochastic systems, human behaviour, natural environment Verification & Validation
How M&S can / should be used to support systems and SoS engineering process What it can do… and cannot do!
M&S for SE Simulation based acquisition Battlelabs and experimentations
This is not a technical course on M&S ! MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #2
1
Simulation-Based Acquisition
ISAE - Jan. 2010
What is simulation ?
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #3
Why should I bother using simulation? Because I like it ☺ Because I was told to! (by my boss, customer…) Because I can’t afford not using it Because I can’t do otherwise Because I don’t know what system I should build Because I’m not sure how I should build it Because I want to design the “best” system (what means “best”?) … MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #4
2
Simulation-Based Acquisition
ISAE - Jan. 2010
When it’s too expensive EXAMPLE #1: a flight simulator A simulator is expensive, but cuts training costs − Price for an aircraft : ~ 40 M€ (Rafale) − Price for a simulator: ~ 10-25 M€ (full flight) ~ 5 M€ (trainer) − Aircraft flying cost: ~ 10 000-20 000 € / hour − Smart bomb: ~ 15 000 € − Tactical missile: ~ 250 000 € (at least!) − Simulator “flying” cost: ~ 300 € / hour (these figures are to be taken as a rough idea of costs)
Reduces nuisances and risks (especially for beginners!) But does not replace real flight time optimization
EXAMPLE #2 : missile testing,1 M€ / unit MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #5
When it’s not possible FORBIDDEN : Nuclear weapons detonation
TOO DANGEROUS : Airbus pilots’ training to failures UNPREDICTABLE : Study of tornados through simulation
NOT ECO-FRIENDLY : Propagation of an oil slick HAS NEVER HAPPENED YET : Nuclear war, large scale natural disaster… ... MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #6
3
Simulation-Based Acquisition
ISAE - Jan. 2010
When it’s too complex Current civilian or military products tend to become complex systems or even systems of systems Complex systems large number of components and interactions, emergent properties Difficult to have a clear global understanding High probability of faulty specification or design Prototype are so expensive that sometimes you can’t afford even one (ex.: Charles de Gaulle Aircraft Carrier)
Simulation can help you with specifying the product and validate the design with a much less expensive and more flexible virtual prototype DGA / RM MOS
MINISTÈRE DE LA DÉFENSE
28-Jan-2010
Slide #7
Typical architecture of a simulation INPUT DATA
SIMULATION
Parameters for simulated operators
Human behaviour models
Parameters for simulated system
System model
User inputs (user interface) OUPUT DATA
Simulation engine
Environment data
Environment
Output of system dynamic behaviour Results analysis
model
Scenario
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #8
4
Simulation-Based Acquisition
ISAE - Jan. 2010
Technical taxonomy of simulations Analogical
Digital Human-In-The-Loop Real Time Interactive
Hardware-In-The-Loop Live
Hybrid (testing)
Virtual
RT Constructive
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
NRT Constructive
Slide #9
Modelling & Simulation Basis
MINISTÈRE DE LA DÉFENSE
5
Simulation-Based Acquisition
ISAE - Jan. 2010
M&S general cycle “real world” system
Simulator or Simulation
Validation
+ Environment + Scenario
M e od d lle by
Validation
Im
e em l p
ed nt
in
Verification
Model(s) From Bernard ZIEGLER (circa 1973) MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #11
“Real World” System SYSTEM : combination of integrated elements (products, humans, processes), organized to achieve, within a giving environment, one or more stated purposes. [ MIL-STD-499B, EIA/IS-632, ISO-12207, SE-CMM, ISO/IEC 15288, EN 9200, DODAF, etc.]
The system can exist or be a future system Any M&S process must always begin with problem and purpose(s) statement
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #12
6
Simulation-Based Acquisition
ISAE - Jan. 2010
Model MODEL : physical, mathematical, or otherwise logical representation of a system, entity, phenomenon, or process. [designed for a stated purpose] [US DoD MSMP, Directive 5000.59-P]
r F
Example : System = Billiard ball falling in gravity field
gr
Model = movement equations Acceleration : Speed : Position :
MINISTÈRE DE LA DÉFENSE
a (t ) = g
v (t ) =
∫ a(t)dt =gt + v
z (t ) =
0
1 2 gt + v0t+ z0 2
DGA / RM MOS
28-Jan-2010
Slide #13
Some various models…
Sélection d'une force militairex Niveau des ressourcess
Sélection d'une force militairex Taux de mobilisation m1
Croissance initiale de la force militaire x dépendant de la densité : m1 * x * (1 - x/s)
Sélection d'un coef. d'attrition par tirs directs : c1
Attrition de type Lanchester due aux tirs directs de la force y sur la force x : - c1 * y
Valeur de la force de x
Défection linéaire dex : - m2 * x * y
Sélection d'un coef. d'attrition par arme à effet de zone :c2
Attrition de type Lanchester due aux tirs à effet de zone de la force x sur la force y : - c2 * x* y
Recrutement linéaire dans la force insurgée y : m2 * m3 * x * y
Valeur de la force de y
Pertes linéaires dues à la forcey : - m4 * y
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #14
7
Simulation-Based Acquisition
ISAE - Jan. 2010
Simulation / simulator SIMULATION : Execution over time of models [for a stated purpose] [IEEE M&S glossary]
Simulation can also name a simulation application “Modelling and simulation” (M&S) is the discipline dedicated to simulation design and execution Simulator : device, computer program, or system that performs simulation (For training, a device which duplicates the essential features of a task situation and provides for direct human operation)
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #15
Modelling vs Simulation Modelling and simulation must not be confused : Modelling is done by domain experts − Technical experts: radars, missiles, etc. − Operational experts / end users: use cases, doctrine, etc. Simulation is implementation of models by “simulationists” − Mathematicians : integration, random generator, Monte Carlo… − Computer scientists : federate, interface, event, state variable…
Need for a close dialogue between stakeholders: Building of “conceptual models” Use of shared processes, methodologies and high-level languages (MDA, UML…) Sharing of “simulation support environments” (ex.: DirectSim)
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #16
8
Simulation-Based Acquisition
ISAE - Jan. 2010
What is a “good” model ?
A model should: – Be as simple and clear as possible – Be valid (and validable!) – Have the best fidelity considering the purposes of the simulation project – Be the most efficient considering the pursued goal There is no such thing as ONE good model :
“All models are wrong but some are useful” (George E. P. Box, Industrial statistician)
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #17
e.g.: choice of a level of details Level of details used to represent a real world system with a model (e.g. : missile flight model) A very acute mathematic model requires more efforts than a behavioural model, more parameters and more computer ressources APPROXIMATIVE
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
↔
SUPER-ACCURATE
28-Jan-2010
Slide #18
9
Simulation-Based Acquisition
ISAE - Jan. 2010
Other taxonomic criteria for M&S Level of details (see given example)
Granularity Size of objects/entities managed by the model Aggregation : an organized group of entities having its own higher-level behaviour (ex.: plane
patrol)
Time management Real Time, Time Stepped, Event Driven
Distributed / Monolithic (standalone) Ex.: designed as an HLA federation Design choices depends on the final purpose, but in any case validity must be preserved MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #19
Can I trust my simulation? Simulation is a powerful decision tool for system engineering Therefore its results can influence decisions with dramatic consequences Examples : choice of an architecture, dimensioning of a bridge, efficiency of a medical drug, safety of an aircraft…
Nevertheless : Simulation is a computer application Simulation is based on a subjective abstraction of real world called a “model” This models needs parameters data
How can I evaluate the trust I can put in my simulation? MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #20
10
Simulation-Based Acquisition
ISAE - Jan. 2010
The answer is : V V & A Verification : process of
Validation : process of de
determining that a model or a simulation is implemented in accordance with its specifications
determining the degree to which a
Did I make the product right? [and can I prove it?]
Did I make the right product? [and can I prove it?]
model or simulation is an accurate representation of the real-world for its intended uses
Acceptation : The official certification [by sponsor, client, end user…] that a model or simulation is acceptable for use for a specific purpose Is my simulation useful to solve my client’s problem ?
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #21
Validation domain is relative Earth modelling for artillery Flat Earth : Suitable for small gun indirect fire up to 20km Spherical Earth : Ok for tactical ballistic missiles Ellipsoidal Earth : Better suited for strategic (intercontinental) ballistic missiles Geoid MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #22
11
Simulation-Based Acquisition
ISAE - Jan. 2010
Why VV&A cannot be ignored Improving trust in simulation results To insure that decision based on simulation results are not biased
Reusing models and simulations To improve efficiency while lowering costs and delays To measure to what extent the reuse application or component can be trusted
Quality insurance To demonstrate that the customer got a product meeting its needs
It is crucial to follow (or to impose) a VV&A process MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #23
Verification Verification is mostly software engineering : Software quality Code and documentation (self or peer) checking Tests Debugging functions and GUI …
Verification is also for data ! Input data checking Mars Climate Orbiter (’99) Consistency with (evolving!) real world system Beware of units use MKSA or explicit units MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #24
12
Simulation-Based Acquisition
ISAE - Jan. 2010
Anecdote: the most stupid way of losing a very expensive system NASA Policy : “Faster, Better, Cheaper” Sep.99: Mars Climate Orbiter disappeared while slowing to be inserted into Mars orbit Everything seemed fine, so what happened? NASA requires subcontractors to use MKSA units Mars Climate Orbiter Lockheed Martin used acceleration data in pounds instead of Newtons (1pd = 4.48N) The probe slowed down too much and entered Mars atmosphere where it was destroyed There was a strict verification process, but the error was so gross that nobody found it First reaction from the subcontractor: to review the contract to see if use of MKSA units was clearly stated !
(’99)
Kosmos 419 (1971), another lost Martian probe: delay on parking orbit 1.5 years instead of 1.5 hours! MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #25
Validation No general technique, but need for a rigorous process V&V is never a one shot activity, but a continuous process Some examples of validation techniques : “Desk checking” by yourself or someone else Documentation checking Sensitivity analysis Testing at limits Consistency checking with real world system Turing test Formal methods Visualization / animation of the model within a GUI Comparison with real world tests of the system Comparison with real world tests of similar system …
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #26
13
Simulation-Based Acquisition
ISAE - Jan. 2010
Comparison between live and simulation Electric gun concept evaluation 'X-ray' from simulation
Simulation
ERA front plate
Residual penetration in main armor
3D Euler 'X-ray' from experiment (EMI)
Experiment.
(EMI) (EMI)
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #27
In any case… A validated simulation MAY be good A non-validated simulation is usually crap useless or worse ! This VV&A thing must be a concern for ALL stakeholders
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #28
14
Simulation-Based Acquisition
ISAE - Jan. 2010
Is there any VV&A standard around? Yes, there is a VV&A overlay for DIS and HLA standards
Yes!
IEEE 1516.4-2007
A SISO VV&A generic standards is expected in 1-2 years Many local standards Abundant literature
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #29
M&S for System Engineering : The DGA use case
MINISTÈRE DE LA DÉFENSE
15
Simulation-Based Acquisition
ISAE - Jan. 2010
Simulation for defence SE process Simulation is a key tool for decision making in SE process : Evaluate global architectural concepts Analyze tradeoffs between operational (mission) capabilities, performances, costs Choose the optimal architecture for the system Manage specifications Prove technical feasibility before building the system Determine the adequate organization to conduct its development (and manufacturing, deployment…) State verifiable specifications Manage Explore different options for manufacturing in order to optimize design & implem. the chosen solution regardless technical constraints, costs, delays… Manage Insure system consistency throughout the different stages of its integration at life cycle a higher level and Facilitate reuse of some elements or sub-systems within other evolutions systems Manage user’s need
DGA / RM MOS
MINISTÈRE DE LA DÉFENSE
28-Jan-2010
Slide #31
Simulation strategy at the DGA Study operational concepts Provides required performance levels 3. Evaluate operational efficiency 4. Assess technical risks 5. Assess human factors risks 6. Evaluate feasibility for each solution 7. Optimize architecture or function 8. Facilitate understanding and sharing of computing and study results 9. Design system manufacturing and/or integration 10. Prepare qualification testing 11. Support system qualification testing 12. Specify evolutions of Human-System Interfaces (in red : industry-only activity) 1. 2.
Granularity
SoS 3 System
1 5
12
7
Function
2
9
11
6 8 10
4
Physical
Life cycle
Prepa MINISTÈRE DE LA DÉFENSE
Design DGA / RM MOS
Implementation 28-Jan-2010
Use, evolutions, disposal Slide #32
16
Simulation-Based Acquisition
ISAE - Jan. 2010
Study operational concepts
1
Preparation stage
Illustrate a concept, with or without human in the loop To make end-user react on the concept by using a clearly understood abstraction Example: in the French Battlelab, use of serious game to illustrate concepts with realistic views
MINISTÈRE DE LA DÉFENSE
2
DGA / RM MOS
28-Jan-2010
Slide #33
Evaluate performance levels Preparation stage
Idea : use of existing high-fidelity models to get an estimation of required performance levels for elements and sub-systems Use these results to feed lower fidelity, higher-level simulations ( see SBA)
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #34
17
Simulation-Based Acquisition
2
ISAE - Jan. 2010
Example: OURANOS : evaluation of damages on concrete walls by a contact detonation (DGA/CEG)
From this virtual experiment, you can deduce : The required wall resistance for a building
The required explosive power for an ammunition
MINISTÈRE DE LA DÉFENSE
3
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Operational efficiency analysis
DGA / RM MOS
28-Jan-2010
Slide #35
Preparation stage
Slide #36
18
Simulation-Based Acquisition
ISAE - Jan. 2010
Risk assessment and system sizing (BEC)
4
Design stage
Visualize Von Mises constraints on ship hull Where are the weak spots and constraints concentration spots? How the constraints evolve with sea state? Is there a risk of structural rupture? How thick and resistant the hull must be? MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #37
Technical feasibility : CEP/Arcueil
6
Design stage
Concepts for Army robots Evaluation of perception algorithms Level of autonomy
Can the required performances be implemented with current technologies?
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #38
19
Simulation-Based Acquisition
ISAE - Jan. 2010
Optimize an architecture (ETBS)
7
Design stage
Study the global firing function : how the different phases of the firing of a “smart” ammo follow on
155 mm gun Ballistic phase
Laser reflection Constant angle descent phase
Detection Final Guidance phase Laser designator
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #39
Facilitate understanding of results
8
Design stage
Example : Virtual Ship (CTSN)
Test or simulation results are difficult to interpret and explain Simulation provides visual restitution of data within an operational scenario
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Fire propagation (LUCIFER)
Slide #40
20
Simulation-Based Acquisition
ISAE - Jan. 2010
Facilitate understanding of results
8
Design stage
Example : Virtual Wind Tunnel (NASA)
Test or simulation results are difficult to interpret and explain Simulation provides visual restitution of data within an operational scenario
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #41
Design system manufacturing
9
Implementation stage
QUEST Simulation, programme AAAV, USA How the production line should be implemented ? Does this organization allows workers to be efficient ? MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #42
21
Simulation-Based Acquisition
ISAE - Jan. 2010
Prepare qualification tests (CEPr)
10
Implementation stage Simulation of potentially dangerous shock waves in the test tunnel for a supersonic vector (with statoreactor)
Field testing of complex system are usually themselves difficult to design and expensive to implement Simulation provides support to help optimize the testing, improving cost/performance ratio In this example, simulation was used to assess potentially hazardous behaviour of the tested system MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #43
Support to qualification testing
11
Implementation stage
Widen system knowledge: Testing the system beyond its definition domain Testing the system with a scenario that can’t be fielded (ex.: strong EW) Multiply (virtual) testing scenarios
Increase confidence in testing: Better understanding of testing results Position a limited number of field testing in a larger (statistically significant) population Improve ability to measure performances
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
ETAS
Slide #44
22
Simulation-Based Acquisition
ISAE - Jan. 2010
Important warning !
Simulation and field testing do not compete, but complements each other M&S needs field testing data « There’s no simulation like the real thing » !
(CEG)
DGA / RM MOS
MINISTÈRE DE LA DÉFENSE
11
28-Jan-2010
Slide #45
Note : Hybrid Simulation (LRBA) Implementation stage GPS Simulator
Testing of homing system for MICA AA missile.
Flight model
Image Generator
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
MIRAGE
28-Jan-2010
Slide #46
23
Simulation-Based Acquisition
ISAE - Jan. 2010
Improve Human-System interface (CEV)
12
Use stage Design stage
Use of representative simulator devices to analyze cockpit ergonomics for initial requirements and later IHS evolutions
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #47
More benefits from simulation Develop technical expertise Support discussions between different experts Encourage transparency
Manage complex systems configuration
Dassault Aviation
Sharing of virtual prototypes
Lower costs and delays
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #48
24
Simulation-Based Acquisition
ISAE - Jan. 2010
Conclusions on the use of M&S for SE cycle Simulation is now widely recognized as an valuable asset for SE Can be a definitive competitive weapon, see Airbus
But it’s often difficult to measure precisely what it bring to system life cycle (or in other words its ROI, especially for systems of systems) When simulation is not downright used as an “adjustment variable” it is often not used in a coherent and integrated way by SE stakeholders This generates costs and increase difficulty to build and reuse required simulations
Simulation-Based Acquisition approach to SE MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #49
Simulation Based Acquisition
MINISTÈRE DE LA DÉFENSE
25
Simulation-Based Acquisition
ISAE - Jan. 2010
Needs of the acquisition process Manage complexity, induced by: SoS approach Stronger interoperability requirements Global life cycle management (think in 4D!)
To be reactive all along the life cycle: Exploit new technologies Adapt to context evolutions
To do at best with available (shrinking) budgets Capability (and SoS) approach
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #51
SBA Initiative : the US response DoD Acquisition Council, Dec 97 : « [SBA is] an acquisition process in which DoD and Industry are enabled by robust, collaborative use of simulation technology that is integrated across acquisition phases and programs »
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #52
26
Simulation-Based Acquisition
ISAE - Jan. 2010
Simulation Based Acquisition (SBA) BEFORE
Stage 4 Stage 3 Stage 2 Stage 1
AFTER
costs
Higher level requirements
Implementation support
Architecture
Behaviour Environment
Simulation during acquisition Sequential use of tools with limited scope and scalability No interoperability between tools Independent and heterogeneous databases, lack of configuration management and traceability MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
Simulation based acquisition ☺ Integrated concurrent process ☺ Reuse at larger scale ☺ Reduce costs and risks
28-Jan-2010
Slide #53
Principles and benefits 3 typical SBA axes of efforts : Processes evolution • Interactive exchanges of system models • Collaborative and distributed teams, mixing all stakeholders: acquirer, customer, operator, manufacturer…
Frameworks and environments • Integrated simulation design and execution environment • Consistency and traceability checking • Direct Link between design and M&S • Automatic generation of products to support the SE activities
Culture • Reduction of SE teams • Integrated teams • Evolution in roles and responsibilities
• Rapid analysis of architectural choices • Communication et shared understanding of design data • Rapid impact analysis following a change in requirements • Online integration and testing • Reduced risks of redesign • Changes and technological breakthrough management • Reuse and re-engineering of existing design patterns • Improved collaborative work between stakeholders • Improved product quality • Reduced costs and delays
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #54
27
Simulation-Based Acquisition
ISAE - Jan. 2010
SBA and SE
System Engineering Process Simulation Based Acquisition Process
DGA / RM MOS
MINISTÈRE DE LA DÉFENSE
28-Jan-2010
Slide #55
A few facts about benefits from SBA According to US reports : (Defence, Aerospace, car manufacturers)
Design:
30 to 60% improvements in delays 30 to 50% decrease in costs Implementation: - 50% changes ( risk reduction) 20-25% decrease in costs
Some examples: Boeing 777 : 60% à 90% decrease in redesign operations COMANCHE : 34 M$ invested in simulation Estimated gain : ~640 M$ (8% off programme costs) MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #56
28
Simulation-Based Acquisition
ISAE - Jan. 2010
NASA Study (1985)
Cost overrun (%)
Cost ratio between preparation phase and design phase
Is it too much?
The more you invest in initial stages, the more you reduce risks for the remaining stages less cost and delay slides DGA / RM MOS
MINISTÈRE DE LA DÉFENSE
28-Jan-2010
Slide #57
Return on Investment for SBA Cost slide
Cumulative expenses frozen by taken decisions SBA
gain due to lower slides gain due to risk reduction
A SBA process generates an additional cost immediately transformed to gain as soon as the first unforeseen even occurs. From D. Luzeaux MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #58
29
Simulation-Based Acquisition
ISAE - Jan. 2010
Distributed Simulations Standards to support SBA
MINISTÈRE DE LA DÉFENSE
Some food for thoughts… No single, monolithic simulation could satisfy the needs of all users All uses of simulations and useful ways of combining them in the future could not be anticipated in advance Future technological capabilities and a variety of operating considerations would have to be accommodated
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #60
30
Simulation-Based Acquisition
ISAE - Jan. 2010
Conclusion by US DMSO “DoD would be best served by adopting a composable approach to constructing simulation federations” COMPONENT-BASED DESIGN FOR SIMULATIONS But for this, standard(s) are needed…
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #61
How models and simulations can be composed… a,b,c: loose coupling Data exchange through filesystems Coupling at each time step or run Very easy to implement, but very limited and does not allows strong interactions between models (and don’t even think about RT)
d: tight coupling Models directly interacts with each others Much more powerful way But much more challenging to implement
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #62
31
Simulation-Based Acquisition
ISAE - Jan. 2010
Example of loose coupling
Fluid mechanics model + Structure model Von Mises constraints in a parachute DGA / RM MOS
MINISTÈRE DE LA DÉFENSE
28-Jan-2010
Slide #63
Several ways to tightly couple models Direct link between models (not good unless simple case) Simulation framework Distributed simulation framework distributed simulation application simulation application model
model
model
simulation framework
simulation application model
model
model
simulation framework
Communication infrastructure (network, software bus, messages, shared memory...) MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #64
32
Simulation-Based Acquisition
ISAE - Jan. 2010
Simulation frameworks A simulation framework provides services required by simulation application library
Models GUI
API
Simulation framework Operating System
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
(simulation engine, GUI, I/O, maths, logs, object mgt… mgt…)
It also usually provides a modelling methodology documentation and sometimes a graphical modelling tool CASE tool It facilitates application development and understanding (sometimes Rapid Application Dvt) Dvt) It fosters reuse of validated models and application scalability and portability
28-Jan-2010
Slide #65
GOTS Example : DirectSim SSE C# / .NET, MS Visual Studio UML Code generation
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #66
33
Simulation-Based Acquisition
ISAE - Jan. 2010
COTS Example: Matlab / Simulink
MATLAB & Simulink : www.mathworks.com MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #67
A few definitions Distributed simulation: simulation application built from software components which are independent applications that can be located on one or several host computers
JANUS AZUR (USA)
JANUS ORANGE (Ennemis)
DATA LOGGER
PLAN VIEW DISPLAY
INTERNET
Interoperability: ability of a model or simulation to provide services to and accept services from other models and simulations, and to use the services so exchanged to enable them to operate effectively together
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
JANUS AZUR (Français)
DATA LOGGER
Slide #68
34
Simulation-Based Acquisition
ISAE - Jan. 2010
Distributed ≠ Parallel PARALLELISM: simulation is designed in order to allow its execution on several CPUs (or cores) on one host DISTRIBUTION: simulation is designed with several autonomous parts, each one can be executed on a different host PADS: parallel and distributed simulation, for closed, digital, monolithic models, in order to accelerated computing speed on multi-CPUs hosts, clusters, grids…
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #69
Short history of distributed simulation 1960-70s: networks, time sharing. Centralised computing model 1980s: workstations, personal computers. Decentralised computing models. Distribution becomes (too?) fashionable 1987: SIMNET 1992: CORBA 1993: DIS (Distributed Interactive Simulation), ALSP 1994: First French distributed simulation experimentations 1994: DSI network (US Defense Simulation Internet) 1994: « DIS vision » 1995: HLA 1.0 1995: U.S. M&S Master Plan (and later NATO MSMP) 2000: HLA IEEE 1516 2004: SEDRIS 2008: TENA (Test & Training Enabling Architecture) 2009: HLA Evolved MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #70
35
Simulation-Based Acquisition
ISAE - Jan. 2010
A few words on DIS Protocol Standard IEEE 1278 Vocabulary: Federation, Simulation, Entity, Interactions Diffusion des variations des attributes (periodic) Attributes extrapolation (dead reckoning) No coordinated time, limitations for non real time No central node, autonomous federates Use of low level standardised binary messages (PDU) Mandatory coordinate system (WGS84) UDP Broadcast coms (non reliable, greedy for bandwidth) Now obsolete, but still alive (40% market share!) Rather well suited for virtual RT simulation (not for SE)
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #71
The High Level Architecture (HLA) is… An interoperabily standard (and NOT a protocol nor a data format) An IEEE standard (IEEE 1516 since 2000) A methodology to design and implement simulations which can : actually cooperate (and not only communicate), in a consistent and significant way
Without imposing any constraint on hardware Applicable to all kind of simulation (LVC, RT/non-RT) Main goals : simulation interoperability & reuse Component-based design MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #72
36
Simulation-Based Acquisition
ISAE - Jan. 2010
In a nutshell HLA is… HLA is defined by 3 documents : HLA Rules (10 commandments!) IFSPEC (interface specifications) OMT (object model template)
HLA compliance = to respect these 3 documents HLA compliancy testing consists in checking with static face verification and dynamic testing with dedicated tools if the simulation (federate) is compliant to these 3 documents
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #73
Important notice HLA is necessary but not enough for simulation interoperability (HLA ain’t no Superman)
HLA
For example, HLA does not handle environment database problems SEDRIS, CDB, OpenFlight…
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #74
37
Simulation-Based Acquisition
ISAE - Jan. 2010
Natural environment isn’t only DTED…
Topographic details (road, vegetation, buildings…) Satellite picture DTED (elevation data)
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #75
… but much more !
Example of Natural Environment database MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #76
38
Simulation-Based Acquisition
ISAE - Jan. 2010
Some vocabulary (1/2) Simulation and tools interoperate within a federation Federation can be seen itself as a simulation ( component-based design) Federation components (simulations, tools, interfaces) are named federates
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #77
Some vocabulary (2/2) Entities handled by federated are objects, instances of an object class These objects have values attached to them, called attributes Object can interact with each other These interactions have values attached to them, called parameters
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #78
39
Simulation-Based Acquisition
ISAE - Jan. 2010
HLA is object-oriented
Vehicle
Class instantiation Inheritance mechanism (“is a…” relationship)
Ground vehicle
Class Tree
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
Car
28-Jan-2010
Aircraft
Truck
Slide #79
Fundamentals of HLA (some rules) Federation is documented by FOM, federates by SOM All exchange of FOM data among federates occurs via the RTI, through the API Each federate manages its objects (or some of their attributes) and share them (publish their attributes) with other federates RTI does not manage objects, the federates do that There is only one instance of any given object (or attribute) within the federation Federates can exchange interactions between their objects Each federate must be able to manage time locally in coordination with other members of the federation
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #80
40
Simulation-Based Acquisition
ISAE - Jan. 2010
FOM / SOM Federation Object Model (FOM) : Describes the shared object, attributes and interactions for the whole federation
Simulation Object Model (SOM) : describes the shared object, attributes and interactions used for a single federate [“public” part of the simulation]
⎛ ⎞ FOM ⊂ ⎜⎜ U SOM i ⎟⎟ ⎝ i ⎠ It is mandatory to write a minimal, standardized documentation of any simulation facilitate reuse
DGA / RM MOS
MINISTÈRE DE LA DÉFENSE
28-Jan-2010
Slide #81
What about legacy (DIS) simulations?
DIS Federate Native HLA Federate
DIS Federate
DIS Federate middleware
PDU DIS gateway
RTI HLA
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #82
41
Simulation-Based Acquisition
ISAE - Jan. 2010
HLA Federation architecture Live participants Interfaces
Tools
Simulations
with
live systems
Interface
Runtime Infrastructure (RTI) Federation management Objects management Time management (+ other services…)
Declarations management Ownership management Data distribution management
DGA / RM MOS
MINISTÈRE DE LA DÉFENSE
28-Jan-2010
Slide #83
HLA FEDEP (or DSEEP) Program Objectives Requirements
Available Resources
Define Federation Objectives
Standard Federation Development Process
Initial Planning Documents
1
Federation Objectives Statement
Federation Develop Scenario Federation Conceptual Model 2 Design Federation Federation Conceptual Model 3 Federation Requirements Allocated Federates
Test Evaluation Criteria
Federation Development Plan
RTI RID File FOM
Develop Federation 4
Scenario Instance Modified Federates
fee db ac k
MINISTÈRE DE LA DÉFENSE
FED File
User Feedback Tested Federation
Integrate and Test Federation 5 Testing Data
Execute Federation and Prepare Results 6 Reusable Products
DGA / RM MOS
28-Jan-2010
Slide #84
42
Simulation-Based Acquisition
ISAE - Jan. 2010
HLA USE CASE : EDISON
Spacecraft Validation with Hardware-In-the-Loop MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #85
HLA : EDISON (ATV)
ATV
Automatic Docking
ISS
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #86
43
Simulation-Based Acquisition
ISAE - Jan. 2010
HLA : EDISON (EPOS)
EPOS
European Proximity Operation Simulator
ATV
ISS DGA / RM MOS
MINISTÈRE DE LA DÉFENSE
28-Jan-2010
Slide #87
EDISON : ARCHITECTURE Distributed Simulation Facility
France
Germany
EDISON operator
ATVATV-FSF
EPOS
ATV
FTC
GPS
PDE
MIL-1553 RVS surrogate
FrontFront-ends Simulators
ATVATV-FSF Command / Control
Ethernet
DGA / RM MOS
ISS Target Motion Device Reflectors
Illumination System
EDISON FSF EPOS kernel surrogate surrogate
Speech Vision Gesture
MINISTÈRE DE LA DÉFENSE
MIL-1553 FTC surrogate
Chaser Motion Device RVS
Ethernet
EPOS Command / Control
Speech Vision Gesture
28-Jan-2010
Slide #88
44
Simulation-Based Acquisition
ISAE - Jan. 2010
EDISON : OBJECTIVES To demonstrate potential of distributed simulation for testing and validation of hardware on remote locations To use distributed simulation is less expensive than moving equipments from one location to another Distributed simulation is particularly interesting in multinational projects HLA can be used in real time (latency < 100 ms) Note: EDISON is an ESPRIT civilian R&T project
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #89
EDISON : DEMONSTRATION
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #90
45
Simulation-Based Acquisition
ISAE - Jan. 2010
Integration of large, heterogeneous federation PABST Ψ-SA
EGMonT Ψ-SA AIME Test Suite
OTB
DIS Logger
DIS-1516 Adaptor
Rapid Multinational Federation Integration … integrated in one week
pRTI pRTI1516 1516v3.0 v3.0 RPR RPRFOM FOM2.0 2.0D17 D17
AIME Duplicator
Ψ-SA KAPLAN ASCOT
ASCOT
Interdaptor Bridge
DMSO DMSONG NG1.3 1.3v6 v6 RPR RPRFOM FOM1.0 1.0 SIMBAD
MINISTÈRE DE LA DÉFENSE
Ψ-SA GPSim
DMSO NG 1.3 v6 DMSO NG 1.3 v6 SIMBAD FOM SIMBAD FOM
Flexibility for reuse provides a way to build SoS simulations! DGA / RM MOS
28-Jan-2010
Ψ-SA Stealth
Slide #91
Is distribution a magic wand ? Distribution is difficult To distribute well ⇒ easy reuse To distribute too much ⇒ poor performance + complexity Network traffic (dead reckoning, grid filtering, …) Synchronisation (event causality…) Alternatives : monolithic application? Loose coupling? Security problem : a model can carry confidential data Distribution is more a constraint than an end in itself Distributed Simulation is unfortunately the best way to build complex synthetic environment for system of systems modelling & simulation
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #92
46
Simulation-Based Acquisition
ISAE - Jan. 2010
To build Synthetic Environments for SoS design
DMSO
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #93
BATTELABS An approach to SoS specification & design
MINISTÈRE DE LA DÉFENSE
47
Simulation-Based Acquisition
ISAE - Jan. 2010
Needs of the acquisition process (déjà vu?) Manage complexity, induced by: SoS approach Stronger interoperability requirements Global life cycle management
To be reactive all along the life cycle: Exploit new technologies Adapt to context evolutions
To do at best with available (shrinking) budgets Capability (and SoS) approach
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #95
In addition to that : Network Centric Warfare more concern about Systems of Systems Need for actual collaborative work (govt – industry – forces) NATO CD&E : Concept Development & Experimentation Simulation-Based Acquisition More mature technologies: HLA,virtual reality, engineering tools… Development of Battlelabs, mostly – but not only – for Defence (USA, UK, Sweden, Australia, France…)
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #96
48
Simulation-Based Acquisition
ISAE - Jan. 2010
What is a Battlelab? Example: French LTO LTO is the French MoD Battlelab Mission: Support analysis studies at capability and SoS levels dealt through 6 axes : doctrines, organization, equipment, personnel, training & sustainment Main issues to be addressed through LTO: Global requirements for SoS Large number of combinations for architectural solutions Large number of issues at stake and stakeholders Complexity of systems and interfaces between systems and partners (Allied Nations)
LTO is not (or not only) a set of technical resources but rather a method to solve complex problems MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #97
Example: BMD – Ballistic Missile Defence How to create a BMD capability using existing systems? Taking advantage of emerging properties
Both a technical and organizational/concept of use problem MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #98
49
Simulation-Based Acquisition
ISAE - Jan. 2010
LTO: from concept to capabilities
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #99
Methodological requirements of LTO Foster cooperation between militaries and engineers Better understand concepts & needs Better knowledge of technologies
Improve system engineering practice : Model SoS and organizations Manage capabilities over time Enable Administration to be arbiter
Take benefit from simulations & XP Illustrate new concepts to operators Compare architectures Integrate human factors
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #100
50
Simulation-Based Acquisition
ISAE - Jan. 2010
Services from LTO Shared with industry Can interoperated with other BL
Services: Brainstorming animation (LTG) (concept exploration, scenario design…) Board games, role playing games Architecture modelling Simulation architecture consulting Concept illustrations through simulation or « serious games » (Sensurprys, VBS2, VR Forces, STAGE…) Support to design of analysis simulation Communication network for experimentations Technical support to experimentations (videoconferences, architecture, debriefing…)
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #101
SoS architecture modelling Architecture modelling tools: System’s architectures: DOORS, SYSTEM ARCHITECT Operational processes & organizations : MEGA
Objectives : Consistency between doctrines, architectures & technologies Impact analysis of engineering changes Capability management over time
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #102
51
Simulation-Based Acquisition
ISAE - Jan. 2010
SOME EXPERIMENTATIONS Capability analysis: TST New Concept Experimentation: PHOENIX 2008
MINISTÈRE DE LA DÉFENSE
Targeting & TST capabilities TST Organizational changes
Moving
Movable
20’
1h
Fixed target 48h Available capability MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Technological changes Slide #104
52
Simulation-Based Acquisition
ISAE - Jan. 2010
Organisational modelling SOCC
FHQ TCE
DGA / RM MOS
MINISTÈRE DE LA DÉFENSE
LCC
CAOC
28-Jan-2010
Avion
Slide #105
time sensitive target
TST scenario & facilities JOINT
CELAr Bruz Dém. COP
FHQ TCE
SICMAR
SICA
SICF Land TST Cell
Navy TST Cell Air control
GRANITE NG SP. OPS TST Cell MAESTRO
PF SENIT 8
TACTICAL
CTSN/Toulon
MAESTRO SIMU EADS SIMU EADS
SIMU C2 SIMU M2000D
SAIS Issy
CEV/Istres MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
Observer
Target
28-Jan-2010
Slide #106
53
Simulation-Based Acquisition
ISAE - Jan. 2010
Tools that were involved in TST:
From modelling
… to experimentation
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #107
PHOENIX 2008
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #108
54
Simulation-Based Acquisition
ISAE - Jan. 2010
Main objectives for the XP Indirect fire management in manoeuvre for French Army Evaluate 2 new tools for Unit Commanders (=Captain): Manoeuvre Management Cell (CCM) Specialized Surveillance Cell (CSS) – RETEX Ph’07 Illustrate new capabilities or optimize existing capabilities: Beyond sight firing (TAVD) and short loop close support Exploit sensor images for Captain’s decision making Coordinate collective actions (firing, moving…) Experiment at SGTIA level (combined tactical group, ~200 men) with additional mortar, missile and enhanced TAVD capacity How to support Army Unit Commander in the future ? Network Sensors Fire support MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #109
LTO Philosophy was applied Mixed team (Industry – Army – DGA) Each one contributes Analyze and meet all participants’ expectations win-win relationship Federate individual know-how
Methods: several steps Common experimentation design Fielding of the experimentation Results analysis and lessons learnt
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #110
55
Simulation-Based Acquisition
ISAE - Jan. 2010
Experimentation design Use of collaborative work laboratory : LTG Technical and operational objectives for the XP Scenarios that include these objectives Metrics
Simulation 3D Terrain Digitalization Setting up the environment to help finalize scenarios
System engineering tool: MEGA Modelling of communication streams Design of networks
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #111
Experimentation fielding Setting up “spies”: MEEFISTO Network communications Permanent logging of any action or message from operators (CSS and CCM)
Continuous ergonomic evaluation: An ergonomist behind every operator Daily debriefing (technical and operational)
Stimulation by injected virtual images: Replacement for faulty fielded sensors, in real time
Combination of existing equipment from Legion Étrangère and demonstrators
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #112
56
Simulation-Based Acquisition
ISAE - Jan. 2010
Concepts assessment Immediate evaluation: Equipments: needs for evolutions of existing equipments Doctrine: repartition of functions, processes Experimentation: logistics, methods, simulation integration
Later, after some work: Lessons learnt for each equipment from industry Equipment and doctrinal benefits Analysis of remaining issues
This is an illustration with a pure military example, but what is important is the experimentation concept, which can be (and IS) used in civilian context: crisis management, company organization, restructuring plan, logistic system design…
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #113
Other Battlelabs in France Thales: Battlespace Transformation Center (Thales Integration Center) DCNS: Naval Future Capability Center (Solaris) EADS: System Design Centre (NetCOS) MBDA: Niteworks Dassault Aviation: Atelier d’Emploi CS: Development & Experimentation Centre for Transformation, Joint Battlelab …
THALES
EADS
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #114
57
Simulation-Based Acquisition
ISAE - Jan. 2010
CONCLUSION
MINISTÈRE DE LA DÉFENSE
Some final words… Modelling & Simulation is a very mighty tool for SE and SoSE It provides best benefits in this context by being integrated within engineering processes and structures (SBA, Battlelabs…) and other techniques (field testing, formal methods…) M&S requires methodology SE of simulations! Simulation development process, VV&A… M&S has now reach a good maturity, but it will still evolve: it hasn’t achieved its full potential yet
MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #116
58
Simulation-Based Acquisition
ISAE - Jan. 2010
THE END...
or just the beginning ? MINISTÈRE DE LA DÉFENSE
DGA / RM MOS
28-Jan-2010
Slide #117
59
