Intelligent vehicles: integration and issues Z. Alsayed, G. Bresson, P. Merdrignac, P. Morignot, F. Nashashibi, E. Pollard, G. Trehard
IMARA Informatique, Mathématiques, Automatique, pour la Route Automatisée
became
RITS Robotics & Intelligent Transportation Systems
Praxitèle
Imara
Rits
1990
2002
2014
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
-2
Robotics vs. Artificial Intelligence (AI) Robotics: science of perceiving and manipulating the physical world through computer-controlled mechanical devices
AI: any computer program which would be said "intelligent" if the same observed behavior would be so qualified when performed by a human.
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
3
Robotics • “ Robotics is the science of perceiving and manipulating the physical world through computer-controlled mechanical devices.” [S. Thrun, 2006]
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
4
An attempt to define an intelligent robot • Ultimate goal: ensure its survival in its environment • Ensure its energy independence • Diagnose its own state • Evaluate its perception abilities
• Achieve a mission • React properly to an unknown/abnormal situation • Learn from experience
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
5
Work in progress with intelligent cars • Ultimate goal: ensure its survival in its environment • Ensure its energy independence • Diagnose its own state • Evaluate its perception abilities
• Achieve a mission: move safely from a point A to B • React properly to an unknown/abnormal situation • Learn from experience
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
6
Inputs
Automation loop Process
Sensors Exteroceptive
Knowledge base
Camera LIDAR RADAR
Perception
Odometry Inertial GPS
Communications
Supervision
Proprioceptive
Planning Global Planning (Route Planning)
Local Planning (Trajectory Planning + Trajectory Coordination)
Outputs
HMI Actuators Steering/Direction
Control (Trajectory Execution)
Speed/Propulsion
How to introduce intelligence (human behavior) into the driving process? Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
7
How do we introduce intelligence into the driving process? 1. Supervision through Software Architectures: a unified framework 2. Dealing with uncertainty 3. Increase the perception using communication 4. Limit the combination explosion
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
8
Supervision through software architectures: a unified framework
Inputs
• reaching a full autonomous driving mode in all situations impossible • self-assessment for the vehicle of its own perception abilities Process
Sensors Exteroceptive
Know. base
Camera LIDAR RADAR
Odometry Inertial GPS
Outputs
Communications
Supervision
Perception Proprioceptive
Planning Global Planning
Local Planning
HMI Actuators Steering/Direction
Control
Speed/Propulsion
E. Pollard et al., An Ontology-based Model to Determine the Automation Level of an Automated Vehicle for Co-Driving Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
9
E. Pollard, An Ontology-based Model to Determine the Automation Level of an Automated Vehicle for Co-Driving
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
10
Longitudinal control layer Levels of automation in terms of decisions to make about…
0: fully driving
P1
Long1: Cruise
Longitudinal control
control
Long2: Dynamic Set Speed Type
P2
Long3: Autonomous CC
P3
Long4: Stop&Go
CLong: Cooperative cruise control communication
C1
Increasing needs in terms of perception and communication Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
11
Dealing with uncertainty
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
12
Dealing with uncertainty
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
13
Supervision: dealing with real time Supervision
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
14
Multi-sensor fusion: • To combine properly data from multiple sensors • Deal with the problem of track spatial and temporal correlation
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
[H. Li, et al, 2013] Track-to-Track Fusion Using Split Covariance Intersection Filter-Information Matrix Filter (SCIF-IMF) for Vehicle Surrounding Environment Perception Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
16
Perception with Transferable Belief Model
[Trehard 2014] Credibilist simultaneous localization and mapping (C-SLAM) with a lidar Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
17
Dealing with uncertainty and building generic architectures SLAM
Low level High Level
Landmark queue Drift handling
Drift estimation Absolute information (GPS, V2I…)
Loop EKF SLAM [Bresson 2013] , A General Consistent Decentralized SLAM Solution
Information from another vehicle (V2V)
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
18
Increasing perception using communications Decorrelated maps Send
v b1 l1
Decentralized map
1
Pv Pb
vu1
Pl
b1
Receive
v
2
Pv
b1 l1
lu1 vu2
Pb
Pl
b2
Pvu1
Pb1 Pb 1
Pau1 Plu1 Pvu2 Pvu2 Pb2 Pb2
vu3
Receive
v b1
3
Pv
Pvu3 Pvu3
Pb 3
b3
Pb
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
19
Limit the combinatorial explosion: SLAM stretching compacted grid map:
• Dedicated to “open” areas • Load in memory only the local neighborhood map slots, since the rest of the map is saved on the hard disc • a coding technique to compact and save the old or non-used far slots
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
20
SLAM stretching compacted grid map
Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
30/06/2014
21
Conclusion • The main issues related to autonomous vehicles can be summarized like this: • •
• •
Deal with integration problems Using redundancy and complementary information to achieve very precise state estimation Using the fact that a vehicle should know that it does not know Imitate human behavior for decision process
More experiment results on: https://team.inria.fr/rits/ Thank you for your attention! Intelligent vehicles: integration and issues – Equipe-projet RITS – E. Pollard
