Polytechnic University
Deconfliction and Collision Avoidance Algorithms for Unmanned Systems
Farshad Khorrami Polytechnic University, Brooklyn NY 11201, USA
Challenges: Autopilot Design + Obstacle Avoidance + Resource Allocation Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Goals
• Act autonomously • Cooperate as team of agents • Integrate into shared space with manned vehicles hi l – Need to obey regulations • FAA regulations for UAVs in national airspace • NAV rules for USSVs/ UUVs • Traffic rules for UGVs
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Application Scenarios • Completely Centralized o Cooperating agents o Central challenges: resource allocation, (offline) path planning • Partly Decentralized o Initial centralized plan is tuned during execution o Additional challenges: online resource allocation and path planning Cooperating agents Full knowledge of environment
Offline resource allocation and path planning could be sufficient
Uncertain environment
Need to sense environment geometry in run-time; but can possibly exchange agent locations and/or partial environment maps with each other
Control/Robotics Research Laboratory
Non-cooperating agents
Need to also sense locations of other agents
Presentation at MAV 08 – Agra, India
Polytechnic University
Sub-problems in Deconfliction and Obstacle Avoidance
•
Resource allocation and mission planning – Agents could be heterogeneous (some UAVs, UAVs some UGVs, UGVs for instance) so that they have varying capabilities – Construction of nominal paths for agents
•
Sensor data processing and fusion – Environment cognition – Feature recognition and obstacle geometry estimation – Situational Si i l awareness
•
Obstacle avoidance – Online recognition of obstacles in the proximity of the agent and trajectory remapping
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Conflict Resolution
• Premise – – – – – – –
Multiple distributed autonomous assets Each asset is capable of making individual decisions Assets are looking to optimize individual efforts Neighboring assets collaborate on resource allocation Operating in same working domain Limited communication bandwidth Decentralized control framework
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Autonomous Decision Making Systems •
Working semi-independently
•
Each individual decision making system looks to optimize individual solutions
•
Limited to no consideration of other assets when forming decisions
•
Two systems forming independent decisions have the potential for conflicting decisions – Both systems perform (or disregard) the same task; degradation of optimality
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Considerations when treating conflicts •
Conflict (assuming two assets: A1 & A2) – Sc = S1∩ S2 ≠ ∅ is the Intersection of Confliction – S1,2 1 2 are defined as the set of task allocations for A1 and A2 respectively
•
Degree of Confliction is a measure of intersecting partial solutions – i.e., S'1 ⊆ S2 or S'2 ⊆ S1 where S'1 ⊂ S1 and S'2 ⊂ S2
•
Communication bandwidth bounds
•
Decentralized control structure – Hierarchical or non-hierarchical non hierarchical (swarm)
•
Individual objectives
•
Cooperative objectives
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Example of Conflict
•
Resource allocation with 2 assets and 6 tasks
•
Individual objectives (Cost Evaluation) – Minimize traveling distance – Minimize fuel consumption
•
Cooperative objective – Market based approach – Barter B t with ith neighboring i hb i assets t tto d determine t i who h will ill perform f what h t ttask k – Cheapest cost wins
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Market Based Conflict Resolution AThey willAcontinue task ANo with to winner with broadcast corresponding bidits costtheir bids cost 1 broadcasts 2 1 responds 2 1 wins 2
Asset 1 (A1)
Data Packets Task: T354(ISR) (SAR) Cost: 876 762 761 628 634 593
Asset 2 (A2)
Δ
S G = Global Task Space
T2
T3 Δ
S = (1) T
A1 Task Space
Control/Robotics Research Laboratory
T5
T6 T1
T4
S
(2) T
Δ
= A2 Task Space
Presentation at MAV 08 – Agra, India
Polytechnic University
Real-World Implementation
•
Many-on-many asset control – Single operators managing multiple assets – Multiple operators managing a single asset – Multiple operators managing multiple assets
•
Required to resolve conflicts between operations and planning
•
This will require teaming of resources where deconfliction is managed by an operator hierarchical structure
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Operator and Mission Deconfliction Air Assets in a Battlespace • Heterogeneous H t assets Q lifi Qualifiable blt • Different capabilities Capabilities • Sea/land/air Adequate Fuel
Littoral ISR - Mission
Availability @t=τ
Sea Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Application Scenarios • Completely Centralized o Cooperating agents o Central challenges: resource allocation, (offline) path planning • Partly Decentralized o Initial centralized plan is tuned during execution o Additional challenges: online resource allocation and path planning Cooperating agents Full knowledge of environment
Offline resource allocation and path planning could be sufficient
Uncertain environment
Need to sense environment geometry in run-time; but can possibly exchange agent locations and/or partial environment maps with each other
Control/Robotics Research Laboratory
Non-cooperating agents
Need to also sense locations of other agents
Presentation at MAV 08 – Agra, India
Polytechnic University
A World of Clutter
•
Lots of objects (weather balloons, balloons other vehicles vehicles, trees trees, buildings buildings, antennas antennas, wires, etc.)
•
Need sensors to detect objects – need to address SWaP (size, weight, and power) constraints – need customized solutions for each type (and form factor) of vehicle
•
Sensor processing and sensor fusion (again, need to keep SWaP in mind)
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Sensors/ Transponders •
Applicable sensor types include – Radar, Synthetic Aperture Radar (SAR) – LIDAR/laser range finder – infrared/visible camera – acoustic sensor
•
In the cooperative agent case, transponders such as ADS-B can be utilized.
•
Suitable choice of sensors depends on particular vehicle type and particular application.
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Sensors/ Transponders UWB Radar, Synthetic Aperture Radar (SAR)
MSSI RaDeKL
ImSAR/Insitu NanoSAR
Sandia MiniSAR
Sample MiniSAR Image Sonars For Marine Applications
Acoustic
Imagenex Yellowfin
Parallax PING)))
SoundMetrics DIDSON
SARA PANCAS
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Sensors/ Transponders (Contd.) Visible/ Infrared
Thermoteknix MIRICLE 307K
Marshall Electronics
STMicroelectronics Single-chip VS6724
Centeye Ladybug optic flow sensor
Paravize Head-On Head On by Foster Flight, Inc.
LIDAR/ Laser range scanner
URG-X002 laser range scanner by Hokuyo Automatic
Laser Optronix 3D LIDAR
Control/Robotics Research Laboratory
Sample LIDAR image
Presentation at MAV 08 – Agra, India
Polytechnic University
Pros/Cons of Sensor and Transducer Types Sensor/Transponder Type ADS-B
Passive acoustic
1D laser range scanner
LIDAR
Advantages
Weaknesses
Light-weight; low power; long-range; provides reliable all-weather operation Light-weight; low-cost; does not rely on line of sight; provides 360 degree field of view; directly outputs estimates of target locations Light-weight; directly outputs range information f to obstacles Directly outputs range information to obstacles
Currently deployed on only a small minority of aircraft in operation
Visible/Infrared camera
Light-weight; large range
UWB Radar and SAR
Large range
Control/Robotics Research Laboratory
Cannot detect aircraft with low noise emission
Scans only along 1 dimension and within some predetermined field of view Heavier; needs more power; could interfere with the operation of approaching aircraft (especially ones with human pilots) Needs image processing to estimate obstacle geometry; affected by weather conditions Typically heavier Presentation at MAV 08 – Agra, India
Polytechnic University
Obstacle Avoidance System
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Approaches to Obstacle Avoidance • • • • • •
Potential Fields (PF) Deformable Virtual Zones (DVZ) Dynamic Programming (e.g., A-Star) Histogram Grid Method (HGM) Graph Theoretic Methods (GTM) Rule-Based Approaches (RBA)
GODZILA • Computationally efficient obstacle avoidance algorithm specially designed to fit within SWaP constraints of micro unmanned vehicles • Does not require any a priori information about the environment and does not rely on building an obstacle map • Works in any finite-dimensional space with provable convergence with probability 1 to target
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
GODZILA LAP Environment (ue)
Sensors
s i = inf {d ∈ [ 0 , ∞ ) : x p + d ( x h + q i ) ∈ u e } s = ( s1 , s 2 , " , s n ) q i : unit - vector defining direction of i th sensor
Sensor Measurements (s) Algorithm
x p : current p position x h : current heading
Heading (yh) Vehicle Kinematics
Position (xp), Heading (xh)
Enforcing a clearance zone: s 'i = si − pi e − si Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
GODZILA Optimization Component
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
GODZILA Optimization Component (Contd.)
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
GODZILA Optimization Component (Contd.)
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
GODZILA Optimization Cost (Contd.)
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
GODZILA Optimization Cost: Special Case
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
GODZILA: Linear Velocity
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
GODZILA: Approaching a visible target and Random Navigation g
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
GODZILA: Sketch of Convergence Proof
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Performance of GODZILA in 3-D
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Performance of GODZILA in 2-D
50
0
0
y
y
50
−50
−50
−100 −80
−60
−40
−20
0 x
Control/Robotics Research Laboratory
20
40
60
80
−100 −80
−60
−40
−20
0 x
20
40
60
Presentation at MAV 08 – Agra, India
80
Polytechnic University
Performance of GODZILA in 2-D 50
y
0
−50
−100 −80
−60
Control/Robotics Research Laboratory
−40
−20
0 x
20
40
60
80
Presentation at MAV 08 – Agra, India
Polytechnic University
Performance of GODZILA in 3-D
50
40 20 z
0 −20 −40
0
−80
y
−60 −40 −20
−50
0 20 40
50 0
60 x
−50 80
−100
−100 −80
−60
−40
−20
0 x
20
40
60
y
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
80
Polytechnic University
Performance of GODZILA in 3-D 40
20
z
0
−20
−40
80
−80
60
40
−60
40
−40
20
20
−20
z
0
0
0
−20
20
−20
−40
40
−40
60 x
−60 50
80 −100
−50
0
50
0
−50
−80 −100
y
y
50
y
0
−50
−100 −80
Control/Robotics Research Laboratory
−60
−40
−20
0 x
20
40
60
80
Presentation at MAV 08 – Agra, India
x
Polytechnic University
Applications
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Development of GODZILA OAS • • • • • • •
Autonomous Helicopter for Urban Warfare Developed under US Army Research Office Backpackable Reconnaissance Platform for Urban Environments U Useable bl by b non-pilot il t skilled kill d operators t Ground Station Commands vehicle to fly to GPS target Transmits back Live Video Navigates Autonomously in Cluttered Environments with GODZILA OAS
UAV
CCD IMAGE
GROUND STATION
GPS CPU Wireless GODZILA Modem OAS
Control/Robotics Research Laboratory
IMU
Laser
Presentation at MAV 08 – Agra, India
Polytechnic University
Auto-Pilot and Miniature IMU
Miniature IMU
Single-Processor Auto-Pilot Board Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Dual Processor Auto-pilot Board Piezo Gyros
GPS
Accelerometers
Pressure Sensors
Control/Robotics Research Laboratory
FPGA Processors
CPLD
Presentation at MAV 08 – Agra, India
Polytechnic University
Helicopter Hardware-In-the-Loop Simulator
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
UUV Application •
•
•
Develop OAS algorithms suitable for mine locating and tagging. – Allow UUV to detect and avoid underwater obstacles and perform mine countermeasures mission. Modeling and Simulation – Formulate math models of vehicle dynamics and sonar – Characterize response of sonar with underwater testing – Adapt p GODZILA Algorithms, g , tailor to UUV and sonar – Simulate in high-fidelity environment as proof of concept Sea Trials h(cos(θ − θ k )) s ' k +ε k =1 n
s d (θ ) = ∑
GODZILA Algorithms Test Data
Matlab Sims
High Fidelity Sims Sea Testing
Sonar Characterization Control/Robotics Research Laboratory
Avionics Development Presentation at MAV 08 – Agra, India
Polytechnic University
Vector P UAV • • • • • • • •
IntelliTech Microsystems has developed the Vector P . 101” Wingspan, 47 lb dry weight. GPS Guidance All composite construction 2 HP Gasoline Engine 25 lb Payload p dash, 70 mph p cruise, 10,000 ft. ceiling g 90 mph 4 Hr + Endurance
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
IntelliTech’s Hand-Launchable Electric UAV • 7 1b (3.2 Kg) • Electric • Hand Launched • 90+ Endurance
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
Unmanned Sea Surface Vehicles Maritime Seaway Navigation System (MSNS) = Hi h P f High-Performance Stabilization St bili ti and d Tracking T ki C Control t lS System t for High Sea States (USSV-NAV) + Ob t l Detection Obstacle D t ti and d Collision C lli i A Avoidance id (GODZILA)
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
MSNS Visualization – USSV-NAV
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
MSNS Visualization – Right of Way -- GODZILA
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
Polytechnic University
MSNS Visualization -- GODZILA
Control/Robotics Research Laboratory
Presentation at MAV 08 – Agra, India
