Selfsupervised Learning From High Dimensional Data for Autonomous OffRoad Driving
Ayse Naz Erkan1 Raia Hadsell1
Marc”Aurelio Ranzato1 Urs Muller2
Pierre Sermanet1,2 Yann LeCun1 Koray Kavukcuoglu1
(1) Courant Institute of Mathematical Sciences, New York University (2) NetScale Technologies, Morganville, NJ
Problem: Autonomous, Vision-based Navigation in Complex Off-Road Environments Stereobased navigation uses simple heuristics to identify pixels as ground or obstacle. Stereo is insufficient: ● sparse, noisy, and shortrange (012 meters) ● pure stereo navigation is myopic – driving in fog
The Platform: LAGR Mobile Robot
●
Challenge: Visionbased Navigation for Mobile Robots Why is it hard? Extreme environmental variability Visual complexity – shadows, clutter Hilly, bumpy, uneven terrain Realtime constraints on processing Tricks – collapsible vegetation, hidden obstacles Position estimation errors – wheel slip, GPS Planning with uncertainty Lighting variability – glare, time of day
●
Challenges for machine learning solutions: ● supervised learning limits the variability of environments ● online learning is adaptive, but has no memory ● large image patches are necessary for accurate learning high dimension ● generalization from nearrange to farrange (inverse size/distance) ● planning with uncertainty from classifiers ● concept drift
LAGR (Learning Applied to Ground Robots)
DARPA program 20052008, 8 competing research labs develop navigation for fixed platform
Periodic testing in unfamiliar terrain
CMU & NREC designed platform and baseline software: 4 color cameras (2 stereo pairs, 640x480) GPS receiver for global navigation 2 front bumper switches Onboard IMU (inertial measurement unit) 4 onboard Linux computers 2 “eye” machines (dual core 2 Gz) 1 “planning” machine (single core 1 Gz) 1 lowlevel control computer (single core)
The Solution: Online Self-Supervised Learning
Strategy: Online NeartoFar Learning Inputs: large windows in image Labels: heuristics from stereo module Classifier: unsupervised autoencoder + online logistic regression
input image
stereo labels (012 m)
classifier prediction (580 m)
Stereobased obstacle detector
i
y=gW ' D
●
W'D
∥Y ' −F dec Z ' ∥2
2
∥Y ' −F dec Z ' ∥
Robust feature extraction Trained offline 100000 training images from log files
Y'=Z
F ' dec Z ' Z'
Y ' =Z
Kernels (2 layers) learned by Auto-Encoder: 20x7x6 in first layer; 300x6x5 in second layer
D
D=F W X
Loss:
samples labels
n
L=−∑ log g y⋅W ' D− RW
●
Learning:
●
Inference: where:
i=1
∂L =y⋅g−y⋅W ' D D ∂W y=gW ' D g z=
X (yuv: 13x24x3)
F ' enc Y '
1 1e
Online Ensemble Learning Mixture of Experts Architecture F ' dec Z ' Z'
Input patch
1. . p
dimensional features extracted via unsupervised autoencoder network Weights W are trained with cross entropy loss function Regularization: decay to default weights, L2 regularization
W
2 Layer Auto-Encoder Network
,Y
The online classifier is trained at each frame using gradient descent on the 100
Input image is normalized such that size of an object is independent of its distance from the robot Allows consistent processing of windows at different scales Distance normalization allows learning using large, context-rich windows
X
1. . p
FW X
Online Learning – Logistic Regression with gradient descent
Distance-Normalized Image Pyramid
Autoencoder FW network
Input: calibrated stereo images Output: training set of labeled feature vectors
Architectures that combine highcapacity slow learners and low capacity, highly adaptive controllers could solve the memory problem: a single online classifier exhibits fast learning and fast forgetting. Online mixture of experts is one such architecture.
code
F ' enc Y '
−z
Output ∑ Controller
Expert
Expert
Expert
Input
Results: Evaluation of Learning and Driving Performance ex. A
road following and man-made obstacle detection
Input image
Stereo Labels
Classifier Output
Input image
Stereo Labels
Classifier Output
ex. B
difficult ground recognition multi-color and shadows
Start
No Learning
With Learning
ex. C
very long range vision to the horizon Input image
Stereo Labels
Classifier Output
Direct path to goal ends in culdesac Shortrange stereo (
Editor of the freakonometrics.hypotheses.org's blog. Editor of Computational ...... samples can be decomposed in three parts Hastie et al. (2001). E[(Y â Ìm(x)). 2. ] ...... 4 Information from URL : http://maps.googleapis.com/maps/api/geocode/.
form has a large number of parameters, and a large number of samples is needed to nar$ row the range of probable models. Because the model is complex and ...
Examples. Molecules, annotated sentences, temporal traces of ... Explanation of molecular 3-D shape, new clauses in a ... Examples of Prolog representation.
The proposed algorithm is valid in a more general case espe- ... application in myopic/unsupervised inversion methods as well as ..... Operations Research.
rem 4.3. We distinguish between the values of k: ⢠When k ⤠pγ for some γ < 1/2, if n is large enough to satisfy the assumption of Proposition 4.1, the quantities ...
a general model form with adjustable parameters and learn a posterior probability distribution for ... an algorithm for processing the subsets in stages. For each ...
Learning Complex Classification Models from Large Data Sets ... the large data set into several smaller data subsets and processing the subsets in stages.
be easily implemented for small to moderate M. For larger values of M multiple ... Therefore, via a one-step easily implementable pro- ...... log(Mn)/n, then Mp.
Feb 4, 2015 - the Gaussian comes back with its initial shape to its initial position at integer t, so we can compute .... matter, the distribution function of matter in the phase space (x,v) â R6,. 12 ..... Spherical Stellar Systems, Publ. Astron.
Chief Economists' workshop: what can central bank policymakers learn from other ... for some loss function l. See also Varian (2014). @freakonometrics. 4 ...
âWe make data,â says Pawan Divakarla, data and analytics business leader for. Progressive, the 79-year-old insurance giant. âData is really the bread and butter for us. It's all we do.â In fact, Progressive views its data as a genuine asset.
Sep 16, 2002 - Learning from Argentina's Crisis. Number 2002â31 ... first under a 1998 Extended Fund Facility (EFF) program and then under a 2000 Stand-by Arrange- ment (SBA). ... ily be shifted to earn more exports. Otherwise, the.
on artificial data and on two data sets coming from databases used in visual learning. Key-words: .... whose dimensionrU is much smaller than{I, and secondly by representations based on an analytical ..... lower index becomes zero. A rigorous ...
HIGH CO DATA. 4 0 3 0 9 6 6 7 0 ..... 3. Valeur d'origine des immobi- lisations en fin d'exercice. 4. IMMOBILISATIONS. CO/. LV. MA. MD. MG. MJ. MM. MP. MS.
May 27, 2009 - values ) or genetic variable ... Find boundaries in the image space that best separate ... Limitation : ignores dependences between variables ...
field that simulated a surface corrugated in depth and a flow field-that simulated a random volume. ...... In Experiment 4, we examined surfacedetection, in.
used model for the probability density function (pdf) of the irradiance because ... normal channel with variance Ï2 = 0.1 and zero mean is ... The pdf of log-normal fading channel .... technique is employed at uncoded BER that is less than 10â3.
and action spaces, and a large number of units that have to ... state space in the order of 101887. ... by adopting a particular bootstrap mechanism; and (4).
enhance the data rate of free-space optical communication (FSO) system in log-normal ... used model for the probability density function (pdf) of the irradiance ...
Generally interfaces with data rates of 10 Gb/s to 20 Gb/s seem suitable for the digital cinema field. In this article an interface based on standard 10G Ethernet. 5.
validate a method that efficiently reconstructs such sparse objects in 3D from a ... We then compared the result obtained with these two methods with traditional.
Few errors on the sign of the curvature are found for self-motion for both field ... that the detection of surface curvature from motion is .... on a dark background.
