Pascal institute
Structuration
ISPR/ComSee
Computers that see
2012
Institut Pascal
Content
ISPR/ComSee: Relevant Keywords
•
• • •
Relevant(keywords(about(ISPR/ComSee(
• Topic&1(:(Threedimensional(Reconstruc=on(of(Rigid( Scenes(and(VisionABased(Metrology(
ComSee&members&(Jun.&2013):((36)(
•
•
•
5(associate(professors( 20(Interna=onal(journals(ar=cles(( 96(ar=cles(in(Interna=onal(Conferences((
Publica2012):&
• • • Institut Pascal
Non(permanent(members:(16(Phd,(5(PostDoc,(1(Engineers(
Publica2010):&
• •
• Topic&2(:(Visual(Iden=fica=on(and(Tracking(
Permanent(members:(9(researchers(
10(Interna=onal(journals(ar=cles(( 78(ar=cles(in(Interna=onal(Conferences(( Patents,&2007>2010&&(Systems(and(soVwares):(5(
Current&funded&projects:(6 Institut Pascal
Quantitative Imaging
Photonics, Microwave, Nanomaterials
Probabilistic based methods
Process Engineering, Energetics and Biosystems
Multiscale Materials and Model
Computer vISion, Perception systems and Robotics
Innovations for Bioprocess
Teams
Mechanical, Materials and Structures
Intelligent and Innovative Machines and Robots
Disciplinary topics
2
Introduction to Computer Vision 2015
Computer Vision: Definition Computer vision is a field that includes methods for acquiring, processing, analyzing, and understanding images and, in general, highdimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions Processing
Acquiring
High Dimensional data
Analyzing
Understanding T. Chateau
T. Chateau
Computer Vision: Related Fields
Computer Vision: State of the art
Institut Pascal
1950: Image processing 1960: first image processing algorithms
binary and edge based technics
(from Wiki) T. Chateau
Institut Pascal
T. Chateau
Institut Pascal
Computer Vision: State of the art
Computer Vision: State of the art
1990: projective geometry, a major contribution to Computer Vision
T. Chateau
>2000: toward robust and realtime algorithms
Institut Pascal
Computer Vision: State of the art
T. Chateau
ComSee, Pascal Institute, 2005
Institut Pascal
Why Computer Vision is Challenging ?
>2000: Learning based methods
• what(about(the(animal(vision(system(?( • perspec=ve(projec=on((from(3D(to(2D)( • object(reflectance( • visual(features(
T. Chateau
Deep learning Institut Pascal
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
Camera Models
what(about(the(animal(vision(system(?
Pinhole(model
Image plane Pinhole
T. Chateau
prey:(360°(field(of(view
predator:(stereoscopic( vision Institut Pascal
Why Computer Vision is Challenging ?
Virtual image 14
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
perspec=ve(projec=on((from(3D(to(2D)(
Scene(rendering(:(the(direct(problem( Produce(an(image(from(parameters
Wikipedia T. Chateau
Institut Pascal
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
Why Computer Vision is Challenging ?
Computer(Vision:(the(inverse(problem(
exemples(of(virtual(images
Es=mate(parameters(from(images
Wikipedia T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
visual(features:( edges
To(solve(the(inverse(problem,(we(need( to(extract(image(features
Wikipedia T. Chateau
Institut Pascal
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
Why Computer Vision is Challenging ?
visual(features:( shadow
visual(features:( edges
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
Relevant(human(features(for(s=ll( images
visual(features:( texture
- Edges - Shadow - Texture and color
T. Chateau
Institut Pascal
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
Why Computer Vision is Challenging ?
Most(popular(geometric(features(used(in(Computer( Vision
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
Edges
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
Interest(Points
But:&should&we&rely&on&human&vision?
T. Chateau
Institut Pascal
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
Why Computer Vision is Challenging ?
Should(we(rely(to(the(human(vision(?
Should(we(rely(to(the(human(vision(? which(is(the(longest(s=ck(? ((which(is(the(longest(table(?
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
T. Chateau
T. Chateau
Institut Pascal
Why Computer Vision is Challenging ?
Should(we(rely(to(the(human(vision(?
Should(we(rely(to(the(human(vision(?
((which(is(the(darkest(square(between(A(and(B?
((which(is(the(darkest(square(between(A(and(B?
Institut Pascal
T. Chateau
Institut Pascal
Typical tasks in Computer Vision
Typical tasks in Computer Vision
Image(Processing/Analysis
• Image(Processing( • Object(detec=on(and(tracking( • 2D(geometry(( • 3D(geometry( for(medical(applica=ons
T. Chateau
Institut Pascal
Typical tasks in Computer Vision
T. Chateau
Typical tasks in Computer Vision
Image(Processing/Analysis
Image(Processing/Analysis
original
denoising
T. Chateau
Institut Pascal
mask(image
result(
Inpain=ng
Institut Pascal
T. Chateau
Institut Pascal
Typical tasks in Computer Vision
Typical tasks in Computer Vision
Object(detec=on(and(tracking
Object(detec=on(and(tracking
pedestrian(detec=on
face(detec=on T. Chateau
Institut Pascal
Typical tasks in Computer Vision
T. Chateau
Typical tasks in Computer Vision
Object(detec=on(and(tracking
Object(detec=on(and(tracking
MOT:(Mul=(objects(tracking
object(detec=on(and(tracking
object(tracking:(meanshiV T. Chateau
Institut Pascal
Institut Pascal
T. Chateau
ComSee, Pascal Institute, 2009
Institut Pascal
Typical tasks in Computer Vision
Typical tasks in Computer Vision
2D(geometry
2D(geometry:(2D(pa^ern(tracking
V.(Lepe=t,(EPFL,(Lausanne,(2006
Interest(point(matching
T. Chateau
Institut Pascal
Typical tasks in Computer Vision
T. Chateau
Typical tasks in Computer Vision
2D(geometry:(video(stabiliza=on
before
2D(geometry:(augmented(reality
aVer
Pascal(Ins=tute,(France,(2006 T. Chateau
Institut Pascal
ComSee,(Pascal(Ins=tute,(France,(2011 Institut Pascal
T. Chateau
Institut Pascal
Typical tasks in Computer Vision
Typical tasks in Computer Vision
3D(geometry:(3D(reconstruc=on
3D(geometry 2D interest points 3D model
2D/3D optical center
2D
matching
Image
T. Chateau
M.(Lhuillier,(2002 Institut Pascal
Typical tasks in Computer Vision
T. Chateau
Typical tasks in Computer Vision
3D(geometry:(3D(reconstruc=on
3D(geometry:(3D(reconstruc=on
ComSee,(Pascal(Ins=tute,(2008 T. Chateau
Institut Pascal
Institut Pascal
T. Chateau
ComSee(Pascal(Ins=tute/Cea,(France,(2005 Institut Pascal
Visual Memory: Matching
ISPR/ComSee: 3D-Localisation
Monocular&based&localisa N
Interest point detection
10
N=300, M=400 11
Visual Memory: 3D textured points
Visual Memory: 3D points
Step&2:(3D(reconstruc=on
Step&2:(3D(reconstruc=on
10 m
3D reconstructed points (125 m, 172 key images, 23000 3D image)
(125 m, 172 key images, 23000 3D points) 18
Visual based localization
19
Processus Innovant : Guidage Temps Réel vidéo
Step&3:(real=me(online(localiza=on Visual memory 3D patchs
2D/3D matching and localization
Current image
Camera localization 6 dof
Realtime localization (15 fps) – precision: 10cm Eric Royer, Maxime Lhuillier, Michel Dhome and Thierry Chateau, Localization in urban environments : monocular vision compared to a differential gps sensor. IEEE CVPR2005, Computer Vision and Pattern Recognition. San Diego, USA, June 2005 20
21
Expérimentation en milieu urbain : Place de jaude
ISPR/ComSee: 3D-Localisation
Monocular&based&localisa
