evolving views on HOA: from technological to pragmatic concerns Jérôme Daniel Orange Labs Ambisonics Symposium, Graz, 2009/06/25
r&d legal direction
outline 1 introduction (chronology) 2 main concepts and promises 3 focus on: 5.0 decoding and HOA microphone array 4 HOA tools and integration (plugins) 5 format, standardization, coding 6 HOA in "real life": learning from recording / production experiments
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 2
r&d legal direction
France Telecom Group
introduction / earlier study and motivations
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 3
r&d legal direction
France Telecom Group
Ambisonics and HOA: a chronology of space expansion… [Gerzon] [Gerzon, Craven,…] [Malham]
70's
90's
[Bamford] [Poletti] [Daniel] [Nicol] [Sontacchi…]
96-00
Ambisonics Symposium…
MPEG AudioBIFS
00-03
03-06
[Laborie et al]
SoundField SRP (Trinnov)
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 4
[Fazzi] [Solvang] [Adriensen] [IEM…]
[Daniel, Nicol, Moreau, Bertet]
r&d legal direction
07/08
09…
Head-tracked binaural Plugins VST, etc.
EigenMike (mh-acoustics) France Telecom Group
higher order ambisonics in short
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 5
r&d legal direction
France Telecom Group
Higher Order Ambisonics (HOA)
increase angular discrimination thanks to additional encoding directivities
cos θ
cos 2θ
spatial encoding ≈ circular Fourier Transform spatial spectrum = {ambisonic components} spatial bandwidth = highest angular frequency
cos 3θ
cos 4θ
enriched spatial bandwidth 0th order
1st order
sin θ Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 6
2nd order
sin 2θ
3rd order
4th order
sin 3θ
sin 4θ
r&d legal direction
France Telecom Group
Higher Order Ambisonics (HOA) Front (X)
increase angular discrimination thanks to additional encoding directivities
Left (Y)
Right
spatial encoding ≈ circular Fourier Transform spatial spectrum = {ambisonic components} spatial bandwidth = highest angular frequency
enhance spatial separation to feed loudspeakers more selectively
synthesize directivities with enhanced beamwidth spatial decoding ≈ multi-directional beamforming
Back
+
+ =
+ =
+ =
= enhanced beamwidth
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 7
r&d legal direction
France Telecom Group
Higher Order Ambisonics (HOA)
increase angular discrimination thanks to additional encoding directivities
enhance spatial separation to feed loudspeakers more selectively
1st order
2nd order
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 8
spatial encoding ≈ circular Fourier Transform spatial spectrum = {ambisonic components} spatial bandwidth = highest angular frequency
synthesize directivities with enhanced beamwidth spatial decoding ≈ multi-directional beamforming ≈ inverse discrete circular Fourier Transform more accurate sound images (reduced spread angle)
3rd order
r&d legal direction
4th order
France Telecom Group
HOA as a "holophonic" approach NFC-HOA [Daniel] filter implementation improvement [Adriaensen] More workable scheme for close sources: High-passed (NFC)-HOA [Daniel, Moreau] Further connections with WFS [Nicol, Daniel] [Fazzi], …
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 9
r&d legal direction
France Telecom Group
HOA claims and promises
claims
format flexibility • (wrt reproduction setups, spatial manipulation, scalability)
spatial "objectivity" and predictability • representation Æ reproduction ⇒ spatial fidelity and transparency?
high res, "true 3D" recording technology
promises for many application contexts
a format for new 3D audio content generation and consumption!? • one generic content for various terminals, transport constraints, consumption styles
immersive telecommunication: teleconferencing, ambience sharing interactive 3D navigation, games…
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 10
r&d legal direction
France Telecom Group
A way to turn dreams into reality
assess or criticize HOA claims to improve the techno
consider format standardization
test out HOA with "real life" concerns
(also matter of maturity)
current format and equipment standards (5.0), current practices get lessons…
bring HOA techno into the hands of content creators / sound engineers
make advertising, adapt practices, facilitate the convergence between research and production worlds
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 11
r&d legal direction
France Telecom Group
focus on: decoding for standard 5.0 setup spherical HOA microphones
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 12
r&d legal direction
France Telecom Group
HOA decoding for standard 5.0 setup (+8.0)
energy vector optimized
Craven [AES24]
energy vector optimized
+ = target, ie ideal sound image) * = "energy vector" (HF prediction) □ = "velocity vector" (LF prediction) = physical limit for energy vector(pair-wise pan-pot)
concerns: image stability, consistency, homogeneity, discreteness…
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 13
r&d legal direction
France Telecom Group
HOA decoding for standard 5.0 setup (+8.0) Mainly these two decoders were involved in recording+production experiments reported later Many other potential decoders, by combining criteria in different ways
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 14
r&d legal direction
France Telecom Group
HOA sphere microphone: basics
Q sensors on the sphere
processing = matrix + EQ
Q HOA signals
EQ: • theoretically -mx6dB / oct ! • one must limit bass-boost
0 1 2 3 4
180 160
rough( r / c ) B00+1
EQ0( r / c , R / c )
rough( r / c ) B11+1
EQ1( r / c , R / c )
NFC( R / c ) B00+1 NFC( R / c ) B11+1
EQ1( r / c , R / c )
NFC( R / c ) B11−1
EQ1( r / c , R / c )
NFC( R / c ) B10+1
rough ( r / c ) B11−1
a = 5 cm 200
rough( r / c ) B10+1
# 140
Matrix
#
120 Amplitude (dB)
Æ sound field sampling Q=32 Æ 4th order, K=25 HOA components
#
100
NxK
#
+1 rough( r / c ) B mm
EQm( r / c , R / c )
+1 NFC( R / c ) Bmm
−1 rough( r / c ) B mm
EQm( r / c , R / c )
−1 NFC( R / c ) Bmm
#
80 60
B mn
40
#
σ rough ( r / c )
rough ( r / c ) B m+10
20
#
0 2
10
3
10 Fréquence (Hz)
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 15
#
# (r / c,R / c) m
EQ
# EQm( r / c , R / c )
#
# σ NFC( R / c ) Bmn #
NFC( R / c ) Bm+10
#
4
10
r&d legal direction
France Telecom Group
HOA sphere mic: limits and tradeoff correct estimation
spatial aliasing 10 0
Order
2
-10 -20
3
4
reduced spatial bandwidth
-30 -40
(dB)
estimation error
0 1
∅7cm, 32 sensors Æ25 components (4th order)
-50
10
2
3
10 Frequency [Hz]
10
4
-60
shift towards LF when radius increases shift towards HF when radius decreases
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 16
r&d legal direction
France Telecom Group
spatial aliasing
rigid sphere arrays: trade-offs (for a given radius)
60 50
increase EQ max level…
etc.
40
bandwidth enlarges towards LF, but… less benefits for higher orders and increase of noise level
(dB)
+18dB
+15dB SNR
30 20
3 oct
2
10
increase the number of sensors Q…
1 oct
10 0
1,5 oct
3
10 Frequency (Hz)
1 oct4 10
Qx4
4 x more sensors to gain 1 octave towards HF SNR and robustness improvment: +10*log10(Q) dB => e.g. 15dB for Q = 32 sensors
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 17
r&d legal direction
France Telecom Group
HOA microphones built and experimented FTR&D
(DPA4060)
(Panasonic 2€)
32 caps Æ 4th order 12 caps Æ 2th order 20 caps Æ 3th order
(Sennheiser mke4)
EigenMike™ (mh-acoustics)
⇒ great improvement in terms of usability!
32 caps Æ 4th order
alternative [Epain & Daniel]
8 caps Æ 3th order, 2D reduced spatial aliasing
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 18
r&d legal direction
France Telecom Group
Impact of imperfect encoding For instance: on Craven, 4th order 5.0 decoding Craven o4 /o4
4th order
Craven o4 /o3
Craven o4 /o2
2nd order truncation
3rd order truncation
Craven o4 /o1
1st order truncation
to lower frequencies
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 19
r&d legal direction
France Telecom Group
HOA tools, integration and demonstrators
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 20
r&d legal direction
France Telecom Group
HOA VST plugins (Orange Labs)
HOAEncoder HOAMicProcessor HOARotator HOASpkDecoder HOABinDecoder
Tested host applications
PlogueBidule, (MaxMSP) (Cubase, Nuendo), Podium
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 21
r&d legal direction
France Telecom Group
format, standardization, compression
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 22
r&d legal direction
France Telecom Group
HOA format: which need for standardization? for interoperability between HOA processing units for sharing HOA sound files
for insertion in 3D interactive multimedia contents
no or few concern with compression ⇒ associated data, extended file header, new 'trunk' ⇒ eg MPEG4 (cf AudioBIFS V3 norm)
for generalized consumption: broadcast, exchange
high concern with compression issues extended issues: format conversion, spatial audio object coding
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 23
r&d legal direction
France Telecom Group
HOA in "real life" (incl. mass market concerns) learning from experimental and collaborative recording opportunities
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 24
r&d legal direction
France Telecom Group
Various recording conditions and constraints immersive ambience music / theatre performance (often codified spatial organisation⇒cultural expectations) without or with video or even stereoscopic video
⇒ mic positioning constraints; ⇒ sound and visual image coherency issues
with or without spot/close microphone mixing direct mixing of post-produced with concurrent mutlichannel system (trees) Æ EWO in collaboration with professional sound engineers Æ RadioFrance
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 25
r&d legal direction
France Telecom Group
Fully immersive "nature ambience"
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 26
r&d legal direction
France Telecom Group
classic organization: frontal scene
front loudspeakers provide the orchestra image (quite dry) rear loudspeakers for field reverberated by the room 32 DPA4060, 4th order
Orchestre National de France, Studio 104, Radio-France, Juin 2008
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 27
r&d legal direction
France Telecom Group
very large front scene Workshop Ears Wide Open, Le Tambour, Rennes, Mars 2008 multi-microphone trees
HOA sphere 20 DPA, order 3
large scene Æ more or less contributions from rear loudspeakers
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 28
r&d legal direction
France Telecom Group
panoramic scene SoundPainting session, Chapelle des Ursulines, Lannion, November 2008 EigenMike, ordre 4
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 29
r&d legal direction
France Telecom Group
Opéra de Rennes: La Trahison Orale
"with height" spatial organisation
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 30
r&d legal direction
France Telecom Group
tuba
front wall
Spatial configuration: "La Trahison Orale"
balcony
stage
orchestra difficult trade-off between sound pit
nce) ie d u a ( s ll a t s
balance and spatial readibility 3DÆ2D projection issues depending on the microphone positioning and "pointing direction" Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 31
r&d legal direction
France Telecom Group
Don Giovanni, Opéra de Rennes (2 June 2009) 3D
video and audio capture direct satellite broadcast to different places •
+ Mezzo TV in SD (in 39 countries)
audio
part: Orange Labs + Radio-France
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 32
r&d legal direction
France Telecom Group
Don Giovanni
Technical aspects
HOA sphere positioning trials Æ 5.0 provided to Radio-France real time mixing with 4 dozens of spot microphones artificial reverb added (even on "ambience HOA 5.0") ⇒ no longer "HOA" spatial model
Results
great success (artistic, technical, popular) very nice and appreciated sound ! … but not really faithful wrt the actual theatre ;-) great communication impact for HOA ! • (websearch: HOA + "Don Giovanni")
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 33
r&d legal direction
France Telecom Group
Some lessons
Mistrust monitoring over another setup:
eg 8.0 vs 5.0 ⇒ front-back instability + influence of the reproduction room (and ldspk, and array size), esp. with particular sounds (applauses, broadband signals, etc.) or recorded scene (very large, etc.) : dry, spherical vs hemi-spherical, spherical vs horizontal ⇒ projection of elevated source binaural vs loudspeaker presentation ⇒ wave interferences, potential coloration and phasing
Shall content creation and recording be generic, or specific to a target setup? Use HOA as end-to-end approach or intermediary toolset?
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 34
r&d legal direction
France Telecom Group
Conclusion
further improvements of the technologic part still expected
system transparency (spatial decoding) 3D scene analysis and manipulation: spatial editing tools (benefiting from new ext. dev.: mic array, et. ) coding / compression, format conversion
get lessons from experiments
adapt HOA to sound engineers practice… then reciprocally! • improve ergonomics • HOA as end-to-end techno and format? Or as a toolkit?
issues regarding: • spatial organisation, • mic positioning, etc. • generic of specific content production
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 35
r&d legal direction
France Telecom Group
Thank you for your attention
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 36
r&d legal direction
France Telecom Group
