Back to list

Multi-Perspective Scene Analysis from Tetrahedral Microphone Recordings

Authors Blochberger, M.
Year 2020
Thesis Type Master's thesis
Topic Spatial Audio
Keywords audio recording and reproduction, B-format, Higher-Order Ambisonics (HOA), signal processing, Spatial Audio
Abstract Convincing immersion in virtual reality requires to enable the user to engage in interactive listening within three-dimensional audio scenes. To achieve a realistic listening experience, the acoustic perspective and orientation has to be real-time controlled with the own body movements. This thesis addresses the task of presenting an interpolated variable perspective to an interacting listener, while the original audio scene is recorded simultaneously at only a few static perspectives. The scene is decomposed into localizable sound objects and a residual signal for the variable-perspective interpolation. Information regarding localizable objects is extracted from a probability map that is composed from the directions detected by the collective of the available single perspectives. This work proposes a particle-filter-based approach for a continuous position estimation of sound objects. The particle filter uses the probability map to estimate a continuous trajectory for each sound object in the scene. The rendering approach extracts signals from the recording for each localized sound object according to its estimated trajectory and embeds it relative to the virtual listener into the residual signal.
Supervisors Zotter, F., Höldrich, R.