The Scholarly Store is available here:
Using the power of deep learning, it is now possible to create a technique that looks at a silent video and synthesize appropriate sound effects for it. The usage is at the moment, limited to hitting these objects with a drumstick.
Note: The authors seem to lean on a database of sounds, i.e., the synthesis does not happen from scratch, but they are not merely fetching the database entry for a given sound, but performing example-based synthesis (Section 5.2 in the paper below). In the video and the paper, they both use the words “synthesized sound” and “predicted sound”, and it may be a bit unclear what degree of synthesis qualifies as a “synthesized sound”. I think this is definitely worthy of further scrutiny.
_____________________________________
The paper “Visually Indicated Sounds” is available here:
Recommended for you:
What Do Virtual Objects Sound Like? –
Synthesizing Sound From Collisions –
Reconstructing Sound From Vibrations –
Our deep learning-related videos are available here (if you are looking for convolutional neural networks, recurrent neural networks):
WE WOULD LIKE TO THANK OUR GENEROUS PATREON SUPPORTERS WHO MAKE TWO MINUTE PAPERS POSSIBLE:
David Jaenisch, Sunil Kim, Julian Josephs, Daniel John Benton.
We also thank Experiment for sponsoring our series. –
Subscribe if you would like to see more of these! –
The thumbnail background image was created by slgckgc –
Splash screen/thumbnail design: Felícia Fehér –
Károly Zsolnai-Fehér’s links:
Facebook →
Twitter →
Web →
source