DEEP LEARNING BASED SPEECH BEAMFORMING: Code and Audio Demo
Kaizhi Qian*, Yang Zhang*, Shiyu Chang, Xuesong Yang, Dinei Florencio, Mark Hasegawa-Johnson
Code Configurations ResultsCode
The DeepBeam code is available here.
For more details regarding the DeepBeam algorithm, please refer to our paper.
Configurations
To verify DeepBeam works in the intended scenario, we recorded a realistic dataset. The data were collected with eight different microphones. These mics were casually placed on the table of a conference room. There are two speakers, reading My Grandfather and The Rainbow respectively.
Here are samples of pure speech of speaker 1 recorded by each microphone:
| Mic ID | Description | Sample |
|---|---|---|
| 1 | Wireless electret, hissing | |
| 2 | Wireless electret | |
| 3 | Wireless electret | |
| 4 | Wireless electret | |
| 5 | Wired electret, closest to speaker 2 | |
| 6 | Wired electret, closest to speaker 1 | |
| 7 | Wired electret | |
| 8 | Wired dynamic, placed by a noisy fan |
Five different types of noise were recorded separately. Each was then mixed with the speech such that the SNR of the closest channel is 10dB.
Here are samples of each pure noise recorded by mic 3:
| Noise ID | Description | Sample |
|---|---|---|
| 1 | Cell phone | |
| 2 | Paper binding machine | |
| 3 | Paper shuffle | |
| 4 | Door slide & bang | |
| 5 | Footstep |
Back to Top Back to Section Start
Results
Below are the audios processed by the five difference algorithms.
- DeepBeam - the proposed deep learning based speech beamforming
- GRAB - a beamforming tecnique guided by glottal residual fitting
- CLOSEST - the closest mic strategy
- IVA - independent vector analysis with Laplacian prior
- MVDR - time-domain MVDR
These audios were part of our subject evaluation on the Amazon Mechanical Turk.
| Noise Type | Speaker | Results | |
|---|---|---|---|
| Cell Phone | 1 | DeepBeam | |
| GRAB | |||
| CLOSEST | |||
| IVA | |||
| MVDR | |||
| 2 | DeepBeam | ||
| GRAB | |||
| CLOSEST | |||
| IVA | |||
| MVDR | |||
| Paper binding machine | 1 | DeepBeam | |
| GRAB | |||
| CLOSEST | |||
| IVA | |||
| MVDR | |||
| 2 | DeepBeam | ||
| GRAB | |||
| CLOSEST | |||
| IVA | |||
| MVDR | |||
| Paper shuffle | 1 | DeepBeam | |
| GRAB | |||
| CLOSEST | |||
| IVA | |||
| MVDR | |||
| 2 | DeepBeam | GRAB | |
| CLOSEST | |||
| IVA | |||
| MVDR | |||
| Door slide & bang | 1 | DeepBeam | |
| GRAB | |||
| CLOSEST | |||
| IVA | |||
| MVDR | |||
| 2 | DeepBeam | ||
| GRAB | |||
| CLOSEST | |||
| IVA | |||
| MVDR | |||
| Footstep | 1 | DeepBeam | |
| GRAB | |||
| CLOSEST | |||
| IVA | |||
| MVDR | |||
| 2 | DeepBeam | ||
| GRAB | |||
| CLOSEST | |||
| IVA | |||
| MVDR |