DeepWave: A Recurrent Neural-Network for Real-Time Imaging

Table of Contents

1. Introduction

2. Background

3. Proximal Gradient Descent

4. Methods

1. Introduction

Previous work: Delay-And-Sum (DAS) beamformer [1, Chapter 5]. Idea: Real-time reconstruction of acoustic spherical maps based on LISTA [2]. Limitations: It can reconstruct only high resolution microphone arrays

2. Background

2.1 Steering Matrix

The steering matrix is a matrix that contains the steering vectors of the microphone array. The steering vector is a vector that contains the phase shifts of the microphones in the array.
Signals can arrive to the microphones from different positions and angles. A direction can be parametrized as:

• Azimuth $\theta$: The horizontal angle of arrival.
• Elevation $\varphi$: The vertical angle of arrival.

The vector that stores the phase shifts or time delays of the signal to each microphone in the array is the steering vector. The steering vector is a complex 1D vector of $M$ elements (microphones), $a(\theta )\in {\mathbb{C}}^M$: $$\begin{array}{}\text{(Eq. 1)}& \mathbf{a}(\theta )=\left(\begin{array}{c}1\\ e^{-jkd\sin (\theta )}\\ e^{-j2kd\sin (\theta )}\\ ⋮\\ e^{-j(M-1)kd\sin (\theta )}\end{array}\right)\end{array}$$ where:

• $k=\frac{2\pi }{\lambda }$ is the wavenumber.
• $d$ is the distance between the microphones.
• $\lambda$ is the wavelength.

When we have multiple sources or multiple angles of arrival, we can generalize the steering vector in a steering matrix. Supposing that we have different signals arriving from angles ${\theta }_1,\cdots ,{\theta }_N$, being $N$ the number of sources, the steering matrix $\mathit{A}\in {\mathbb{C}}^{M\times N}$ is defined as: $$\begin{array}{}\text{(Eq. 2)}& \mathit{A}=(a({\theta }_1),...,a({\theta }_N))=\left(\begin{array}{cccc}1& 1& \cdots & 1\\ e^{-jkd\sin ({\theta }_1)}& e^{-jkd\sin ({\theta }_2)}& \cdots & e^{-jkd\sin ({\theta }_N)}\\ e^{-j2kd\sin ({\theta }_1)}& e^{-j2kd\sin ({\theta }_2)}& \cdots & e^{-j2kd\sin ({\theta }_N)}\\ ⋮& ⋮& \ddots & ⋮\\ e^{-j(M-1)kd\sin ({\theta }_1)}& e^{-j(M-1)kd\sin ({\theta }_2)}& \cdots & e^{-j(M-1)kd\sin ({\theta }_N)}\end{array}\right)\end{array}$$

3. Proximal Gradient Descent

DeepWave reconstructs the images with a Recurrent Neural Network trained with Proximal Gradient Descent (PGD) by optimizing: $$\begin{array}{}\text{(Eq. 3)}& \hat{x}=\arg \underset{\mathbf{x}\in {\mathbb{R}}_{+}^N}{min}\frac{1}{2}\Vert \hat{\mathrm{\Sigma }}-\mathbf{A}\text{diag}(\mathbf{x}){\mathbf{A}}^H{\Vert }_F^2+\lambda [\gamma \Vert \mathbf{x}{\Vert }_1+(1-\gamma )\Vert \mathbf{x}{\Vert }_2^2],\end{array}$$ which, after vectorization becomes: $$\begin{array}{}\text{(Eq. 4)}& \hat{x}=\arg \underset{x\in {\mathbb{R}}_{+}^N}{min}\frac{1}{2}\Vert \text{vec}(\hat{\mathrm{\Sigma }})-(\bar{\mathbf{A}}\circ \mathbf{A}){\Vert }_F^2+\lambda [\gamma \Vert \mathbf{x}{\Vert }_1+(1-\gamma )\Vert \mathbf{x}{\Vert }_2^2],\end{array}$$ where:

• $\hat{x}$ is variable being optimized..
• $\hat{\mathrm{\Sigma }}$ is the estimated covariance matrix.
• $\mathit{A}\in {\mathbb{C}}^{M\times N}$ is the steering matrix.
• ${\mathit{A}}^H$ is the Hermitian of the steering matrix.
• $\text{diag}(x)$ is a diagonal matrix with the elements of $x$.
• $\lambda$ is the regularization parameter.
• $\gamma$ is the regularization parameter, the trade-off parameter between the ${\ell }_1$ and ${\ell }_2$ norms.
• $\bar{\mathit{A}}$ is the conjugate of the steering matrix.
• $\circ$ is the Hadamard product.
• $\Vert \cdot {\Vert }_F$: The Frobenius norm, which measures the difference between the target matrix and the approximation.

Proximal Gradient Descend (PGD) allows to approximate the expressions above with the following: $$\begin{array}{}\text{(Eq. 5)}& {\mathbf{x}}^k=\text{ReLU}\left(\frac{{\mathbf{x}}^{k-1}-\alpha (\bar{\mathbf{A}}\circ \mathbf{A}{)}^H[(\mathbf{A}\circ \mathbf{A}){\mathbf{x}}^{k-1}-\text{vec}\left(\hat{\mathrm{\Sigma }}\right)]-\lambda \alpha \gamma }{2\lambda \alpha (1-\gamma )+1}\right)k\ge 1,\end{array}$$

4. Methods

Given the covariance matrix $\hat{\mathrm{\Sigma }}\in {\mathbb{C}}^{M\times M}$, where $M$ is the number of microphones, the network reconstructs the spherical acoustic map (SAM) with 2 layers. $\mathbf{x}$ denotes the neuron at layer $l$ in Eq. 6 and Fig. 1.

The trainable parameters are $P_{\theta }(\mathbf{L})$, $\hat{\mathbf{B}}$, and $\tau$. The network is trained with the objective function in Eq. 4. $$\begin{array}{}\text{(Eq. 6)}& {\mathbf{x}}^l=\sigma (P_{\theta }(\mathbf{L}){\mathbf{x}}^{l-1}+[\hat{\mathbf{B}}\circ \mathbf{B}{\mathbf{]}}^{\mathbf{H}}\mathrm{vec}(\hat{\mathrm{\Sigma }})-\tau ),l=1,...,L\end{array}$$ where:

• ${\mathbf{x}}^l$ is the neuron at layer $l$.
• $\sigma$ is the ReLU activation function.
• $P_{\theta }$ is the deblurring operator and $P_{\theta }(\mathbf{L})$ the deblurring matrix.
• $\mathbf{L}\in {\mathbb{R}}^{N\times N}$ is the graph Laplacian.
• $\hat{\mathbf{B}}$ is the conjugate of the steering matrix.
• $\mathbf{B}$ is the steering matrix.
• $\tau$ is the threshold.
• $\hat{\mathrm{\Sigma }}$ is the estimated covariance matrix.
• $\mathrm{vec}(\cdot )$ is the vectorization operation.
  Let $\mathbf{A}\in {\mathbb{C}}^{M\times N}$ be a matrix, where $M$ is the number of rows and $N$ is the number of columns. The vectorization operator $\mathrm{vec}(\cdot )$ reshapes the matrix $\mathbf{A}$ into a vector of dimension $MN\times 1$ by stacking its columns: $\mathrm{vec}(A)\in {\mathbb{C}}^{MN\times 1}$. The operation is defined as: $$\begin{array}{}\text{(Eq. 7)}& [\mathrm{vec}(\mathbf{A})]M(j-1)+i=[\mathbf{A}]ij\mathrm{for}i=1,\dots ,M\mathrm{and}j=1,\dots ,N\end{array}$$

The