Experimental analysis of optimal window length for independent low-rank matrix analysis

Editor's Notes

• #5: This talk treats blind source separation problem, BSS, which is a separation technique of individual sources from the recorded mixture. The word “blind” means that the method does not require any prior information about the recording conditions, such as locations of microphones, sources, and room geometry. This kind of technique is very useful for many audio applications as a front-end system. In this talk, we only consider a “determined” situation, namely, the number of microphones is always equal to the number of sources.
• #6: This slid shows a history of basic theories in audio BSS. For acoustic signals, independent component analysis, ICA, was applied to the frequency domain signals as FDICA. After that, many permutation solvers for FDICA have been proposed, but eventually, an elegant solution, independent vector analysis, IVA was proposed. It is still extended to more flexible models. On the other hand, nonnegative matrix factorization, NMF, is also developed and extended to a multichannel signals for source separation problems. Recently, we have developed a new framework, which unifies these two powerful theories, called independent low-rank matrix analysis, ILRMA. I will explain about the detail, but in this talk,
• #7: we only focus on only these algorithms, FDICA, IVA, and ILRMA.
• #8: I here explain the motivation of this talk. In many frequency-domain BSS techniques, this equation, x=As, is always assumed, where x is a multichannel mixture signal in the frequency domain, i is a frequency bin and j is a time frames, A is a frequency-wise mixing matrix, and s is an original source. This is often called “linear time-invariant mixture” or a “rank-1 spatial model,” and this assumption is valid only when the window length in STFT is much longer than the length of room reverberation. So, we must use a longer window in STFT for validating this mixing assumption. However, if we use too much long window in STFT, the statistical bias will increase and the estimation becomes unstable. This is because the number of time frames J decreases. Therefore, there is a trade-off between short and long window lengths like this figure. In the paper in 2003, this trade-off was revealed only for FDICA. But we don’t know about this issue for the BSS methods that employ s structural source model, I mean it’s an IVA or ILRMA. So, in this talk, we experimentally confirm about this point, about the optimal window length for the new BSS techniques including ILRMA.
• #10: I briefly explain the separation mechanism in FDICA and IVA. In FDICA, ICA is applied to each frequency bin considering the scalar time-series signal as a random variable, and we maximize its non-Gaussianity to estimate the frequency-wise demixing matrix. In IVA, we consider a vector time-series random variable including all frequencies like this figure, then we assume a multivariate non-Gaussian distribution with a spherical property. Since spherical property ensures higher-order correlation among frequency bins, the permutation problem can be avoided in IVA.
• #11: The spherical source distribution in IVA can be extended to a more flexible model. We have extend it to a local Gaussian model, which employs a zero-mean complex Gaussian distribution with time-frequency-varying variance. Namely, in each time-frequency slot, i and j, complex Gaussian distribution is defined, and its variance, r, can fluctuate depending on time and frequency. This generative model is equivalent to that in Itakura-Saito NMF, and the variance r can be decomposed into a basis matrix T and an activation matrix V.
• #12: This is a graphical interpretation of the source model in ISNMF. In each time-frequency slot, zero-mean complex Gaussian distribution is defined, and they are mutually independent in all time, frequency, and sources. Now, the variance of these Gaussians is corresponding to the power spectrogram. Therefore, in the slot that has a strong power, such as a spectral peak, the Gaussian becomes wider, and the large power component can easily be generated. Note that, even though each slot is Gaussian, the marginal distribution in terms of time is non-Gaussian, because the variance fluctuates. So, since this matrix generative model is non-Gaussian, we can use this distribution as a source model in ICA-based method
• #13: resulting in an independent low-rank matrix analysis (ILRMA). Therefore, ILRMA is a unified method of IVA and ISNMF, and we employed NMF source model to capture the low-rank time-frequency structures of each source. This source model can improve the estimation accuracy of the demixing matrix.
• #14: This is a comparison of source models in FDICA, IVA, and ILRMA again. The important idea used in ILRMA is that the rank of TF matrix of mixture signal is always grater than the rank of TF matrix of each source before mixing. So, if we assume not only the independence between source but also a low-rank TF structure for each source, the separation will be done accurately.
• #16: As I already explained, the window length in STFT affects the performance of ICA-based separation. If we use too short window, the mixing assumption, x=As, does not hold anymore, and if we use too long window, the estimation becomes unstable because the number of time frames J decreases. However, ILRMA employs a full time-frequency modeling of sources. This model may improve the robustness to a decrease in J. This is our expectation. Let’s check about this issue by the experiment.
• #17: Here we used 4 music and 4 speech signals obtained from SiSEC database, and we produced the observed signal by convoluting the impulse response shown in the bottom. We used two types of impulse response, one has 300-ms-long reverberation, and the other one is 470 ms.
• #18: We compared 4 methods, FDICA + ideal permutation solver, FDICA + DOA-based permutation solver, IVA, and ILRMA. In FDICA+IPS, we used the reference, oracle source spectrogram. So this is an upper limit of FDICA. FDICA+DOA is a blind method that uses DOA clustering for solving the permutation problem. Of cause IVA and ILRMA are also blind method. Then, we used Hamming window with various window lengths.
• #19: First, we show the results with ideal initialization case. Namely, we first give a correct answer of demixing matrix as an initial value, which can be calculated using the oracle source s. So, the initial value provides the best separation performance here. In addition, only for ILRMA, we set the initial value of NMF model T and V as the oracle values. Therefore, FDICA+DOA and IVA are using the spatial oracle initialization, and FDICA+IPS and ILRMA are using spatial and spectral oracle initialization.
• #20: This is the result. The left ones are music, and right ones are the speech, and the reverberation time is short (top) and long (bottom). The horizontal axis shows the window length, and the vertical axis shows the separation performance. The colored lines are the results of ILRMA with various numbers of NMF bases. In the music results, we can see that FDICA and IVA could not achieve the good separation when the window becomes long. In ILRMA, the performance maintains even in a long long windows. This is obtained from the full modeling of time-frequency structure of each source. However, for the speech signals, the performance of ILRMA becomes worse. We guess this is because speech does not have a low-rank time-frequency structures, and the source model could not capture the precise speech structures even if we set the source model as an oracle one.
• #21: Next, we show the results with fully blind situation. Initial W is set to identity matrix, and the initial source model is randomized. Note that FDICA+IPS still uses the oracle spectrogram for solving the permutation.
• #22: This is the result. We could not obtain the same results as the previous case with ideal initialization. The performance of all the methods is degraded when the window length becomes long. Therefore, at least we can say that, ILRMA has a good potential to separate the sources even in a long window case, but in practice, the blind estimation of precise source model is a difficult problem.
• #35: This figure shows the difference of source models in IVA and ILRMA. Since IVA assumes frequency-uniform scale, it is almost an NMF with only one flat basis. On the other hand, ILRMA has more flexible source model with arbitrary number of spectral bases. So we can capture more precise TF structure of each source.
• #36: 提案手法ILRMAの対数尤度関数はこのように得られます．ここで（クリック）青丸で囲った空間分離フィルタWと，赤丸で囲ったNMF音源モデルTVが求めるべき変数になります．（クリック） さらにこの式は，（クリック）前半が従来のIVAのコスト関数と等価であり，（クリック）後半が従来のNMFのコスト関数と等価です．（クリック） したがって，IVAとNMFの反復更新式を交互に反復することで全変数を容易に推定できます． さらに，音源毎に適切なランク数を潜在変数で適応的に決定することも可能です． これは，冒頭で示した通り，音楽信号といえどもボーカルはあまり低ランクにならず，ドラム信号は低ランク，といったことが起こりえますので，音源毎の適切なランクが変わります． そのような状況に対して尤度最大化の基準で自動的に基底を割り振るのがこの潜在変数の役割です．
• #37: ILRMAの反復更新式はこのように導出できます． 空間分離フィルタの更新と音源モデルの更新を交互に行うことで，全変数が最適化されます． これらの反復計算で尤度が単調増加することが保証されているので，初期値近傍の局所解への収束が保証されています．
• #38: つまり，提案手法はまず空間分離フィルタを学習し，それで分離された信号の音色構造をNMFで学習，その結果得られる音源モデルを空間分離フィルタの学習に再利用し，さらに高精度な分離信号が得られる，という反復になります． このプロセスを何度も更新することで，音源毎の明確な音色構造が捉えられ，空間分離フィルタの性能向上が期待できます．
• #45: また，論文ではNMFの多チャネル信号への拡張手法である多チャネルNMFとILRMAが密接に関連しているという事実を明らかにしています． 簡単に説明いたしますと，従来の多チャネルNMFで定義されている空間情報に関するモデル「空間相関行列」のランクが1となる制約を課した場合とILRMAが等価となる，という事実です． ただし，多チャネルNMFは混合系を推定する手法であり，ILRMAやIVAのように分離系を推定する技術とは異なります．そのため，多チャネルNMFは計算効率や不安定性の観点から実用性にやや欠ける点があります．これに関しては比較実験で示します．
• #46: さて，IVAの音源モデルと提案手法の音源モデルの違いについて説明します． IVAは，球対称な多次元分布に基づくため，周波数方向に一様な分散を持つ音源モデルになります．これは1本の基底のNMFに非常に近いと言えます． 一方提案手法は，任意の数の基底を用いたNMFが音源モデルになります．従って，この図のように音楽信号の具体的な調波構造を捉えることができます． このように明確な音源モデルを推定することで，独立性基準での分離性能の向上が期待できます．
• #48: we only focus on these three algorithms, FDICA, IVA, and ILRMA.