ECG Denoising by Dictionary Learning Olivier Laligant IUT Le Creusot

Anastasia Zakharova IUT Le Creusot 12 rue de la Fonderie 71200 France

[email protected]

[email protected]

Christophe Stolz IUT Le Creusot [email protected]

Abstract In this paper we propose an algorithm for ECG denoising based on dictionary learning. The experimental results are compared to the results of the algorithm based on sparse 2-dimensional separable transform with overcomplete dictionaries. Besides, we study the behavior of the algorithm on the signals with natural noise.

1. Introduction An important issue in the diagnosis of cardiovascular diseases is the analysis of the form of ECG signals. These signals are usually damaged with noise coming from different sources, therefore to identify a waveform or to fix an anomaly we need to remove noise. The most important features of the signal which should be preserved while denoising are the form of QRS complex and time localization of peaks since they contain the key information about the signal. Among the most popular approaches one could mention wavelet denoising first proposed by Donoho in [6]. In this paper we will introduce a new algorithm of ECG denoising based on dictionary learning. That is, we will first build an overcomplete dictionary adapted to different types of ECG signals and use it for denoising. We will see that a dictionary learned on a sufficiently big training set performs well on different ECG signals. In particular, the denoising method based on such a dictionary preserves the form of QRS complex that allows us to recognize an anomaly. We show that the proposed algorithm outperforms the algorithm of ECG denoising by sparse 2d-separable transform introduced in [1].

2. Sparse decompositions with overcomplete dictionaries The idea of compressed sensing is based on an observation that many signals can be well-approximated in a sparse manner (i.e. with few non-zero coefficients) provided we have a suitable basis. Such a representation enables us to perform efficiently the analysis of a signal: it suffices to take a small number of non-adaptive (not depending on the given signal) measurements in a right basis instead of measuring all the signal. That’s why sparse representations are widely used for different signal processing tasks such as reconstruction, denoising, classification, compression. A nice overview of theoretical and numerical aspects of compressed sensing could be found in [9] or in [4] while the basic principles were introduced in [5]. Let us consider a signal x ∈ Rm and a dictionary D = [d1 , . . . , dk ] ∈ Rm×k (the case k > m is allowed, meaning that the dictionary is overcomplete). If we define ∥α∥0 = |supp(α)|

(1)

(we use a conventional notation ∥ · ∥0 though it is not a norm), then the decomposition coefficients are found by solving the l0 -minimization problem min ∥α∥0 s.t. Dα = x,

(2)

i.e., we are looking for the sparsest decomposition in D (the case of an overcomplete dictionary corresponds to the case of underdefined system of linear equations and we choose among them the sparsest one). However, since this problem is non-convex and non-smooth, and was shown to be N P -hard in general, it is usually replaced by a convex minimization problem referred to as l1 -minimization problem (known also under the name of basis pursuit): min ∥α∥1 s.t. Dα = x,

(3)

∑k where ∥α∥1 = i=1 |αi |. If 2 has a sparse solution, it will be the solution of 3 as well ([5]). Indeed, the realvalued case of 3 is equivalent to a linear programming task, so standard numerical methods could be applied to solve it, although more performing methods also exist. A common denoising method is to transform the signal using a fixed (complete or overcomplete) dictionary, to perform shrinkage using either hard or soft thresholding and finally to apply inverse transform. For example, Ghaffari et al. [1] used two-dimensional overcomplete DCT and DCT+Wavelet dictionaries for ECG denoising. It appears that the choice of the dictionary is crucial for the performance of a denoising method. However it is not always clear which dictionary to choose. Usually several dictionaries are compared ’manually’ and the one giving the sparsest representation is chosen. To avoid this, we will train our dictionary so that the representation of typical signals is as sparse as possible.

which is convex with respect to each of two variables D and α when the other one is fixed. The technique used by Lee et al. [7] for solving this problem consists in alternating between two variables: while one of them is minimized, the other one is fixed.

We will use the algorithm of online dictionary learning proposed by Mairal et al. [8]. That is, we use the method of block-coordinate descent with warm restarts described by Bertsekas [3] to update the dictionary and LARS-Lasso algorithm ([2]) to optimize the value of α. The advantage of block-coordinate descent is that it is parameter free and does not require much memory for storage, since the columns of D are modified sequentially. Furthermore, while calculating Di the previous iteration Di−1 could be used as warm restart. The motivation for this comes from the fact that

2.1. Dictionary learning Let D0 ∈ Rm×k be the initial dictionary, X = [x1 , . . . , xn ] ∈ Rm×n be the training set. The number of signals n in the training set is significantly larger than the signal dimension m. The size of the dictionary k is equal or bigger than m. Generally, k ≪ n. We are looking for a dictionary D which will provide us a good sparse approximation for the signals from the training set. To do this, we optimize the cost function 1∑ l(xi , D), n i=1

1∑ fˆi (x, Di ) = i j=1 i

(

1 ∥xi − Di αi ∥22 + λ∥αi ∥1 2

n

f (D) =

(4)

where l(x, D) is a loss function which is small if the representation of x is sparse in D. Following Lee et al. [7], we define function l(x, D) as the minimal value of l1 minimization problem: l(x, D) = min

α∈Rk

1 ∥x − Dα∥22 + λ∥α∥1 2

(5)

with λ being a regularization parameter. It’s well known that the solution α of 5 is sparse. It is not yet clear how the sparsity of x depends on the value of the parameter λ. We nevertheless choose a positive regularization parameter to avoid overfitting for the training set which can lead to a bad representation of test signals. In order that the values of α were not arbitrarily small we will put an additional constraint on the l2 -norms of the vectors in D. Let C be the convex set of matrices verifying this constraint: C = {D ∈ Rm×k s.t. ∀j = 1, . . . , k, dTj dj ≤ 1}.

(6)

To find the dictionary we solve a joint optimization problem min

D∈C,α∈Rk

1 ∥x − Dα∥22 + λ∥α∥1 , 2

(7)

converges almost surely [8].

2.2. Algorithm

) (8)

Dictionary Learning - Preprocessing Step

for i=1 to n 1. Using LARS-Lasso optimization method, compute αi = arg min 12 ∥xi − Di−1 α∥22 + λ∥α∥1

25

Output SNR

Input: [x1 , . . . , xn ] ∈ Rm×n - training set, D0 ∈ Rm×k - initial dictionary λ - regularization parameter

30

20

15

10

α∈Rk

2. Compute Di using Di−1 as warm restart: i ( ) ∑ 1 2 Di = arg min 1i 2 ∥xi − Dαi ∥2 + λ∥α∥1 D∈C

Trained dictionary 2d overcomplete dictionary 5

0

5

10

15

Input SNR

j=1

end

Figure 1. Comparison of denoising methods in terms of SNR

Output: D = Dn ∈ Rm×k - trained dictionary sampling rate of given signals: the signal period contains about 200 samples (since it depends on a patient it varies from signal to another). The training of the dictionary is more efficient if the size of the signals in the training set is Input: x ∈ RN - signal damaged with noise, comparable to the length of the period. Besides, taking m D ∈ Rm×k - trained dictionary reasonably big accelerates the algorithm. λ - regularization parameter As a test set we use signals from the MIT-BIH Arrythmia 1. Extract the segments of length m from the signal database which is claimed to be a representative selection of (the overlap of the segments does not exceed ⌊m/2⌋ samples) the variety of waveforms and artifacts that occur in routine and concatenate them so that we get a matrix X with m lines clinical use. 2. Decompose each column of the matrix X in D (using Lasso optimization technique) 3.1. Case of added Gaussian noise 3. Calculate the inverse transform Taking a signal from the test set, we add to it randomly 4. Put the segments back in their original places; generated Gaussian noise with different variances. Then in the overlap domain, calculate the mean of two signals the denoising is performed and output SNR is calculated for the method based on dictionary learning and for the 2Output: xr - denoised signal dimensional sparse separable method [1]. The results presented in Fig.1 are average results of thirteen test signals. 3. Experimental results As one can see from it, the denoising method based on dicLet us now examine the performance of the proposed altionary learning outperforms the 2-dimensional sparse sepgorithm. This section consists of two parts. First, we add arable method. a randomly generated white Gaussian noise to a signal and Let us mention that in [1] the 2-dimensional sparse sepapply the denoising algorithm. Then we compare the result arable algorithm is compared to the methods of soft threshwith the original signal by the means of SNR (here we use olding [6] and extended Kalman smoother filtering [10] in ∥x∥ the usual definition of SNR in = 20 log( ∥n∥ ) for input SNR terms of SNR and it is experimentally shown that it outper∥x∥ forms both of them (the MIT-BIH Normal Sinus Rhythm and SNR out = 20 log( ∥x−xr ∥ ) for output SNR, where by Database was used for simulations). x we denote the original signal, by xr the reconstructed signal, by n added noise). 3.2. Case of natural noise In the second part we take noisy signals and apply the denoising algorithm. For this simulation we take a signal from the MIT-BIH Arrythmia Database that contains a noisy segment. In this To learn the dictionary we use the database of ECG case, we have no idea of the form of the original signal. records obtained by the Creusot - Montceau-les-Mines hosHowever, since ECG signal is almost periodical it is reapital. We choose 14 signals taking a segment of 1000 samsonable to suppose that its denoised version might look like ples from each of them in such a way that they represent the the previous segment of the same signal. Using the same variety of clinically important phenomena. The training set parameters as in the previous subsection we perform the deis then formed by all the segments of length m = 128 of noising using the learned dictionary. each of these signals. The choice of m is motivated by the Signal Denoising

Initial signal

Initial signal with Gaussian noise

1

1

0

0

−1

−1 0

500 1000 1500 2000 2500

Denoising by dictionary learning

0

Denoising by sparse 2d transform

1

1

0

0

−1

−1 0

500 1000 1500 2000 2500

500 1000 1500 2000 2500

0

500 1000 1500 2000 2500

Figure 2. Performance of denoising algorithms on the signal with added Gaussian noise

The results of the simulation are presented on 3. We see that the form of the denoised signal is very similar to the one of the segment of the same signal. For completeness, we also give the result of denoising by using the 2-dimensional sparse separable method.

3.3. Conclusion We proposed the denoising algorithm based on learned dictionary. The simulation results show that the method performs well on noisy signals and outperforms the 2dimensional sparse separable method for the signals with added Gaussian noise.

References [1] M. B.-Z. C. J. A. Ghaffari, H. Palangi. Ecg denoising and compression by sparse 2d separable transform with overcomplete mixed dictionaries. 2010. [2] I. J. B. Efron, T. Hastie and R. Tibshirani. Least angle regression. Annals of Statistics, 32. [3] D. Bertsekas. Nonlinear programming. Athena Scientific Belmont. [4] E. Cand`es and M. Wakin. An introduction to compressive sampling. IEEE Signal Processing Magazine, 25.

[5] D. Donoho. Compressed sensing. IEEE Trans. on Information Theory, 52. [6] D. Donoho. Denoising by soft-thresholding. IEEE Trans. on Information Theory, 41:613–627, 1995. [7] R. R. H. Lee, A. Battle and A. Ng. Efficient sparse coding algorithms. Advances in Neural Information Processing Systems, 19:801–808, 2007. [8] J. P.-G. S. J. Mairal, F. Bach. Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research, 11:19–60, 2010. [9] M.Fornasier and H. Rauhut. Compressive sensing, 2011. Chapter in Part 2 of the ”Handbook of Mathematical Methods in Imaging” (O. Scherzer Ed.), Springer, 2011. [10] C. J. R. Sameni, M.B. Shamsollahi and G. Clifford. A nonlinear bayesian filtering framework for ecg denoising. IEEE Trans. on Biomedical Engineering, 54:2172–2185, 1007.

Noisy segment

Denoising by dictionary learning

1

1

0.5

0.5

0

0

−0.5

−0.5

−1

−1

500

600

700

800

500

Similar segment without noise

1

0.5

0.5

0

0

−0.5

−0.5

−1

−1 300

400

500

700

800

Denoising by sparse 2d separable transform

1

200

600

500

600

700

800

Figure 3. Performance of the denoising algorithm on the signal with natural noise