Abstract
Learning sparsifying dictionaries from a set of training signals has been shown to have much better performance than pre-designed dictionaries in many signal processing tasks, including image enhancement. To this aim, numerous practical dictionary learning (DL) algorithms have been proposed over the last decade. This paper introduces an accelerated DL algorithm based on iterative proximal methods. The new algorithm efficiently utilizes the iterative nature of DL process, and uses accelerated schemes for updating dictionary and coefficient matrix. Our numerical experiments on dictionary recovery show that, compared with some well-known DL algorithms, our proposed one has a better convergence rate. It is also able to successfully recover underlying dictionaries for different sparsity and noise levels.
This work has been funded by ERC project 2012-ERC-AdG-320684 CHESS and by the Center for International Scientific Studies and Collaboration (CISSC).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
It should be mentioned that, a modification to OMP has been proposed in [12], which reuses the coefficients obtained in each DL iteration in order to initialize OMP for the next DL iteration.
- 2.
For K-SVD and OMP, we have used K-SVD-Box v10 and OMP-Box v10 available at http://www.cs.technion.ac.il/ronrubin/software.html.
- 3.
The MATLAB implementation of our proposed algorithm together with those of the other compared algorithms will be made available at https://sites.google.com/site/fatemeghayem/.
References
Candès, E.J., Wakin, M.B.: An introduction to compressive sampling. IEEE Sig. Proc. Mag. 25(2), 21–30 (2008)
Rubinstein, R., Bruckstein, A.M., Elad, M.: Dictionaries for sparse representation modeling. Proc. IEEE 98(6), 1045–1057 (2010)
Engan, K., Aase, S.O., Hakon Husoy, J.: Method of optimal directions for frame design. In: Proceedings of IEEE ICASSP (1999)
Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Sig. Process. 54(11), 4311–4322 (2006)
Sadeghi, M., Babaie-Zadeh, M., Jutten, C.: Learning over-complete dictionaries based on atom-by-atom updating. IEEE Trans. Sig. Proc. 62(4), 883–891 (2014)
Sadeghi, M., Babaie-Zadeh, M., Jutten, C.: Dictionary learning for sparse representation: a novel approach. IEEE Sig. Proc. Lett. 20(12), 1195–1198 (2013)
Yaghoobi, M., Blumensath, T., Davies, M.E.: Dictionary learning for sparse approximations with the majorization method. IEEE Trans. Sig. Process. 57(6), 2178–2191 (2009)
Bao, C., Ji, H., Quan, Y., Shen, Z.: Dictionary learning for sparse coding: algorithms and convergence analysis. IEEE Trans. Pattern Anal. Mach. Intell. 38(7), 1356–1369 (2015)
Mousavi, H.S., Monga, V., Tran, T.D.: Iterative convex refinement for sparse recovery. IEEE Sig. Proc. Lett. 22(11), 1903–1907 (2015)
Sadeghi, M., Babaie-Zadeh, M.: Iterative sparsification-projection: fast and robust sparse signal approximation. IEEE Trans. Sig. Proc. 64(21), 5536–5548 (2016)
Pati, Y.C., Rezaiifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: Proceedings of Asilomar Conference on Signals, Systems, and Computers (1993)
Smith, L.N., Elad, M.: Improving dictionary learning: multiple dictionary updates and coefficient reuse. IEEE Sig. Proc. Lett. 20(1), 79–82 (2013)
Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1(3), 123–231 (2014)
Xu, Y., Yin, W.: A globally convergent algorithm for nonconvex optimization based on block coordinate update (2015)
Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: inertial proximal algorithm for nonconvex optimization. SIAM J. Imag. Sci. 7(2), 388–1419 (2014)
Meyer, C.D.: Matrix Analysis and Applied Linear Algebra. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (2000)
Elad, M.: Sparse and Redundant Representations. Springer, New York (2010)
Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2(1), 183–202 (2009)
Rubinstein, R., Zibulevsky, M., Elad, M.: Efficient implementation of the K-SVD algorithm using batch orthogonal matching pursuit. Technical report, Technion University (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Ghayem, F., Sadeghi, M., Babaie-Zadeh, M., Jutten, C. (2017). Accelerated Dictionary Learning for Sparse Signal Representation. In: Tichavský, P., Babaie-Zadeh, M., Michel, O., Thirion-Moreau, N. (eds) Latent Variable Analysis and Signal Separation. LVA/ICA 2017. Lecture Notes in Computer Science(), vol 10169. Springer, Cham. https://doi.org/10.1007/978-3-319-53547-0_50
Download citation
DOI: https://doi.org/10.1007/978-3-319-53547-0_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-53546-3
Online ISBN: 978-3-319-53547-0
eBook Packages: Computer ScienceComputer Science (R0)