结构稀疏模型
|
摘要
由于生物信息学、心理学诊断、计算语言与语音学、计算机视觉、门户网站、电子商务、移动互联网、物联网中处理高维和超高维数据的需求不断涌现,迫切需要研究具有变量选择和特征降维功能的回归和分类模型,所以以Lasso、自适应Lasso和elastic net等为代表的稀疏模型近年来在机器学习领域中非常流行.然而,这些稀疏模型没有考虑变量中存在的组结构、重叠组结构、双层稀疏结构、多层稀疏结构、树结构和图结构等结构化信息.结构稀疏模型考虑了这些结构先验信息,改善了模型对特征选择的结果和稀疏模型在相应结构稀疏化数据背景下的统计特性.结构稀疏化模型是当前稀疏学习领域的研究方向,近几年来涌现出很多研究成果,文中对主流的结构稀疏模型,如组结构稀疏模型、结构稀疏字典学习、双层结构稀疏模型、树结构稀疏模型和图结构稀疏模型进行了总结,对结构稀疏模型目标函数中包含非可微、非凸和不可分离变量的结构稀疏模型目标函数近似转换为可微、凸和可分离变量的近似目标函数的技术如控制-受控不等式(Majority-Minority,MM),Nesterov双目标函数近似方法,一阶泰勒展开和二阶泰勒展开技术,对求解结构稀疏化模型近似目标函数的优化算法如最小角回归算法、组最小角回归算法(Group Least Angle Regression,Group LARS)、块坐标下降算法(block coordinate descent algorithm)、分块坐标梯度下降算法(block coordinate gradient descent algorithm)、局部坐标下降算法(local coordinate descent algorithm)、谱投影梯度法(Spectral Projected Gradient algorithm)、主动集算法(active set algrithm)和交替方向乘子算法(Alternating Direction Method of Multipliers,ADMM)进行了比较分析,并且对结构稀疏模型未来的研究方向进行了探讨.
As continuing to emerge demand of high dimensional and ultra-high dimensional regression and classification in bioinformatics,psychology diagnosis,computational linguistics and phonetics,computer vision,the Portal site,e-commerce,mobile Internet,and Internet of Things,there is an urgent need to study high dimensional and ultra-high dimensional variable selection and feature dimension reduction in regression and classification model.Thus the sparse models have been quite popular in recent years,such as the Lasso,adaptive Lasso and the elastic net.However,these sparse models ignore the structural information of the variables,such as the group structure sparsity,overlapping group structure sparsity,bi-level sparse structure,Multi-layer Sparse structure,tree structure sparsity and graph structure sparsity.The structured sparse models that consider this structural prior information can improve the statistic properties of the sparse models when facing with the corresponding structure sparse datasets.The structuredsparse models are the hot research direction of the sparse model learning and many research findings appear in recent years.This paper gives a systematic survey of mainstream of structured sparsity model,such as group structure sparse model,structure sparse dictionary learning,bi-level structure sparse model,and tree structure sparse model and graphical structure sparse model.As objective function of structure sparse model contains non-differential,non-convex and non-separable variable,objective function of structure sparse model first needs to be approximately transform into differentiable,convex and separable variable ones. The main approximate transformation methods are summarized,including majority-minority inequality,approximate method of Nesterov's double objective function,first order Taylor expansion and second order Taylor expansion.Optimization algorithms solving approximate objective function of structure sparse model are carried out a detailed comparative analysis on the conception,the features and performance,which involves minimum angle regression,group Least angle regression,block coordinate descent algorithm,block coordinate gradient descent algorithm,local coordinate descent algorithm,spectrum projection gradient method,active set algorithm and alternating direction method of multipliers,some future research directions are discussed in the final section.
As continuing to emerge demand of high dimensional and ultra-high dimensional regression and classification in bioinformatics,psychology diagnosis,computational linguistics and phonetics,computer vision,the Portal site,e-commerce,mobile Internet,and Internet of Things,there is an urgent need to study high dimensional and ultra-high dimensional variable selection and feature dimension reduction in regression and classification model.Thus the sparse models have been quite popular in recent years,such as the Lasso,adaptive Lasso and the elastic net.However,these sparse models ignore the structural information of the variables,such as the group structure sparsity,overlapping group structure sparsity,bi-level sparse structure,Multi-layer Sparse structure,tree structure sparsity and graph structure sparsity.The structured sparse models that consider this structural prior information can improve the statistic properties of the sparse models when facing with the corresponding structure sparse datasets.The structuredsparse models are the hot research direction of the sparse model learning and many research findings appear in recent years.This paper gives a systematic survey of mainstream of structured sparsity model,such as group structure sparse model,structure sparse dictionary learning,bi-level structure sparse model,and tree structure sparse model and graphical structure sparse model.As objective function of structure sparse model contains non-differential,non-convex and non-separable variable,objective function of structure sparse model first needs to be approximately transform into differentiable,convex and separable variable ones. The main approximate transformation methods are summarized,including majority-minority inequality,approximate method of Nesterov's double objective function,first order Taylor expansion and second order Taylor expansion.Optimization algorithms solving approximate objective function of structure sparse model are carried out a detailed comparative analysis on the conception,the features and performance,which involves minimum angle regression,group Least angle regression,block coordinate descent algorithm,block coordinate gradient descent algorithm,local coordinate descent algorithm,spectrum projection gradient method,active set algorithm and alternating direction method of multipliers,some future research directions are discussed in the final section.
引文
[1]Tibshirani R.Regression shrinkage and selection via the lasso:A retrospective.Journal of the Royal Statistical Society:Series B(Methodological),2011,73(3):273-282
[2]Hosmer Jr D W,Lemeshow S.Applied Logistic Regression.USA New Jersey:John Wiley&Sons,2004
[3]Yuan M,Lin Y.Model selection and estimation in regression with grouped variables.Journal of the Royal Statistical Society:Series B(Statistical Methodology),2006,68(1):49-67
[4]Engelhardt B E,Adams R P.Bayesian structured sparsity from Gaussian fields.arXiv preprint arXiv:1407.2235,2014
[5]Ayaz U,Dirksen S,Rauhut H.Uniform recovery of fusion frame structured sparse signals.arXiv preprint arXiv:1407.7680,2014
[6]Doshi-Velez F,Wallace B,Adams R.Graph-sparse LDA:A topic model with structured sparsity.arXiv preprint arXiv:1410.4510,2014
[7]Zhang S,Qian H,Zhang Z.A nonconvex approach for structured sparse learning.arXiv preprint arXiv:1503.02164,2015
[8]Chadwick J N,Bindel D S.An efficient solver for sparse linear systems based on rank-structured cholesky factorization.arXiv preprint arXiv:1507.05593,2015
[9]Qiu C,Vaswani N,Lois B.Recursive robust PCA or recursive sparse recovery in large but structured noise.IEEE Transactions on Information Theory,2014,60(8):5007-5039
[10]Oymak S,Jalali A,Fazel M.Simultaneously structured models with application to sparse and low-rank matrices.IEEE Transactions on Information Theory,2015,61(5):2886-2908
[11]Wimalajeewa T,Eldar Y C,Varshney P K.Subspace recovery from structured union of subspaces.IEEE Transactions on Information Theory,2015,61(4):2101-2114
[12]Soni A,Haupt J.On the fundamental limits of recovering tree sparse vectors from noisy linear measurements.IEEE Transactions on Information Theory,2014,60(1):133-149
[13]Sun X,Qu Q,Nasrabadi N M.Structured priors for sparserepresentation-based hyperspectral image classification.IEEE Geoscience and Remote Sensing Letters,2014,11(7):1235-1239
[14]Kyrillidis A,Baldassarre L,El Halabi M.Structured sparsity:Discrete and convex approaches//Boche H,Calderbank R,Kutyniok G,Vybíral J eds.Compressed Sensing and its Applications.Springer International Publishing,Switzerland:Lausanne,2015:341-387
[15]Vila J P.Empirical-Bayes Approaches to Recovery of Structured Sparse Signals via Approximate Message Passing[Ph.D.dissertation].The Ohio State University,Ohio,USA,2015
[16]Bai T,Li Y.Robust visual tracking using flexible structured sparse representation.IEEE Transactions on Industrial Informatics,2014,10(1):538-547
[17]Hegde C,Indyk P,Schmidt L.A nearly-linear time framework for graph-structured sparsity//Proceedings of the 32nd International Conference on Machine Learning(ICML 2015).Lille,France,2015:928-937
[18]Huang J,Zhang T.The benefit of group sparsity.The Annals of Statistics,2010,38(4):1978-2004
[19]Sra S.Fast projections onto mixed-norm balls with applications.Data Mining and Knowledge Discovery,2012,25(2):358-377
[20]Lazzaro D,Montefusco L B,Papi S.Blind cluster structured sparse signal recovery:A nonconvex approach.Signal Processing,2015,109(4):212-225
[21]Vogt J,Roth V.A complete analysis of the L1,pGroupLasso.arXiv preprint arXiv:1206.4632,2012
[22]Rakotomamonjy A,Flamary R,Gasso G,Stephane C.Lp-Lq penalty for sparse linear and sparse multiple kernel multi-task learning.IEEE Transactions on Neural Networks,2011,22(8):1307-1320
[23]Vogt J E,Roth V.The group-Lasso:l1,∞regularization versus l1,2 regularization//Proceedings of the 32nd DAGM Conference on Pattern Recognition.Darmstadt,Germany:Springer-Verlag,2010:252-261
[24]Obozinski G,Jacob L,Vert J P.Group Lasso with overlaps:The latent Group Lasso approach.arXiv preprint arXiv:1110.0413,2011
[25]Jenatton R,Audibert J Y,Bach F.Structured variable selection with sparsity-inducing norms.The Journal of Machine Learning Research,2011,12(1):2777-2824
[26]Percival D.Structured,sparse aggregation.Journal of the American Statistical Association,2012,107(498):814-823
[27]Wang H,Leng C.A note on adaptive Group Lasso.Computational Statistics and Data Analysis,2008,52(12):5277-5286
[28]Wei F,Huang J.Consistent group selection in highdimensional linear regression.Bernoulli,2010,16(4):1369-1384
[29]Meier L,van de Geer S,Bühlmann P.The Group Lasso for logistic regression.Journal of the Royal Statistical Society:Series B(Statistical Methodology),2008,70(1):53-71
[30]Wang L,Chen G,Li H.Group SCAD regression analysis for microarray time course gene expression data.Bioinformatics,2007,23(12):1486-1494
[31]Fan J,Li R.Variable selection via nonconcave penalized likelihood and its oracle properties.Journal of the American Statistical Association,2001,96(456):1348-1360
[32]Huang J,Ma S,Xie H,Zhang C H.A group bridge approach for variable selection.Biometrika,2009,96(2):339-355
[33]Fu W J.Penalized regressions:The bridge versus the Lasso.Journal of Computational and Graphical Statistics,1998,7(3):397-416
[34]Huang J,Breheny P,Ma S.A selective review of group selection in high-dimensional models.Statistical Science,2012,27(4):481-499
[35]Zhang C H,Zhang S S.Confidence intervals for low dimensional parameters in high dimensional linear models.Journal of the Royal Statistical Society,2014,76(1):217-242
[36]Huang J,Ma S,Xie H,Zhang C H.Amendments and corrections:A group bridge approach for variable selection.Biometrika,2009,96(4):1024-1024
[37]Breheny P.The group exponential lasso for bi-level variable selection.Biometrics,2015,71(3):1-10
[38]Friedman J,Hastie T,Tibshirani R.A note on the Group Lasso and a sparse Group Lasso.arXiv preprint arXiv:1001.0736,2010
[39]Yuan X T,Liu X,Yan S.Visual classification with multitask joint sparse representation.IEEE Transactions on Image Processing,2012,21(10):4349-4360
[40]Gao S,Chia L T,Tsang I W H.Multi-layer group sparse coding—For concurrent image classification and annotation//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,USA,2011:2809-2816
[41]Seetharaman I.Consistent Bi-Level Variable Selection via Composite Group Bridge Penalized Regression[Ph.D.dissertation].Kansas State Univesity,Kansas,USA,2013
[42]Liu M,Zhang D,Yap P T.Tree-guided sparse coding for brain disease classification//Proceedings of the Medical Image Computing and Computer-Assisted Intervention(MICCAI2012).Berlin Heidelberg,Germany:Springer,2012:239-247
[43]Liu J,Ye J.Moreau-Yosida regularization for grouped tree structure learning//Proceedings of the Advances in Neural Information Processing Systems 23:24th Annual Conference on Neural Information Processing Systems.Vancouver,Canada,2010:1459-1467
[44]Kim S,Xing E P.Tree-guided Group Lasso for multi-task regression with structured sparsity//Proceedings of the 27th International Conference on Machine Learning.Haifa,Israel,2010:543-550
[45]Turlach B A,Venables W N,Wright S J.Simultaneous variable selection.Technometrics,2005,47(3):349-363
[46]Friedman J,Hastie T,Tibshirani R.Sparse inverse covariance estimation with the graphical lasso.Biostatistics,2008,9(3):432-441
[47]Olshausen B A,Field D J.Emergence of simple-cell receptive field properties by learning a sparse code for natural images.Nature,1996,381(6):607-609
[48]Olshausen B A,Field D J.Sparse coding with an overcomplete basis set:A strategy employed by V1?Vision Research,1997,37(23):3311-3325
[49]Olshausen B A,Field D J.How close are we to understanding V1?Neural Computation,2005,17(8):1665-1699
[50]Simoncelli E P,Olshausen B A.Natural image statistics and neural representation.Annual Review of Neuroscience,2001,24(1):1193-1216
[51]Jenatton R,Obozinski G,Bach F.Structured sparse principal component analysis//Proceedings of the 13th International Conference on Artificial Intelligence and Statistics(AISTATS2010).Chia Laguna Resort,Sardinia,Italy:JMLR Proceedings,2010:366-373
[52]Ramirez I,Sprechmann P,Sapiro G.Classification and clustering via dictionary learning with structured incoherence and shared features//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,USA,2010:3501-3508
[53]Yu G,Sapiro G,Mallat S.Solving inverse problems with piecewise linear estimators:From Gaussian mixture models to structured sparsity.IEEE Transactions on Image Processing,2012,21(5):2481-2499
[54]Mairal J,Bach F,Ponce J,Sapiro G.Online learning for matrix factorization and sparse coding.Journal of Machine Learning Research,2010,11(1):19-60
[55]Mairal J,Jenatton R,Obozinski G,Bach F.Network flow algorithms for structured sparsity//Proceedings of the Advances in Neural Information Processing Systems 23:24th Annual Conference on Neural Information Processing Systems.Vancouver,Canada,2010:1558-1566
[56]Mairal J,Jenatton R,Obozinski G,Bach F.Convex and network flow optimization for structured sparsity.Journal of Machine Learning Research,2011,12(9):2681-2720
[57]Mairal J,Bach F,Ponce J.Task-driven dictionary learning.IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(4):791-804
[58]Zhu S-C,Guo C-E,Wang Y,Xu Z.What are textons?International Journal of Computer Vision,2005,62(1-2):121-143
[59]Zeiler M D,Krishnan D,Taylor G W.Deconvolutional networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,USA,2010:2528-2535
[60]Zeiler M D,Taylor G W,Fergus R.Adaptive deconvolutional networks for mid and high level feature learning//Proceedings of the IEEE International Conference on Computer Vision(ICCV 2011).Barcelona,Spain,2011:2018-2025
[61]Bo L,Ren X,Fox D.Multipath sparse coding using hierarchical matching pursuit//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,USA,2013:660-667
[62]Dalal N,Triggs B.Histograms of oriented gradients for human detection//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,USA,2005:886-893
[63]Ren X,Ramanan D.Histograms of sparse codes for object detection//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,2013:3246-3253
[64]Rigamonti R,Sironi A,Lepetit V,Fua P.Learning separable filters//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,USA,2013:2754-2761
[65]Kavukcuoglu K,Sermanet P,Boureau Y-L,et al.Learning convolutional feature hierarchies for visual recognition//Proceedings of the Advances in Neural Information Processing Systems 23:24th Annual Conference on Neural Information Processing Systems.Vancouver,Canada,2010:1090-1098
[66]Garrigues P,Olshausen B A.Group sparse coding with a Laplacian scale mixture prior//Proceedings of the Advances in Neural Information Processing Systems 23:24th Annual Conference on Neural Information Processing Systems.Vancouver,Canada,2010:676-684
[67]Gregor K,Szlam A,LeCun Y.Structured sparse coding via lateral inhibition//Proceedings of the 28th International Conference on Machine Learning(ICML 2011).Bellevue,Washington,USA:Omnipress,2011:1116-1124
[68]Raman S,Fuchs T J,Wild P J,et al.The Bayesian groupLasso for analyzing contingency tables//Proceedings of the26th Annual International Conference on Machine Learning.Montreal,Canada,2009:881-888
[69]Chandran M.Analysis of Bayesian Group-Lasso in Regression Models[Ph.D.dissertation].University of Florida,Florida,USA,2011
[70]Huang J,Zhang T,Metaxas D.Learning with structured sparsity.Journal of Machine Learning Research,2011,12(11):3371-3412
[71]Bach F R.Structured sparsity-inducing norms through submodular functions//Proceedings of the Advances in Neural Information Processing Systems 23:24th Annual Conference on Neural Information Processing Systems.Vancouver,Canada,2010:118-126
[72]Micchelli C A,Morales J M,Pontil M.Regularizers for structured sparsity.Advances in Computational Mathematics,2013,38(3):455-489
[73]Bach F R.Consistency of the Group Lasso and multiple kernel learning.The Journal of Machine Learning Research,2008,9(1):1179-1225
[74]Sharma D B,Bondell H D,Zhang H H.Consistent group identification and variable selection in regression with correlated predictors.Journal of Computational and Graphical Statistics,2013,22(2):319-340
[75]Mairal J,Yu B.Supervised feature selection in graphs with path coding penalties and network flows.Journal of Machine Learning Research,2013,14(1):2449-2485
[76]Rosset S,Zhu J.Piecewise linear regularized solution paths.The Annals of Statistics,2007,35(3):1012-1030
[77]Lv X,Bi G,Wan C.The group lasso for stable recovery of block-sparse signal representations.IEEE Transactions on Signal Processing,2011,59(4):1371-1382
[78]Jacob L,Obozinski G,Vert J P.Group Lasso with overlap and graph lasso//Proceedings of the 26th Annual International Conference on Machine Learning(ICML 2009).Montreal,Canada,2009:433-440
[79]Vincent M,Hansen N R.Sparse group lasso and high dimensional multinomial classification.Computational Statistics&Data Analysis,2014,71(3):771-786
[80]Geng Z,Wang S,Yu M,et al.Group variable selection via convex log-exp-sum penalty with application to a breast cancer survivor study.Biometrics,2015,71(1):53-62
[81]Wu F,Yuan Y,Rui Y.Annotating web images using NOVA:NOn-conVex group spArsity//Proceedings of the20th ACM Multimedia Conference(MM’12).Nara,Japan,2012:509-518
[82]Zeng L,Xie J.Group variable selection via SCAD-L2.Statistics,2014,48(1):49-66
[83]Yuan M,Lin Y.Model selection and estimation in the Gaussian graphical model.Biometrika,2007,94(1):19-35
[84]Simon N,Friedman J,Hastie T.A sparse-Group Lasso.Journal of Computational and Graphical Statistics,2013,22(2):231-245
[85]Zhou H,Sehl M,Sinsheimer J,Lange K.Association screening of common and rare genetic variants by penalized regression.Bioinformatics,2010,26(19):2375-2382
[86]Sohn K A,Kim S.Joint estimation of structured sparsity and output structure in multiple-output regression via inversecovariance regularization//Proceedings of the 15th International Conference on Artificial Intelligence and Statistics(AISTATS2012).La Palma,Canary Islands,2012:1081-1089
[87]Swirszcz G,Lozano A C.Multi-level lasso for sparse multitask regression//Proceedings of the 29th International Conference on Machine Learning(ICML 2012).Edinburgh,UK,2012:361-368
[88]Wang J,Ye J.Two-layer feature reduction for Sparse-Group Lasso via decomposition of convex sets//Proceedings of the Advances in Neural Information Processing Systems 27:Annual Conference on Neural Information Processing Systems2014.Montreal,Canada,2014:2132-2140
[89]Caner M,Han X.Selecting the correct number of factors in approximate factor models:The large panel case with group bridge estimators.Journal of Business&Economic Statistics,2014,32(3):359-374
[90]Breheny P,Huang J.Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors.Statistics and Computing,2015,25(2):173-187
[91]Chen X,Lin Q,Kim S,et al.Smoothing proximal gradient method for general structured sparse regression.The Annals of Applied Statistics,2010,6(2):719-752
[92]Nesterov Y.Smooth minimization of non-smooth functions.Mathematical Programming,2005,103(1):127-152
[93]Jenatton R,Mairal J,Obozinski G,Bach F.Proximal methods for hierarchical sparse coding.Journal of Machine Learning Research,2011,12(7):2297-2334
[94]Beck A,Teboulle M.A fast iterative shrinkage-thresholding algorithm for linear inverse problems.SIAM Journal on Imaging Sciences,2009,2(1):183-202
[95]Breheny P,Huang J.Penalized methods for bi-level variable selection.Statistics and Its Interface,2009,2(3):369-380
[96]Tao Qing,Gao Qian-Kun,Jiang Ji-Yuan,Chu De-Jun.Survey of solving the optimization problems for sparse learning.Journal of Software,2013,24(11):2498-2507(in Chinese)(陶卿,高乾坤,姜纪远,储德军.稀疏学习优化问题的求解综述.软件学报,2013,24(11):2498-2507)
[97]Rish I,Grabarnik G.Sparse Modeling:Theory,Algorithms,and Applications.New York,USA:CRC Press,2014
[98]Rao N,Nowak R,Cox C.Classification with the sparse Group Lasso.IEEE Transactions on Signal Processing,2016,64(2):448-463
[99]Wahab S,Zakaria M N,Sidek D,et al.Evaluation of auditory hallucinations in patients with schizophrenia:A validation study of the Malay version of Psychotic Symptom Rating Scales(PSYRATS).Psychiatry Research,2015,228(3):462-467
[100]Huang S,Li J,Ye J,et al.A sparse structure learning algorithm for Gaussian Bayesian network identification from high-dimensional data.IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(6):1328-1342
[101]Honorio J,Ortiz L,Samaras D,et al.Sparse and locally constant Gaussian graphical models//Proceedings of the Advances in Neural Information Processing Systems 22:23rd Annual Conference on Neural Information Processing Systems 2009.Vancouver,Canada:Curran Associates,2009:745-753
[102]Honorio J,Samaras D,Rish I,Cecchi G.Variable selection for Gaussian graphical models//Proceedings of the 15th International Conference on Artificial Intelligence and Statistics(AISTATS 2012).La Palma,Canary Islands:JMLR Proceedings,2012:538-546
[103]Klami A,Virtanen S,Leppaho E,et al.Group factor analysis.IEEE Transactions on Neural Networks and Learning Systems,2015,26(9):2136-2147
[104]Chatterjee S,Steinhaeuser K,Banerjee A.Sparse Group Lasso:Consistency and climate applications//Proceedings of the 12th SIAM International Conference on Data Mining.Anaheim,USA,2012:47-58
[105]Gao L,Zhou S,Guan J.Effectively classifying short texts by structured sparse representation with dictionary filtering.Information Sciences,2015,323(11):130-142
[106]Downey D,Bhagavatula C,Yang Y.Efficient methods for inferring large sparse topic hierarchies//Proceedings of the53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing.Beijing,China,2015:774-784
[107]Zheng M,Bu J,Chen C,et al.Graph regularized sparse coding for image representation.IEEE Transactions on Image Processing,2011,20(5):1327-1336
[108]Liu H,Yu H,Deng Z H.Multi-document summarization based on two-level sparse representation model//Proceedings of the 29th AAAI Conference on Artificial Intelligence.Austin,USA.2015:196-202
[109]Pnevmatikakis E A,Gao Y,Soudry D.A structured matrix factorization framework for large scale calcium imaging data analysis.arXiv preprint arXiv:1409.2903,2014
[110]Han Y,Wu F,Tian Q.Image annotation by input—output structural grouping sparsity.IEEE Transactions on Image Processing,2012,21(6):3066-3079
[111]Moayedi F,Azimifar Z,Boostani R.Structured sparse representation for human action recognition.Neurocomputing,2015,161(8):38-46
[112]Ly N H,Du Q,Fowler J E.Sparse graph-based discriminant analysis for hyperspectral imagery.IEEE Transactions on Geoscience and Remote Sensing,2014,52(7):3872-3884
[113]Hegde C,Indyk P,Schmidt L.A fast approximation algorithm for tree-sparse recovery//Proceeding of the IEEE International Symposium on Information Theory(ISIT).New Jersey,USA,2014:1842-1846
[114]Bui H Q,La C N H,Do M N.A fast tree-based algorithm for compressed sensing with sparse-tree prior.Signal Processing,2015,108(3):628-641
[115]Chen H,Wan Q,Fan R.Double-constraint flexible tree search-based orthogonal matching pursuit for DOA estimation using dynamic sensor arrays.International Journal of Electronics,2015,103(5):1-9
[116]Castro R M,Tanczos E.Adaptive sensing for estimation of structured sparse signals.IEEE Transactions on Information Theory,2015,61(4):2060-2080
[117]Peleg T,Eldar Y C,Elad M.Exploiting statistical dependencies in sparse representations for signal recovery.IEEE Transactions on Signal Processing,2012,60(5):2286-2303
[118]Gui J,Tao D,Sun Z.Group sparse multiview patch alignment framework with view consistency for image classification.IEEE Transactions on Image Processing,2014,23(7):3126-3137
[119]Yang Y,Zou H.A fast unified algorithm for solving grouplasso penalize learning problems.Statistics and Computing,2015,25(6):1129-1141
[120]Scheetz T E,Kim K Y A,Swiderski R E,et al.Regulation of gene expression in the mammalian eye and its relevance to eye disease.Proceedings of the National Academy of Sciences,2006,103(39):14429-14434
[121]Segal M R,Dahlquist K D,Conklin B R.Regression approaches for microarray data analysis.Journal of Computational Biology,2003,10(6):961-980
[122]SФbФS,AlmФy T,AarФe J,et al.ST-PLS:A multidirectional nearest shrunken centroid type classifier via PLS.Journal of Chemometrics,2008,22(1):54-62
[123]Graham K,de Las Morenas A,Tripathi A,et al.Gene expression in histologically normal epithelium from breast cancer patients and from cancer-free prophylactic mastectomy patients shares a similar profile.British Journal of Cancer,2010,102(8):1284-1293
[124]Alon U,Barkai N,Notterman D A,et al.Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays.Proceedings of the National Academy of Sciences,1999,96(12):6745-6750
[125]Singh D,Febbo P G,Ross K,et al.Gene expression correlates of clinical prostate cancer behavior.Cancer Cell,2002,1(2):203-209
[126]Gorman R P,Sejnowski T J.Analysis of hidden units in a layered network trained to classify sonar targets.Neural Networks,1988,1(1):75-89
[127]Zhang T.Analysis of multi-stage convex relaxation for sparse regularization.The Journal of Machine Learning Research,2010,11(3):1081-1107
[128]Zhang T.Multi-stage convex relaxation for feature selection.arXiv preprint arXiv:1106.0565,2011
[129]Candes E J,Wakin M B,Boyd S P.Enhancing sparsity by reweighted L1minimization.Journal of Fourier Analysis and Applications,2008,14(5-6):877-905
[130]Geman D,Yang C.Nonlinear image recovery with halfquadratic regularization.IEEE Transactions on Image Processing,1995,4(7):932-946
[131]Foucart S,Lai M J.Sparsest solutions of underdetermined linear systems via Lqminimization for 0 [132]Yang H,Xu Z,King I.Online learning for Group Lasso//Proceedings of the 27th International Conference on Machine Learning.Scotland,UK:Omnipress,2010:1191-1198
[133]van de Geer S A,Bühlmann P.On the conditions used to prove oracle results for the Lasso.Electronic Journal of Statistics,2009,3(11):1360-1392
[134]Zhang T.Some sharp performance bounds for least squares regression with L1regularization.The Annals of Statistics,2009,37(5):2109-2144
[135]Ye F,Zhang C H.Rate minimaxity of the Lasso and dantzig selector for the Lqloss in Lrballs.The Journal of Machine Learning Research,2010,11:3519-3540
[136]Chandrasekaran V,Parrilo P A,Willsky A S.Latent variable graphical model selection via convex optimization.The Annals of Statistics,2012,40(4):1935-1967
[137]Ma S,Xue L,Zou H.Alternating direction methods for latent variable gaussian graphical model selection.Neural Computation,2013,25(8):2172-2198
[138]Lauritzen S,Meinshausen N.Discussion:Latent variable graphical model selection via convex optimization.The Annals of Statistics,2012,40(4):1973-1977
[139]Shojaie A,Michailidis G.Penalized likelihood methods for estimation of sparse high-dimensional directed acyclic graphs.Biometrika,2010,97(3):519-538
[140]Fu F,Zhou Q.Learning sparse causal Gaussian networks with experimental intervention:Regularization and coordinate descent.Journal of the American Statistical Association,2013,108(501):288-300
[141]Ravikumar P,Wainwright M J,Lafferty J D.High-dimensional Ising model selection using1-regularized logistic regression.The Annals of Statistics,2010,38(3):1287-1319
[142]Wainwright M J,Ravikumar P,Lafferty J D.High-dimensional graphical model selection using L1-regularized logistic regression//Proceedings of the 20th Annual Conference on Neural Information Processing Systems.Vancouver,Canada:MIT Press,2006:1465-1472
[143]Liu H,Han F,Yuan M.High-dimensional semiparametric Gaussian copula graphical models.The Annals of Statistics,2012,40(4):2293-2326
[144]Allen G I,Liu Z.A log-linear graphical model for inferring genetic networks from high-throughput sequencing data//Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine(BIBM).Philadelphia,USA,2012:1-6
[145]Yang E,Ravikumar P D,Allen G I.Graphical models via generalized linear models//Proceedings of the Advances in Neural Information Processing Systems 25:26th Annual Conference on Neural Information Processing Systems.Lake Tahoe,USA:Curran Associates,2012:1367-1375
[146]Liu Q,Ihler A T.Learning scale free networks by reweighted1 regularization//Proceedings of the International Conference on Artificial Intelligence and Statistics.Fort Lauderdale,USA,2011:40-48
[147]Danaher P,Wang P,Witten D M.The joint graphical lasso for inverse covariance estimation across multiple classes.Journal of the Royal Statistical Society:Series B(Statistical Methodology),2013,76(2):373-397
[148]Yang S,Pan Z,Shen X.Fused multiple graphical lasso.arXiv preprint arXiv:1209.2139,2012
[149]Zhang B,Wang Y.Learning structural changes of Gaussian graphical models in controlled experiments.arXiv preprint arXiv:1203.3532,2012
[150]Honorio J,Samaras D.Multi-task learning of Gaussian graphical models//Proceedings of the 27th International Conference on Machine Learning.Haifa,Israel:Omnipress,2010:447-454
[151]Committee on the Analysis of Massive Data et al.Frontiers in Massive Data Analysis.Washington,USA:National Academies Press,2013
[152]Franke B,Plante J F,Roscher R,et al.Statistical Inference,Learning and Models in Big Data.arXiv preprint arXiv:1509.02900,2015
[153]Hammer B,He H,Martinetz T.Learning and modeling big data//Proceedings of the 22nd European Symposium on Artificial Neural Networks,Computational Intelligence and Machine Learning.Bruges,Belgium,2014:105-110
[154]Richtárik P,Taká M.Distributed coordinate descent method for learning with big data.arXiv preprint arXiv:1310.2059,2013
[155]Richtárik P,Taká M.Parallel coordinate descent methods for big data optimization.Mathematical Programming,2016,156(1):433-484
[156]Mahajan D,Keerthi S S,Sundararajan S.A distributed block coordinate descent method for trainingá1regularized linear classifiers.arXiv preprint arXiv:1405.4544,2014
[157]Kang D,Lim W,Shin K,et al.Data/feature distributed stochastic coordinate descent for logistic regression//Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management.Shanghai,China,2014:1269-1278
[158]Boyd S,Parikh N,Chu E,et al.Distributed optimization and statistical learning via the alternating direction method of multipliers.Foundations and Trends in Machine Learning,2011,3(1):1-122
[2]Hosmer Jr D W,Lemeshow S.Applied Logistic Regression.USA New Jersey:John Wiley&Sons,2004
[3]Yuan M,Lin Y.Model selection and estimation in regression with grouped variables.Journal of the Royal Statistical Society:Series B(Statistical Methodology),2006,68(1):49-67
[4]Engelhardt B E,Adams R P.Bayesian structured sparsity from Gaussian fields.arXiv preprint arXiv:1407.2235,2014
[5]Ayaz U,Dirksen S,Rauhut H.Uniform recovery of fusion frame structured sparse signals.arXiv preprint arXiv:1407.7680,2014
[6]Doshi-Velez F,Wallace B,Adams R.Graph-sparse LDA:A topic model with structured sparsity.arXiv preprint arXiv:1410.4510,2014
[7]Zhang S,Qian H,Zhang Z.A nonconvex approach for structured sparse learning.arXiv preprint arXiv:1503.02164,2015
[8]Chadwick J N,Bindel D S.An efficient solver for sparse linear systems based on rank-structured cholesky factorization.arXiv preprint arXiv:1507.05593,2015
[9]Qiu C,Vaswani N,Lois B.Recursive robust PCA or recursive sparse recovery in large but structured noise.IEEE Transactions on Information Theory,2014,60(8):5007-5039
[10]Oymak S,Jalali A,Fazel M.Simultaneously structured models with application to sparse and low-rank matrices.IEEE Transactions on Information Theory,2015,61(5):2886-2908
[11]Wimalajeewa T,Eldar Y C,Varshney P K.Subspace recovery from structured union of subspaces.IEEE Transactions on Information Theory,2015,61(4):2101-2114
[12]Soni A,Haupt J.On the fundamental limits of recovering tree sparse vectors from noisy linear measurements.IEEE Transactions on Information Theory,2014,60(1):133-149
[13]Sun X,Qu Q,Nasrabadi N M.Structured priors for sparserepresentation-based hyperspectral image classification.IEEE Geoscience and Remote Sensing Letters,2014,11(7):1235-1239
[14]Kyrillidis A,Baldassarre L,El Halabi M.Structured sparsity:Discrete and convex approaches//Boche H,Calderbank R,Kutyniok G,Vybíral J eds.Compressed Sensing and its Applications.Springer International Publishing,Switzerland:Lausanne,2015:341-387
[15]Vila J P.Empirical-Bayes Approaches to Recovery of Structured Sparse Signals via Approximate Message Passing[Ph.D.dissertation].The Ohio State University,Ohio,USA,2015
[16]Bai T,Li Y.Robust visual tracking using flexible structured sparse representation.IEEE Transactions on Industrial Informatics,2014,10(1):538-547
[17]Hegde C,Indyk P,Schmidt L.A nearly-linear time framework for graph-structured sparsity//Proceedings of the 32nd International Conference on Machine Learning(ICML 2015).Lille,France,2015:928-937
[18]Huang J,Zhang T.The benefit of group sparsity.The Annals of Statistics,2010,38(4):1978-2004
[19]Sra S.Fast projections onto mixed-norm balls with applications.Data Mining and Knowledge Discovery,2012,25(2):358-377
[20]Lazzaro D,Montefusco L B,Papi S.Blind cluster structured sparse signal recovery:A nonconvex approach.Signal Processing,2015,109(4):212-225
[21]Vogt J,Roth V.A complete analysis of the L1,pGroupLasso.arXiv preprint arXiv:1206.4632,2012
[22]Rakotomamonjy A,Flamary R,Gasso G,Stephane C.Lp-Lq penalty for sparse linear and sparse multiple kernel multi-task learning.IEEE Transactions on Neural Networks,2011,22(8):1307-1320
[23]Vogt J E,Roth V.The group-Lasso:l1,∞regularization versus l1,2 regularization//Proceedings of the 32nd DAGM Conference on Pattern Recognition.Darmstadt,Germany:Springer-Verlag,2010:252-261
[24]Obozinski G,Jacob L,Vert J P.Group Lasso with overlaps:The latent Group Lasso approach.arXiv preprint arXiv:1110.0413,2011
[25]Jenatton R,Audibert J Y,Bach F.Structured variable selection with sparsity-inducing norms.The Journal of Machine Learning Research,2011,12(1):2777-2824
[26]Percival D.Structured,sparse aggregation.Journal of the American Statistical Association,2012,107(498):814-823
[27]Wang H,Leng C.A note on adaptive Group Lasso.Computational Statistics and Data Analysis,2008,52(12):5277-5286
[28]Wei F,Huang J.Consistent group selection in highdimensional linear regression.Bernoulli,2010,16(4):1369-1384
[29]Meier L,van de Geer S,Bühlmann P.The Group Lasso for logistic regression.Journal of the Royal Statistical Society:Series B(Statistical Methodology),2008,70(1):53-71
[30]Wang L,Chen G,Li H.Group SCAD regression analysis for microarray time course gene expression data.Bioinformatics,2007,23(12):1486-1494
[31]Fan J,Li R.Variable selection via nonconcave penalized likelihood and its oracle properties.Journal of the American Statistical Association,2001,96(456):1348-1360
[32]Huang J,Ma S,Xie H,Zhang C H.A group bridge approach for variable selection.Biometrika,2009,96(2):339-355
[33]Fu W J.Penalized regressions:The bridge versus the Lasso.Journal of Computational and Graphical Statistics,1998,7(3):397-416
[34]Huang J,Breheny P,Ma S.A selective review of group selection in high-dimensional models.Statistical Science,2012,27(4):481-499
[35]Zhang C H,Zhang S S.Confidence intervals for low dimensional parameters in high dimensional linear models.Journal of the Royal Statistical Society,2014,76(1):217-242
[36]Huang J,Ma S,Xie H,Zhang C H.Amendments and corrections:A group bridge approach for variable selection.Biometrika,2009,96(4):1024-1024
[37]Breheny P.The group exponential lasso for bi-level variable selection.Biometrics,2015,71(3):1-10
[38]Friedman J,Hastie T,Tibshirani R.A note on the Group Lasso and a sparse Group Lasso.arXiv preprint arXiv:1001.0736,2010
[39]Yuan X T,Liu X,Yan S.Visual classification with multitask joint sparse representation.IEEE Transactions on Image Processing,2012,21(10):4349-4360
[40]Gao S,Chia L T,Tsang I W H.Multi-layer group sparse coding—For concurrent image classification and annotation//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,USA,2011:2809-2816
[41]Seetharaman I.Consistent Bi-Level Variable Selection via Composite Group Bridge Penalized Regression[Ph.D.dissertation].Kansas State Univesity,Kansas,USA,2013
[42]Liu M,Zhang D,Yap P T.Tree-guided sparse coding for brain disease classification//Proceedings of the Medical Image Computing and Computer-Assisted Intervention(MICCAI2012).Berlin Heidelberg,Germany:Springer,2012:239-247
[43]Liu J,Ye J.Moreau-Yosida regularization for grouped tree structure learning//Proceedings of the Advances in Neural Information Processing Systems 23:24th Annual Conference on Neural Information Processing Systems.Vancouver,Canada,2010:1459-1467
[44]Kim S,Xing E P.Tree-guided Group Lasso for multi-task regression with structured sparsity//Proceedings of the 27th International Conference on Machine Learning.Haifa,Israel,2010:543-550
[45]Turlach B A,Venables W N,Wright S J.Simultaneous variable selection.Technometrics,2005,47(3):349-363
[46]Friedman J,Hastie T,Tibshirani R.Sparse inverse covariance estimation with the graphical lasso.Biostatistics,2008,9(3):432-441
[47]Olshausen B A,Field D J.Emergence of simple-cell receptive field properties by learning a sparse code for natural images.Nature,1996,381(6):607-609
[48]Olshausen B A,Field D J.Sparse coding with an overcomplete basis set:A strategy employed by V1?Vision Research,1997,37(23):3311-3325
[49]Olshausen B A,Field D J.How close are we to understanding V1?Neural Computation,2005,17(8):1665-1699
[50]Simoncelli E P,Olshausen B A.Natural image statistics and neural representation.Annual Review of Neuroscience,2001,24(1):1193-1216
[51]Jenatton R,Obozinski G,Bach F.Structured sparse principal component analysis//Proceedings of the 13th International Conference on Artificial Intelligence and Statistics(AISTATS2010).Chia Laguna Resort,Sardinia,Italy:JMLR Proceedings,2010:366-373
[52]Ramirez I,Sprechmann P,Sapiro G.Classification and clustering via dictionary learning with structured incoherence and shared features//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,USA,2010:3501-3508
[53]Yu G,Sapiro G,Mallat S.Solving inverse problems with piecewise linear estimators:From Gaussian mixture models to structured sparsity.IEEE Transactions on Image Processing,2012,21(5):2481-2499
[54]Mairal J,Bach F,Ponce J,Sapiro G.Online learning for matrix factorization and sparse coding.Journal of Machine Learning Research,2010,11(1):19-60
[55]Mairal J,Jenatton R,Obozinski G,Bach F.Network flow algorithms for structured sparsity//Proceedings of the Advances in Neural Information Processing Systems 23:24th Annual Conference on Neural Information Processing Systems.Vancouver,Canada,2010:1558-1566
[56]Mairal J,Jenatton R,Obozinski G,Bach F.Convex and network flow optimization for structured sparsity.Journal of Machine Learning Research,2011,12(9):2681-2720
[57]Mairal J,Bach F,Ponce J.Task-driven dictionary learning.IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(4):791-804
[58]Zhu S-C,Guo C-E,Wang Y,Xu Z.What are textons?International Journal of Computer Vision,2005,62(1-2):121-143
[59]Zeiler M D,Krishnan D,Taylor G W.Deconvolutional networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,USA,2010:2528-2535
[60]Zeiler M D,Taylor G W,Fergus R.Adaptive deconvolutional networks for mid and high level feature learning//Proceedings of the IEEE International Conference on Computer Vision(ICCV 2011).Barcelona,Spain,2011:2018-2025
[61]Bo L,Ren X,Fox D.Multipath sparse coding using hierarchical matching pursuit//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,USA,2013:660-667
[62]Dalal N,Triggs B.Histograms of oriented gradients for human detection//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,USA,2005:886-893
[63]Ren X,Ramanan D.Histograms of sparse codes for object detection//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,2013:3246-3253
[64]Rigamonti R,Sironi A,Lepetit V,Fua P.Learning separable filters//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).New Jersey,USA,2013:2754-2761
[65]Kavukcuoglu K,Sermanet P,Boureau Y-L,et al.Learning convolutional feature hierarchies for visual recognition//Proceedings of the Advances in Neural Information Processing Systems 23:24th Annual Conference on Neural Information Processing Systems.Vancouver,Canada,2010:1090-1098
[66]Garrigues P,Olshausen B A.Group sparse coding with a Laplacian scale mixture prior//Proceedings of the Advances in Neural Information Processing Systems 23:24th Annual Conference on Neural Information Processing Systems.Vancouver,Canada,2010:676-684
[67]Gregor K,Szlam A,LeCun Y.Structured sparse coding via lateral inhibition//Proceedings of the 28th International Conference on Machine Learning(ICML 2011).Bellevue,Washington,USA:Omnipress,2011:1116-1124
[68]Raman S,Fuchs T J,Wild P J,et al.The Bayesian groupLasso for analyzing contingency tables//Proceedings of the26th Annual International Conference on Machine Learning.Montreal,Canada,2009:881-888
[69]Chandran M.Analysis of Bayesian Group-Lasso in Regression Models[Ph.D.dissertation].University of Florida,Florida,USA,2011
[70]Huang J,Zhang T,Metaxas D.Learning with structured sparsity.Journal of Machine Learning Research,2011,12(11):3371-3412
[71]Bach F R.Structured sparsity-inducing norms through submodular functions//Proceedings of the Advances in Neural Information Processing Systems 23:24th Annual Conference on Neural Information Processing Systems.Vancouver,Canada,2010:118-126
[72]Micchelli C A,Morales J M,Pontil M.Regularizers for structured sparsity.Advances in Computational Mathematics,2013,38(3):455-489
[73]Bach F R.Consistency of the Group Lasso and multiple kernel learning.The Journal of Machine Learning Research,2008,9(1):1179-1225
[74]Sharma D B,Bondell H D,Zhang H H.Consistent group identification and variable selection in regression with correlated predictors.Journal of Computational and Graphical Statistics,2013,22(2):319-340
[75]Mairal J,Yu B.Supervised feature selection in graphs with path coding penalties and network flows.Journal of Machine Learning Research,2013,14(1):2449-2485
[76]Rosset S,Zhu J.Piecewise linear regularized solution paths.The Annals of Statistics,2007,35(3):1012-1030
[77]Lv X,Bi G,Wan C.The group lasso for stable recovery of block-sparse signal representations.IEEE Transactions on Signal Processing,2011,59(4):1371-1382
[78]Jacob L,Obozinski G,Vert J P.Group Lasso with overlap and graph lasso//Proceedings of the 26th Annual International Conference on Machine Learning(ICML 2009).Montreal,Canada,2009:433-440
[79]Vincent M,Hansen N R.Sparse group lasso and high dimensional multinomial classification.Computational Statistics&Data Analysis,2014,71(3):771-786
[80]Geng Z,Wang S,Yu M,et al.Group variable selection via convex log-exp-sum penalty with application to a breast cancer survivor study.Biometrics,2015,71(1):53-62
[81]Wu F,Yuan Y,Rui Y.Annotating web images using NOVA:NOn-conVex group spArsity//Proceedings of the20th ACM Multimedia Conference(MM’12).Nara,Japan,2012:509-518
[82]Zeng L,Xie J.Group variable selection via SCAD-L2.Statistics,2014,48(1):49-66
[83]Yuan M,Lin Y.Model selection and estimation in the Gaussian graphical model.Biometrika,2007,94(1):19-35
[84]Simon N,Friedman J,Hastie T.A sparse-Group Lasso.Journal of Computational and Graphical Statistics,2013,22(2):231-245
[85]Zhou H,Sehl M,Sinsheimer J,Lange K.Association screening of common and rare genetic variants by penalized regression.Bioinformatics,2010,26(19):2375-2382
[86]Sohn K A,Kim S.Joint estimation of structured sparsity and output structure in multiple-output regression via inversecovariance regularization//Proceedings of the 15th International Conference on Artificial Intelligence and Statistics(AISTATS2012).La Palma,Canary Islands,2012:1081-1089
[87]Swirszcz G,Lozano A C.Multi-level lasso for sparse multitask regression//Proceedings of the 29th International Conference on Machine Learning(ICML 2012).Edinburgh,UK,2012:361-368
[88]Wang J,Ye J.Two-layer feature reduction for Sparse-Group Lasso via decomposition of convex sets//Proceedings of the Advances in Neural Information Processing Systems 27:Annual Conference on Neural Information Processing Systems2014.Montreal,Canada,2014:2132-2140
[89]Caner M,Han X.Selecting the correct number of factors in approximate factor models:The large panel case with group bridge estimators.Journal of Business&Economic Statistics,2014,32(3):359-374
[90]Breheny P,Huang J.Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors.Statistics and Computing,2015,25(2):173-187
[91]Chen X,Lin Q,Kim S,et al.Smoothing proximal gradient method for general structured sparse regression.The Annals of Applied Statistics,2010,6(2):719-752
[92]Nesterov Y.Smooth minimization of non-smooth functions.Mathematical Programming,2005,103(1):127-152
[93]Jenatton R,Mairal J,Obozinski G,Bach F.Proximal methods for hierarchical sparse coding.Journal of Machine Learning Research,2011,12(7):2297-2334
[94]Beck A,Teboulle M.A fast iterative shrinkage-thresholding algorithm for linear inverse problems.SIAM Journal on Imaging Sciences,2009,2(1):183-202
[95]Breheny P,Huang J.Penalized methods for bi-level variable selection.Statistics and Its Interface,2009,2(3):369-380
[96]Tao Qing,Gao Qian-Kun,Jiang Ji-Yuan,Chu De-Jun.Survey of solving the optimization problems for sparse learning.Journal of Software,2013,24(11):2498-2507(in Chinese)(陶卿,高乾坤,姜纪远,储德军.稀疏学习优化问题的求解综述.软件学报,2013,24(11):2498-2507)
[97]Rish I,Grabarnik G.Sparse Modeling:Theory,Algorithms,and Applications.New York,USA:CRC Press,2014
[98]Rao N,Nowak R,Cox C.Classification with the sparse Group Lasso.IEEE Transactions on Signal Processing,2016,64(2):448-463
[99]Wahab S,Zakaria M N,Sidek D,et al.Evaluation of auditory hallucinations in patients with schizophrenia:A validation study of the Malay version of Psychotic Symptom Rating Scales(PSYRATS).Psychiatry Research,2015,228(3):462-467
[100]Huang S,Li J,Ye J,et al.A sparse structure learning algorithm for Gaussian Bayesian network identification from high-dimensional data.IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(6):1328-1342
[101]Honorio J,Ortiz L,Samaras D,et al.Sparse and locally constant Gaussian graphical models//Proceedings of the Advances in Neural Information Processing Systems 22:23rd Annual Conference on Neural Information Processing Systems 2009.Vancouver,Canada:Curran Associates,2009:745-753
[102]Honorio J,Samaras D,Rish I,Cecchi G.Variable selection for Gaussian graphical models//Proceedings of the 15th International Conference on Artificial Intelligence and Statistics(AISTATS 2012).La Palma,Canary Islands:JMLR Proceedings,2012:538-546
[103]Klami A,Virtanen S,Leppaho E,et al.Group factor analysis.IEEE Transactions on Neural Networks and Learning Systems,2015,26(9):2136-2147
[104]Chatterjee S,Steinhaeuser K,Banerjee A.Sparse Group Lasso:Consistency and climate applications//Proceedings of the 12th SIAM International Conference on Data Mining.Anaheim,USA,2012:47-58
[105]Gao L,Zhou S,Guan J.Effectively classifying short texts by structured sparse representation with dictionary filtering.Information Sciences,2015,323(11):130-142
[106]Downey D,Bhagavatula C,Yang Y.Efficient methods for inferring large sparse topic hierarchies//Proceedings of the53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing.Beijing,China,2015:774-784
[107]Zheng M,Bu J,Chen C,et al.Graph regularized sparse coding for image representation.IEEE Transactions on Image Processing,2011,20(5):1327-1336
[108]Liu H,Yu H,Deng Z H.Multi-document summarization based on two-level sparse representation model//Proceedings of the 29th AAAI Conference on Artificial Intelligence.Austin,USA.2015:196-202
[109]Pnevmatikakis E A,Gao Y,Soudry D.A structured matrix factorization framework for large scale calcium imaging data analysis.arXiv preprint arXiv:1409.2903,2014
[110]Han Y,Wu F,Tian Q.Image annotation by input—output structural grouping sparsity.IEEE Transactions on Image Processing,2012,21(6):3066-3079
[111]Moayedi F,Azimifar Z,Boostani R.Structured sparse representation for human action recognition.Neurocomputing,2015,161(8):38-46
[112]Ly N H,Du Q,Fowler J E.Sparse graph-based discriminant analysis for hyperspectral imagery.IEEE Transactions on Geoscience and Remote Sensing,2014,52(7):3872-3884
[113]Hegde C,Indyk P,Schmidt L.A fast approximation algorithm for tree-sparse recovery//Proceeding of the IEEE International Symposium on Information Theory(ISIT).New Jersey,USA,2014:1842-1846
[114]Bui H Q,La C N H,Do M N.A fast tree-based algorithm for compressed sensing with sparse-tree prior.Signal Processing,2015,108(3):628-641
[115]Chen H,Wan Q,Fan R.Double-constraint flexible tree search-based orthogonal matching pursuit for DOA estimation using dynamic sensor arrays.International Journal of Electronics,2015,103(5):1-9
[116]Castro R M,Tanczos E.Adaptive sensing for estimation of structured sparse signals.IEEE Transactions on Information Theory,2015,61(4):2060-2080
[117]Peleg T,Eldar Y C,Elad M.Exploiting statistical dependencies in sparse representations for signal recovery.IEEE Transactions on Signal Processing,2012,60(5):2286-2303
[118]Gui J,Tao D,Sun Z.Group sparse multiview patch alignment framework with view consistency for image classification.IEEE Transactions on Image Processing,2014,23(7):3126-3137
[119]Yang Y,Zou H.A fast unified algorithm for solving grouplasso penalize learning problems.Statistics and Computing,2015,25(6):1129-1141
[120]Scheetz T E,Kim K Y A,Swiderski R E,et al.Regulation of gene expression in the mammalian eye and its relevance to eye disease.Proceedings of the National Academy of Sciences,2006,103(39):14429-14434
[121]Segal M R,Dahlquist K D,Conklin B R.Regression approaches for microarray data analysis.Journal of Computational Biology,2003,10(6):961-980
[122]SФbФS,AlmФy T,AarФe J,et al.ST-PLS:A multidirectional nearest shrunken centroid type classifier via PLS.Journal of Chemometrics,2008,22(1):54-62
[123]Graham K,de Las Morenas A,Tripathi A,et al.Gene expression in histologically normal epithelium from breast cancer patients and from cancer-free prophylactic mastectomy patients shares a similar profile.British Journal of Cancer,2010,102(8):1284-1293
[124]Alon U,Barkai N,Notterman D A,et al.Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays.Proceedings of the National Academy of Sciences,1999,96(12):6745-6750
[125]Singh D,Febbo P G,Ross K,et al.Gene expression correlates of clinical prostate cancer behavior.Cancer Cell,2002,1(2):203-209
[126]Gorman R P,Sejnowski T J.Analysis of hidden units in a layered network trained to classify sonar targets.Neural Networks,1988,1(1):75-89
[127]Zhang T.Analysis of multi-stage convex relaxation for sparse regularization.The Journal of Machine Learning Research,2010,11(3):1081-1107
[128]Zhang T.Multi-stage convex relaxation for feature selection.arXiv preprint arXiv:1106.0565,2011
[129]Candes E J,Wakin M B,Boyd S P.Enhancing sparsity by reweighted L1minimization.Journal of Fourier Analysis and Applications,2008,14(5-6):877-905
[130]Geman D,Yang C.Nonlinear image recovery with halfquadratic regularization.IEEE Transactions on Image Processing,1995,4(7):932-946
[131]Foucart S,Lai M J.Sparsest solutions of underdetermined linear systems via Lqminimization for 0
[133]van de Geer S A,Bühlmann P.On the conditions used to prove oracle results for the Lasso.Electronic Journal of Statistics,2009,3(11):1360-1392
[134]Zhang T.Some sharp performance bounds for least squares regression with L1regularization.The Annals of Statistics,2009,37(5):2109-2144
[135]Ye F,Zhang C H.Rate minimaxity of the Lasso and dantzig selector for the Lqloss in Lrballs.The Journal of Machine Learning Research,2010,11:3519-3540
[136]Chandrasekaran V,Parrilo P A,Willsky A S.Latent variable graphical model selection via convex optimization.The Annals of Statistics,2012,40(4):1935-1967
[137]Ma S,Xue L,Zou H.Alternating direction methods for latent variable gaussian graphical model selection.Neural Computation,2013,25(8):2172-2198
[138]Lauritzen S,Meinshausen N.Discussion:Latent variable graphical model selection via convex optimization.The Annals of Statistics,2012,40(4):1973-1977
[139]Shojaie A,Michailidis G.Penalized likelihood methods for estimation of sparse high-dimensional directed acyclic graphs.Biometrika,2010,97(3):519-538
[140]Fu F,Zhou Q.Learning sparse causal Gaussian networks with experimental intervention:Regularization and coordinate descent.Journal of the American Statistical Association,2013,108(501):288-300
[141]Ravikumar P,Wainwright M J,Lafferty J D.High-dimensional Ising model selection using1-regularized logistic regression.The Annals of Statistics,2010,38(3):1287-1319
[142]Wainwright M J,Ravikumar P,Lafferty J D.High-dimensional graphical model selection using L1-regularized logistic regression//Proceedings of the 20th Annual Conference on Neural Information Processing Systems.Vancouver,Canada:MIT Press,2006:1465-1472
[143]Liu H,Han F,Yuan M.High-dimensional semiparametric Gaussian copula graphical models.The Annals of Statistics,2012,40(4):2293-2326
[144]Allen G I,Liu Z.A log-linear graphical model for inferring genetic networks from high-throughput sequencing data//Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine(BIBM).Philadelphia,USA,2012:1-6
[145]Yang E,Ravikumar P D,Allen G I.Graphical models via generalized linear models//Proceedings of the Advances in Neural Information Processing Systems 25:26th Annual Conference on Neural Information Processing Systems.Lake Tahoe,USA:Curran Associates,2012:1367-1375
[146]Liu Q,Ihler A T.Learning scale free networks by reweighted1 regularization//Proceedings of the International Conference on Artificial Intelligence and Statistics.Fort Lauderdale,USA,2011:40-48
[147]Danaher P,Wang P,Witten D M.The joint graphical lasso for inverse covariance estimation across multiple classes.Journal of the Royal Statistical Society:Series B(Statistical Methodology),2013,76(2):373-397
[148]Yang S,Pan Z,Shen X.Fused multiple graphical lasso.arXiv preprint arXiv:1209.2139,2012
[149]Zhang B,Wang Y.Learning structural changes of Gaussian graphical models in controlled experiments.arXiv preprint arXiv:1203.3532,2012
[150]Honorio J,Samaras D.Multi-task learning of Gaussian graphical models//Proceedings of the 27th International Conference on Machine Learning.Haifa,Israel:Omnipress,2010:447-454
[151]Committee on the Analysis of Massive Data et al.Frontiers in Massive Data Analysis.Washington,USA:National Academies Press,2013
[152]Franke B,Plante J F,Roscher R,et al.Statistical Inference,Learning and Models in Big Data.arXiv preprint arXiv:1509.02900,2015
[153]Hammer B,He H,Martinetz T.Learning and modeling big data//Proceedings of the 22nd European Symposium on Artificial Neural Networks,Computational Intelligence and Machine Learning.Bruges,Belgium,2014:105-110
[154]Richtárik P,Taká M.Distributed coordinate descent method for learning with big data.arXiv preprint arXiv:1310.2059,2013
[155]Richtárik P,Taká M.Parallel coordinate descent methods for big data optimization.Mathematical Programming,2016,156(1):433-484
[156]Mahajan D,Keerthi S S,Sundararajan S.A distributed block coordinate descent method for trainingá1regularized linear classifiers.arXiv preprint arXiv:1405.4544,2014
[157]Kang D,Lim W,Shin K,et al.Data/feature distributed stochastic coordinate descent for logistic regression//Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management.Shanghai,China,2014:1269-1278
[158]Boyd S,Parikh N,Chu E,et al.Distributed optimization and statistical learning via the alternating direction method of multipliers.Foundations and Trends in Machine Learning,2011,3(1):1-122