Sparse inverse covariance estimation
- 1. Sparse inverse covariance estimation with graphical lasso Ayush Singh CCIS, Northeastern University
- 2. Motivation ● All matrices based on gaussian dist. have mean and covariance matrix ● Covariance is zero if elements are independent ● Inverse covariance matrix would have zeros making its sparse sometimes ● Density functions not suitable for Sparse matrices ● Estimate covariance and apply L1 to reduce to zero
- 3. Current Work ● L1 has been proposed for estimation of sparse undirected graphical model ● Meinshausen and Bühlmann (MB) estimate a sparse graphical model by fitting a lasso model to each variable, using the others as predictors. ● MB approach consistently estimates nonzero in Inverse covariance matrix ● R Packages: COVSEL based on work of Banerjee and others ● Exact maximization of the L1 -penalized log-likelihood is another way ● Some used Interior-point optimization methods after proving convexity ● Banerjee and others proves MB approach does not yield the maximum likelihood estimator.
- 4. Intuition ● Instead use blockwise coordinate descent algorithms to solve lasso problem ● Graphical Lasso : L1 Regularization using Coordinate Descent ● Blockwise: Partition keeping target always last and estimate based on r.v. ● Update original matrix from results of previous step ● Repeat above until convergence: change in estimate , t = 0.001 ● Resulting matrix is sparse so computations are cheap
- 5. Results ● Birth of a new R package: GLASSO ● 30-4000x faster vs COVSEL and 2-10x slower vs MB approach on a sample 1k node and 500k params graph ● The computation time depends strongly on the value of ρ
- 6. Future ● Allow application to large datasets N with thousands parameters p ● Can even be used on datasets with p > N ● Sensitive to ρ, should be chosen empirically and with due diligence Thanks for your attention!