Supervised clustering of variables

Advances in Data Analysis and Classification volume 10, pages85–101(2016)Cite this article

Abstract

In predictive modelling, highly correlated predictors lead to unstable models that are often difficult to interpret. The selection of features, or the use of latent components that reduce the complexity among correlated observed variables, are common strategies. Our objective with the new procedure that we advocate here is to achieve both purposes: to highlight the group structure among the variables and to identify the most relevant groups of variables for prediction. The proposed procedure is an iterative adaptation of a method developed for the clustering of variables around latent variables (CLV). Modification of the standard CLV algorithm leads to a supervised procedure, in the sense that the variable to be predicted plays an active role in the clustering. The latent variables associated with the groups of variables, selected for their “proximity” to the variable to be predicted and their “internal homogeneity”, are progressively added in a predictive model. The features of the methodology are illustrated based on a simulation study and a real-world application.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

34,95 €

Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Barnes RJ, Dhanoa MS, Lister SJ (1989) Standard normal variate transformation and de-trending of near-infrared diffuse reflectance spectra. Appl Spectrosc 45:772–777

    Article  Google Scholar 

  2. Chun H, Keles S (2010) Sparse partial least squares regression for simultaneous dimension reduction and variable selection. J R Stat Soc B 72(1):3–25

    Article  MathSciNet  Google Scholar 

  3. Filzmoser P, Liebmann B, Varmuza K (2009) Repeated double cross validation. J Chemom 23:160–171

    Article  Google Scholar 

  4. Hastie T, Tibshirani R, Botstein D, Brown P (2001) Supervised harvesting of expression trees. Genom Biol 2(1):1–12

  5. Jolliffe IT, Trendafilov NT, Uddin M (2003) A modified principal component technique based on the lasso. J Comput Graph Stat 12:531–547

    Article  MathSciNet  Google Scholar 

  6. Le Cao KA, Rossouw D, Robert-Grani C, Besse P (2008) Sparse PLS: variable selection when integrating omics data. Stat Appl Genet Mol Biol 7(1): Art No 35

  7. Le Thi HA, Le HM, Nguyen VV, Dinh TP (2008) A DC programming approach for feature selection in support vector machines learning. Adv Data Anal Classif 2:259–278

    Article  MathSciNet  MATH  Google Scholar 

  8. Leardi R, Boggia R, Terrile M (1992) Genetic algorithms as a strategy for feature selection. J Chemom 6(5):267–281

    Article  Google Scholar 

  9. Naes T, Kowalski B (1989) Predicting sensory profiles from external instrumental measurements. Food Qual Prefer 1:135–147

    Article  Google Scholar 

  10. Park MY, Hastie T, Tibshirani R (2007) Averaged gene expressions for regression. Biostatistics 8(2):212–227

  11. Subedi S, Punzo A, Ingrassia S, McNicholas PD (2013) Clustering and classification via cluster-weighted factor analysers. Adv Data Anal Classif 7(1):5–40

    Article  MathSciNet  MATH  Google Scholar 

  12. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J Roy Stat Soc B 58(1):267–288

    MathSciNet  MATH  Google Scholar 

  13. Vichi M, Saporta G (2009) Clustering and disjoint principal component analysis. Comput Stat Data Anal 53:3194–3208

    Article  MathSciNet  MATH  Google Scholar 

  14. Vigneau E, Qannari E (2003) Clustering of variables around latent components. Commun Stat Simul Comput 32(4):1131–1150

    Article  MathSciNet  MATH  Google Scholar 

  15. Vigneau E, Thomas F (2012) Model calibration and feature selection for orange juice authentication by 1H NMR spectroscopy. Chemom Intell Lab 117:22–30

    Article  Google Scholar 

  16. Vigneau E, Sahmer K, Qannari EM, Bertrand D (2005) Clustering of variables to analyze spectral data. J Chemom 19(3):122–128

  17. Vigneau E, Endrizzi I, Qannari E (2011) Finding and explaining clusters of consumers using the CLV approach. Food Qual Pref 22(4):705–713

    Article  Google Scholar 

  18. Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J Roy Stat Soc B 67(3):301–320

  19. Zou H, Hastie T, Tibshirani R (2006) Sparse principal component analysis. J Comput Graph Stat 15:265–286

    Article  MathSciNet  Google Scholar 

Download references

Author information

Affiliations

  1. Sensometrics and Chemometrics Laboratory, LUNAM University, ONIRIS, Site de la Geraudiere, BP 82225, 44300, Nantes CEDEX 3, France

    Mingkun Chen & Evelyne Vigneau

Authors
  1. Mingkun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Evelyne Vigneau
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Evelyne Vigneau.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chen, M., Vigneau, E. Supervised clustering of variables. Adv Data Anal Classif 10, 85–101 (2016). https://doi.org/10.1007/s11634-014-0191-5

Download citation

Keywords

  • Prediction
  • Clustering of variables around latent variables (CLV)
  • Forward regression model
  • Sparse PLS regression

Mathematics Subject Classification

  • 62H30
  • 62J05