Dimensionality Reduction for Classification with High-dimensional Data

High-dimensional data refers to data with a large number of variables. Classifying these data is a difficult problem because the enormous number of variables poses challenges to conventional classification methods and renders many classical techniques impractical. A natural solution is to add a dimensionality reduction step before a classification technique is applied. We Propose three methods to deal with this problem: a simulated annealing (SA) based method, a multivariate adaptive stochastic search (MASS) method, and a functional adaptive classification (FAC) method. The third method considers functional predictors. They all utilize stochastic search algorithms to select a handful of optimal transformation directions from a large number of random directions in each iteration. These methods are designed to mimic variable selection type methods, such as the Lasso, or variable combination methods, such as PCA, or a method that combines the two approaches. We demonstrate the strengths of our methods on an extensive range of simulation and real-world studies.