Dimensionality reduction<\/strong> is an unsupervised learning technique.<\/p>\n Nevertheless, it can be used as a data transform pre-processing step for machine learning algorithms on classification and regression predictive modeling datasets with supervised learning algorithms.<\/p>\n There are many dimensionality reduction algorithms to choose from and no single best algorithm for all cases. Instead, it is a good idea to explore a range of dimensionality reduction algorithms and different configurations for each algorithm.<\/p>\n In this tutorial, you will discover how to fit and evaluate top dimensionality reduction algorithms in Python.<\/p>\n After completing this tutorial, you will know:<\/p>\n Discover data cleaning, feature selection, data transforms, dimensionality reduction and much more in my new book<\/a>, with 30 step-by-step tutorials and full Python source code.<\/p>\n Let’s get started.<\/p>\n Dimensionality Reduction Algorithms With Python This tutorial is divided into three parts; they are:<\/p>\n Dimensionality reduction refers to techniques for reducing the number of input variables in training data.<\/p>\n When dealing with high dimensional data, it is often useful to reduce the dimensionality by projecting the data to a lower dimensional subspace which captures the “essence” of the data. This is called dimensionality reduction.<\/p>\n<\/blockquote>\n — Page 11, Machine Learning: A Probabilistic Perspective<\/a>, 2012.<\/p>\n High-dimensionality might mean hundreds, thousands, or even millions of input variables.<\/p>\n Fewer input dimensions often means correspondingly fewer parameters or a simpler structure in the machine learning model, referred to as degrees of freedom<\/a>. A model with too many degrees of freedom is likely to overfit the training dataset and may not perform well on new data.<\/p>\n It is desirable to have simple models that generalize well, and in turn, input data with few input variables. This is particularly true for linear models where the number of inputs and the degrees of freedom of the model are often closely related.<\/p>\n Dimensionality reduction is a data preparation technique performed on data prior to modeling. It might be performed after data cleaning and data scaling and before training a predictive model.<\/p>\n … dimensionality reduction yields a more compact, more easily interpretable representation of the target concept, focusing the user’s attention on the most relevant variables.<\/p>\n<\/blockquote>\n — Page 289, Data Mining: Practical Machine Learning Tools and Techniques<\/a>, 4th edition, 2016.<\/p>\n As such, any dimensionality reduction performed on training data must also be performed on new data, such as a test dataset, validation dataset, and data when making a prediction with the final model<\/a>.<\/p>\n<\/p>\n Take my free 7-day email crash course now (with sample code).<\/p>\n Click to sign-up and also get a free PDF Ebook version of the course.<\/p>\n\n
![\"Dimensionality](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/07\/Dimensionality-Reduction-Algorithms-With-Python.jpg\")
<\/p>\n
Photo by Bernard Spragg. NZ<\/a>, some rights reserved.<\/p>\n<\/div>\nTutorial Overview<\/h2>\n
\n
\n
Dimensionality Reduction<\/h2>\n
\n
\n
Want to Get Started With Data Preparation?<\/h3>\n