Many machine learning models perform better when input variables are carefully transformed or scaled prior to modeling.<\/p>\n
It is convenient, and therefore common, to apply the same data transforms, such as standardization and normalization, equally to all input variables. This can achieve good results on many problems. Nevertheless, better results may be achieved by carefully selecting which data transform to apply to each input variable<\/strong> prior to modeling.<\/p>\n In this tutorial, you will discover how to apply selective scaling of numerical input variables.<\/p>\n After completing this tutorial, you will know:<\/p>\n Discover data cleaning, feature selection, data transforms, dimensionality reduction and much more in my new book<\/a>, with 30 step-by-step tutorials and full Python source code.<\/p>\n Let’s get started.<\/p>\n How to Selectively Scale Numerical Input Variables for Machine Learning This tutorial is divided into three parts; they are:<\/p>\n As the basis of this tutorial, we will use the so-called “diabetes” dataset that has been widely studied as a machine learning dataset since the 1990s.<\/p>\n The dataset classifies patients’ data as either an onset of diabetes within five years or not. There are 768 examples and eight input variables. It is a binary classification problem.<\/p>\n You can learn more about the dataset here:<\/p>\n No need to download the dataset; we will download it automatically as part of the worked examples that follow.<\/p>\n Looking at the data, we can see that all nine input variables are numerical.<\/p>\n We can load this dataset into memory using the Pandas library.<\/p>\n The example below downloads and summarizes the diabetes dataset.<\/p>\n Running the example first downloads the dataset and loads it as a DataFrame.<\/p>\n The shape of the dataset is printed, confirming the number of rows, and nine variables, eight input, and one target.<\/p>\n Finally, a plot is created showing a histogram for each variable in the dataset.<\/p>\n This is useful as we can see that some variables have a Gaussian or Gaussian-like distribution (1, 2, 5) and others have an exponential-like distribution (0, 3, 4, 6, 7). This may suggest the need for different numerical data transforms for the different types of input variables.<\/p>\n Histogram of Each Variable in the Diabetes Classification Dataset<\/p>\n<\/div>\n Now that we are a little familiar with the dataset, let’s try fitting and evaluating a model on the raw dataset.<\/p>\n We will use a logistic regression model as they are a robust and effective linear model for binary classification tasks. We will evaluate the model using repeated stratified k-fold cross-validation, a best practice, and use 10 folds and three repeats.<\/p>\n The complete example is listed below.<\/p>\n Running the example evaluates the model and reports the mean and standard deviation accuracy for fitting a logistic regression model on the raw dataset.<\/p>\n Your specific results may differ given the stochastic nature of the learning algorithm, the stochastic nature of the evaluation procedure, and differences in precision across machines and library versions. Try running the example a few times.<\/p>\n In this case, we can see that the model achieved an accuracy of about 76.8 percent.<\/p>\n Now that we have established a baseline in performance on the dataset, let’s see if we can improve the performance using data scaling.<\/p>\n<\/p>\n Take my free 7-day email crash course now (with sample code).<\/p>\n Click to sign-up and also get a free PDF Ebook version of the course.<\/p>\n\n
![\"How](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/07\/How-to-Selectively-Scale-Numerical-Input-Variables-for-Machine-Learning.jpg\")
<\/p>\n
Photo by Marco Verch<\/a>, some rights reserved.<\/p>\n<\/div>\nTutorial Overview<\/h2>\n
\n
\n
\n
Diabetes Numerical Dataset<\/h2>\n
\n
6,148,72,35,0,33.6,0.627,50,1\r\n1,85,66,29,0,26.6,0.351,31,0\r\n8,183,64,0,0,23.3,0.672,32,1\r\n1,89,66,23,94,28.1,0.167,21,0\r\n0,137,40,35,168,43.1,2.288,33,1\r\n...<\/pre>\n
# load and summarize the diabetes dataset\r\nfrom pandas import read_csv\r\nfrom pandas.plotting import scatter_matrix\r\nfrom matplotlib import pyplot\r\n# Load dataset\r\nurl = \"https:\/\/raw.githubusercontent.com\/jbrownlee\/Datasets\/master\/pima-indians-diabetes.csv\"\r\ndataset = read_csv(url, header=None)\r\n# summarize the shape of the dataset\r\nprint(dataset.shape)\r\n# histograms of the variables\r\ndataset.hist()\r\npyplot.show()<\/pre>\n
(768, 9)<\/pre>\n
![\"Histogram](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/06\/Histogram-of-Each-Variable-in-the-Diabetes-Classification-Dataset.png\")
<\/p>\n# evaluate a logistic regression model on the raw diabetes dataset\r\nfrom numpy import mean\r\nfrom numpy import std\r\nfrom pandas import read_csv\r\nfrom sklearn.preprocessing import LabelEncoder\r\nfrom sklearn.model_selection import cross_val_score\r\nfrom sklearn.model_selection import RepeatedStratifiedKFold\r\nfrom sklearn.linear_model import LogisticRegression\r\n# load dataset\r\nurl = 'https:\/\/raw.githubusercontent.com\/jbrownlee\/Datasets\/master\/pima-indians-diabetes.csv'\r\ndataframe = read_csv(url, header=None)\r\ndata = dataframe.values\r\n# separate into input and output elements\r\nX, y = data[:, :-1], data[:, -1]\r\n# minimally prepare dataset\r\nX = X.astype('float')\r\ny = LabelEncoder().fit_transform(y.astype('str'))\r\n# define the model\r\nmodel = LogisticRegression(solver='liblinear')\r\n# define the evaluation procedure\r\ncv = RepeatedStratifiedKFold(n_splits=10, n_repeats=3, random_state=1)\r\n# evaluate the model\r\nm_scores = cross_val_score(model, X, y, scoring='accuracy', cv=cv, n_jobs=-1)\r\n# summarize the result\r\nprint('Accuracy: %.3f (%.3f)' % (mean(m_scores), std(m_scores)))<\/pre>\nAccuracy: 0.768 (0.040)<\/pre>\n
Want to Get Started With Data Preparation?<\/h3>\n