Missing values can cause problems when modeling classification and regression prediction problems with machine learning algorithms.<\/p>\n
A common approach is to replace missing values with a calculated statistic, such as the mean of the column. This allows the dataset to be modeled as per normal but gives no indication to the model that the row original contained missing values.<\/p>\n
One approach to address this issue is to include additional binary flag input features that indicate whether a row or a column contained a missing value that was imputed. This additional information may or may not be helpful to the model in predicting the target value.<\/p>\n
In this tutorial, you will discover how to add binary flags for missing values<\/strong> for modeling.<\/p>\n After completing this tutorial, you will know:<\/p>\n Discover data cleaning, feature selection, data transforms, dimensionality reduction and much more in my new book<\/a>, with 30 step-by-step tutorials and full Python source code.<\/p>\n Let’s get started.<\/p>\n Add Binary Flags for Missing Values for Machine Learning This tutorial is divided into three parts; they are:<\/p>\n The horse colic dataset describes medical characteristics of horses with colic and whether they lived or died.<\/p>\n There are 300 rows and 26 input variables with one output variable. It is a binary classification prediction task that involves predicting 1 if the horse lived and 2 if the horse died.<\/p>\n There are many fields we could select to predict in this dataset. In this case, we will predict whether the problem was surgical or not (column index 23), making it a binary classification problem.<\/p>\n The dataset has numerous missing values<\/a> for many of the columns where each missing value is marked with a question mark character (“?”).<\/p>\n Below provides an example of rows from the dataset with marked missing values.<\/p>\n You can learn more about the dataset here:<\/p>\n No need to download the dataset as we will download it automatically in the worked examples.<\/p>\n Marking missing values with a NaN (not a number) value in a loaded dataset using Python is a best practice.<\/p>\n We can load the dataset using the read_csv() Pandas<\/a> function and specify the “na_values<\/em>” to load values of ‘?’ as missing, marked with a NaN value.<\/p>\n The example below downloads the dataset, marks “?” values as NaN (missing) and summarizes the shape of the dataset.<\/p>\n Running the example downloads the dataset and reports the number of rows and columns, matching our expectations.<\/p>\n Next, we can evaluate a model on this dataset.<\/p>\n\n
![\"Add](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/07\/Add-Binary-Flags-for-Missing-Values-for-Machine-Learning.jpg\")
<\/p>\n
Photo by keith o connell<\/a>, some rights reserved.<\/p>\n<\/div>\nTutorial Overview<\/h2>\n
\n
Imputing the Horse Colic Dataset<\/h2>\n
2,1,530101,38.50,66,28,3,3,?,2,5,4,4,?,?,?,3,5,45.00,8.40,?,?,2,2,11300,00000,00000,2\r\n1,1,534817,39.2,88,20,?,?,4,1,3,4,2,?,?,?,4,2,50,85,2,2,3,2,02208,00000,00000,2\r\n2,1,530334,38.30,40,24,1,1,3,1,3,3,1,?,?,?,1,1,33.00,6.70,?,?,1,2,00000,00000,00000,1\r\n1,9,5290409,39.10,164,84,4,1,6,2,2,4,4,1,2,5.00,3,?,48.00,7.20,3,5.30,2,1,02208,00000,00000,1\r\n...<\/pre>\n
\n
# summarize the horse colic dataset\r\nfrom pandas import read_csv\r\n# load dataset\r\nurl = 'https:\/\/raw.githubusercontent.com\/jbrownlee\/Datasets\/master\/horse-colic.csv'\r\ndataframe = read_csv(url, header=None, na_values='?')\r\ndata = dataframe.values\r\n# split into input and output elements\r\nix = [i for i in range(data.shape[1]) if i != 23]\r\nX, y = data[:, ix], data[:, 23]\r\nprint(X.shape, y.shape)<\/pre>\n
(300, 27) (300,)<\/pre>\n