Radius Neighbors Classifier is a classification machine learning algorithm.<\/p>\n
It is an extension to the k-nearest neighbors algorithm that makes predictions using all examples in the radius of a new example rather than the k-closest neighbors.<\/p>\n
As such, the radius-based approach to selecting neighbors is more appropriate for sparse data<\/a>, preventing examples that are far away in the feature space from contributing to a prediction.<\/p>\n In this tutorial, you will discover the Radius Neighbors Classifier<\/strong> classification machine learning algorithm.<\/p>\n After completing this tutorial, you will know:<\/p>\n Let’s get started.<\/p>\n Radius Neighbors Classifier Algorithm With Python This tutorial is divided into three parts; they are:<\/p>\n Radius Neighbors is a classification machine learning algorithm.<\/p>\n It is based on the k-nearest neighbors algorithm, or kNN. kNN involves taking the entire training dataset and storing it. Then, at prediction time, the k-closest examples in the training dataset are located for each new example for which we want to predict. The mode (most common value) class label from the k neighbors is then assigned to the new example.<\/p>\n For more on the k-nearest neighbours algorithm, see the tutorial:<\/p>\n The Radius Neighbors Classifier is similar in that training involves storing the entire training dataset. The way that the training dataset is used during prediction is different.<\/p>\n Instead of locating the k-neighbors, the Radius Neighbors Classifier locates all examples in the training dataset that are within a given radius of the new example. The radius neighbors are then used to make a prediction for the new example.<\/p>\n The radius is defined in the feature space and generally assumes that the input variables are numeric and scaled to the range 0-1, e.g. normalized.<\/p>\n The radius-based approach to locating neighbors is appropriate for those datasets where it is desirable for the contribution of neighbors to be proportional to the density of examples in the feature space.<\/p>\n Given a fixed radius, dense regions of the feature space will contribute more information and sparse regions will contribute less information. It is this latter case that is most desirable and it prevents examples very far in feature space from the new example from contributing to the prediction.<\/p>\n As such, the Radius Neighbors Classifier may be more appropriate for prediction problems where there are sparse regions of the feature space.<\/p>\n Given that the radius is fixed in all dimensions of the feature space, it will become less effective as the number of input features is increased, which causes examples in the feature space to spread further and further apart. This property is referred to as the curse of dimensionality<\/a>.<\/p>\n The Radius Neighbors Classifier is available in the scikit-learn Python machine learning library via the RadiusNeighborsClassifier class<\/a>.<\/p>\n The class allows you to specify the size of the radius used when making a prediction via the “radius<\/em>” argument, which defaults to 1.0.<\/p>\n Another important hyperparameter is the “weights<\/em>” argument that controls whether neighbors contribute to the prediction in a ‘uniform<\/em>‘ manner or inverse to the distance (‘distance<\/em>‘) from the example. Uniform weight is used by default.<\/p>\n We can demonstrate the Radius Neighbors Classifier with a worked example.<\/p>\n First, let’s define a synthetic classification dataset.<\/p>\n We will use the make_classification() function<\/a> to create a dataset with 1,000 examples, each with 20 input variables.<\/p>\n The example below creates and summarizes the dataset.<\/p>\n Running the example creates the dataset and confirms the number of rows and columns of the dataset.<\/p>\n We can fit and evaluate a Radius Neighbors Classifier model using repeated stratified k-fold cross-validation via the RepeatedStratifiedKFold class<\/a>. We will use 10 folds and three repeats in the test harness.<\/p>\n We will use the default configuration.<\/p>\n It is important that the feature space is scaled prior to preparing and using the model.<\/p>\n We can achieve this by using the MinMaxScaler<\/a> to normalize the input features and use a Pipeline<\/a> to first apply the scaling, then use the model.<\/p>\n The complete example of evaluating the Radius Neighbors Classifier model for the synthetic binary classification task is listed below.<\/p>\n Running the example evaluates the Radius Neighbors Classifier algorithm on the synthetic dataset and reports the average accuracy across the three repeats of 10-fold cross-validation.<\/p>\n Your specific results may vary given the stochastic nature of the learning algorithm. Consider running the example a few times.<\/p>\n In this case, we can see that the model achieved a mean accuracy of about 75.4 percent.<\/p>\n We may decide to use the Radius Neighbors Classifier as our final model and make predictions on new data.<\/p>\n This can be achieved by fitting the model pipeline on all available data and calling the predict()<\/em> function passing in a new row of data.<\/p>\n We can demonstrate this with a complete example listed below.<\/p>\n Running the example fits the model and makes a class label prediction for a new row of data.<\/p>\n Next, we can look at configuring the model hyperparameters.<\/p>\n The hyperparameters for the Radius Neighbors Classifier method must be configured for your specific dataset.<\/p>\n Perhaps the most important hyperparameter is the radius controlled via the “radius<\/em>” argument. It is a good idea to test a range of values, perhaps around the value of 1.0.<\/p>\n We will explore values between 0.8 and 1.5 with a grid of 0.01 on our synthetic dataset.<\/p>\n Note that we are grid searching the “radius<\/em>” hyperparameter of the RadiusNeighborsClassifier<\/em> within the Pipeline<\/em> where the model is named “model<\/em>” and, therefore, the radius parameter is accessed via model->radius<\/em> with a double underscore (__<\/em>) separator, e.g. “model__radius<\/em>“.<\/p>\n The example below demonstrates this using the GridSearchCV class<\/a> with a grid of values we have defined.<\/p>\n Running the example will evaluate each combination of configurations using repeated cross-validation.<\/p>\n Your specific results may vary given the stochastic nature of the learning algorithm. Try running the example a few times.<\/p>\n In this case, we can see that we achieved better results using a radius of 0.8 that gave an accuracy of about 87.2 percent compared to a radius of 1.0 in the previous example that gave an accuracy of about 75.4 percent.<\/p>\n Another key hyperparameter is the manner in which examples in the radius contribute to the prediction via the “weights<\/em>” argument. This can be set to “uniform<\/em>” (the default), “distance<\/em>” for inverse distance, or a custom function.<\/p>\n We can test both of these built-in weightings and see which performs better with our radius of 0.8.<\/p>\n The complete example is listed below.<\/p>\n Running the example fits the model and discovers the hyperparameters that give the best results using cross-validation.<\/p>\n Your specific results may vary given the stochastic nature of the learning algorithm. Try running the example a few times.<\/p>\n In this case, we can see an additional lift in mean classification accuracy from about 87.2 percent with ‘uniform<\/em>‘ weights in the previous example to about 89.3 percent with ‘distance<\/em>‘ weights in this case.<\/p>\n Another metric that you might wish to explore is the distance metric used via the ‘metric<\/em>‘ argument that defaults to ‘minkowski<\/em>‘.<\/p>\n It might be interesting to compare results to ‘euclidean<\/em>‘ distance and perhaps ‘cityblock<\/em>‘.<\/p>\n This section provides more resources on the topic if you are looking to go deeper.<\/p>\n In this tutorial, you discovered the Radius Neighbors Classifier classification machine learning algorithm.<\/p>\n Specifically, you learned:<\/p>\n Do you have any questions?<\/strong> The post Radius Neighbors Classifier Algorithm With Python<\/a> appeared first on Machine Learning Mastery<\/a>.<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" Author: Jason Brownlee Radius Neighbors Classifier is a classification machine learning algorithm. It is an extension to the k-nearest neighbors algorithm that makes predictions using […] Read More<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":3919,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[24],"tags":[],"_links":{"self":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/3918"}],"collection":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/comments?post=3918"}],"version-history":[{"count":0,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/3918\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media\/3919"}],"wp:attachment":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media?parent=3918"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/categories?post=3918"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/tags?post=3918"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
![\"Radius](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/08\/Radius-Neighbors-Classifier-Algorithm-With-Python.jpg\")
<\/p>\n
Photo by J. Triepke<\/a>, some rights reserved.<\/p>\n<\/div>\nTutorial Overview<\/h2>\n
\n
Radius Neighbors Classifier<\/h2>\n
\n
Radius Neighbors Classifier With Scikit-Learn<\/h2>\n
...\r\n# create the model\r\nmodel = RadiusNeighborsClassifier(radius=1.0)<\/pre>\n
...\r\n# create the model\r\nmodel = RadiusNeighborsClassifier(weights='uniform')<\/pre>\n
# test classification dataset\r\nfrom sklearn.datasets import make_classification\r\n# define dataset\r\nX, y = make_classification(n_samples=1000, n_features=20, n_informative=15, n_redundant=5, random_state=1)\r\n# summarize the dataset\r\nprint(X.shape, y.shape)<\/pre>\n
(1000, 20) (1000,)<\/pre>\n
...\r\n# create the model\r\nmodel = RadiusNeighborsClassifier()<\/pre>\n
...\r\n# define model\r\nmodel = RadiusNeighborsClassifier()\r\n# create pipeline\r\npipeline = Pipeline(steps=[('norm', MinMaxScaler()),('model',model)])<\/pre>\n# evaluate an radius neighbors classifier model on the dataset\r\nfrom numpy import mean\r\nfrom numpy import std\r\nfrom sklearn.datasets import make_classification\r\nfrom sklearn.model_selection import cross_val_score\r\nfrom sklearn.model_selection import RepeatedStratifiedKFold\r\nfrom sklearn.pipeline import Pipeline\r\nfrom sklearn.preprocessing import MinMaxScaler\r\nfrom sklearn.neighbors import RadiusNeighborsClassifier\r\n# define dataset\r\nX, y = make_classification(n_samples=1000, n_features=20, n_informative=15, n_redundant=5, random_state=1)\r\n# define model\r\nmodel = RadiusNeighborsClassifier()\r\n# create pipeline\r\npipeline = Pipeline(steps=[('norm', MinMaxScaler()),('model',model)])\r\n# define model evaluation method\r\ncv = RepeatedStratifiedKFold(n_splits=10, n_repeats=3, random_state=1)\r\n# evaluate model\r\nscores = cross_val_score(pipeline, X, y, scoring='accuracy', cv=cv, n_jobs=-1)\r\n# summarize result\r\nprint('Mean Accuracy: %.3f (%.3f)' % (mean(scores), std(scores)))<\/pre>\nMean Accuracy: 0.754 (0.042)<\/pre>\n
# make a prediction with a radius neighbors classifier model on the dataset\r\nfrom sklearn.datasets import make_classification\r\nfrom sklearn.pipeline import Pipeline\r\nfrom sklearn.preprocessing import MinMaxScaler\r\nfrom sklearn.neighbors import RadiusNeighborsClassifier\r\n# define dataset\r\nX, y = make_classification(n_samples=1000, n_features=20, n_informative=15, n_redundant=5, random_state=1)\r\n# define model\r\nmodel = RadiusNeighborsClassifier()\r\n# create pipeline\r\npipeline = Pipeline(steps=[('norm', MinMaxScaler()),('model',model)])\r\n# fit model\r\npipeline.fit(X, y)\r\n# define new data\r\nrow = [2.47475454,0.40165523,1.68081787,2.88940715,0.91704519,-3.07950644,4.39961206,0.72464273,-4.86563631,-6.06338084,-1.22209949,-0.4699618,1.01222748,-0.6899355,-0.53000581,6.86966784,-3.27211075,-6.59044146,-2.21290585,-3.139579]\r\n# make a prediction\r\nyhat = pipeline.predict([row])\r\n# summarize prediction\r\nprint('Predicted Class: %d' % yhat)<\/pre>\nPredicted Class: 0<\/pre>\n
Tune Radius Neighbors Classifier Hyperparameters<\/h2>\n
...\r\n# define grid\r\ngrid = dict()\r\ngrid['model__radius'] = arange(0.8, 1.5, 0.01)<\/pre>\n
# grid search radius for radius neighbors classifier\r\nfrom numpy import arange\r\nfrom sklearn.datasets import make_classification\r\nfrom sklearn.model_selection import GridSearchCV\r\nfrom sklearn.model_selection import RepeatedStratifiedKFold\r\nfrom sklearn.pipeline import Pipeline\r\nfrom sklearn.preprocessing import MinMaxScaler\r\nfrom sklearn.neighbors import RadiusNeighborsClassifier\r\n# define dataset\r\nX, y = make_classification(n_samples=1000, n_features=20, n_informative=15, n_redundant=5, random_state=1)\r\n# define model\r\nmodel = RadiusNeighborsClassifier()\r\n# create pipeline\r\npipeline = Pipeline(steps=[('norm', MinMaxScaler()),('model',model)])\r\n# define model evaluation method\r\ncv = RepeatedStratifiedKFold(n_splits=10, n_repeats=3, random_state=1)\r\n# define grid\r\ngrid = dict()\r\ngrid['model__radius'] = arange(0.8, 1.5, 0.01)\r\n# define search\r\nsearch = GridSearchCV(pipeline, grid, scoring='accuracy', cv=cv, n_jobs=-1)\r\n# perform the search\r\nresults = search.fit(X, y)\r\n# summarize\r\nprint('Mean Accuracy: %.3f' % results.best_score_)\r\nprint('Config: %s' % results.best_params_)<\/pre>\nMean Accuracy: 0.872\r\nConfig: {'model__radius': 0.8}<\/pre>\n...\r\n# define grid\r\ngrid = dict()\r\ngrid['model__weights'] = ['uniform', 'distance']<\/pre>\n
# grid search weights for radius neighbors classifier\r\nfrom sklearn.datasets import make_classification\r\nfrom sklearn.model_selection import GridSearchCV\r\nfrom sklearn.model_selection import RepeatedStratifiedKFold\r\nfrom sklearn.pipeline import Pipeline\r\nfrom sklearn.preprocessing import MinMaxScaler\r\nfrom sklearn.neighbors import RadiusNeighborsClassifier\r\n# define dataset\r\nX, y = make_classification(n_samples=1000, n_features=20, n_informative=15, n_redundant=5, random_state=1)\r\n# define model\r\nmodel = RadiusNeighborsClassifier(radius=0.8)\r\n# create pipeline\r\npipeline = Pipeline(steps=[('norm', MinMaxScaler()),('model',model)])\r\n# define model evaluation method\r\ncv = RepeatedStratifiedKFold(n_splits=10, n_repeats=3, random_state=1)\r\n# define grid\r\ngrid = dict()\r\ngrid['model__weights'] = ['uniform', 'distance']\r\n# define search\r\nsearch = GridSearchCV(pipeline, grid, scoring='accuracy', cv=cv, n_jobs=-1)\r\n# perform the search\r\nresults = search.fit(X, y)\r\n# summarize\r\nprint('Mean Accuracy: %.3f' % results.best_score_)\r\nprint('Config: %s' % results.best_params_)<\/pre>\nMean Accuracy: 0.893\r\nConfig: {'model__weights': 'distance'}<\/pre>\nFurther Reading<\/h2>\n
Tutorials<\/h3>\n
\n
Books<\/h3>\n
\n
APIs<\/h3>\n
\n
Articles<\/h3>\n
\n
Summary<\/h2>\n
\n
\nAsk your questions in the comments below and I will do my best to answer.<\/p>\n