Cancer detection is a popular example of an imbalanced classification problem because there are often significantly more cases of non-cancer than actual cancer.<\/p>\n
A standard imbalanced classification dataset is the mammography dataset that involves detecting breast cancer from radiological scans, specifically the presence of clusters of microcalcifications that appear bright on a mammogram. This dataset was constructed by scanning the images, segmenting them into candidate objects, and using computer vision techniques to describe each candidate object.<\/p>\n
It is a popular dataset for imbalanced classification because of the severe class imbalance, specifically where 98 percent of candidate microcalcifications are not cancer and only 2 percent were labeled as cancer by an experienced radiographer.<\/p>\n
In this tutorial, you will discover how to develop and evaluate models for the imbalanced mammography cancer classification dataset.<\/p>\n
After completing this tutorial, you will know:<\/p>\n
- \n
- How to load and explore the dataset and generate ideas for data preparation and model selection.<\/li>\n
- How to evaluate a suite of machine learning models and improve their performance with data cost-sensitive techniques.<\/li>\n
- How to fit a final model and use it to predict class labels for specific cases.<\/li>\n<\/ul>\n
Discover SMOTE, one-class classification, cost-sensitive learning, threshold moving, and much more in my new book<\/a>, with 30 step-by-step tutorials and full Python source code.<\/p>\n
Let’s get started.<\/p>\n
![\"Develop](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/03\/Develop-an-Imbalanced-Classification-Model-to-Detect-Microcalcifications.jpg\")
<\/p>\nDevelop an Imbalanced Classification Model to Detect Microcalcifications
Photo by Bernard Spragg. NZ<\/a>, some rights reserved.<\/p>\n<\/div>\nTutorial Overview<\/h2>\n
This tutorial is divided into five parts; they are:<\/p>\n
- \n
- Mammography Dataset<\/li>\n
- Explore the Dataset<\/li>\n
- Model Test and Baseline Result<\/li>\n
- Evaluate Models\n
- \n
- Evaluate Machine Learning Algorithms<\/li>\n
- Evaluate Cost-Sensitive Algorithms<\/li>\n<\/ol>\n<\/li>\n
- Make Predictions on New Data<\/li>\n<\/ol>\n
Mammography Dataset<\/h2>\n
In this project, we will use a standard imbalanced machine learning dataset referred to as the “mammography<\/em>” dataset or sometimes “Woods Mammography<\/em>.”<\/p>\n
The dataset is credited to Kevin Woods, et al. and the 1993 paper titled “Comparative Evaluation Of Pattern Recognition Techniques For Detection Of Microcalcifications In Mammography<\/a>.”<\/p>\n
The focus of the problem is on detecting breast cancer from radiological scans, specifically the presence of clusters of microcalcifications that appear bright on a mammogram.<\/p>\n
The dataset involved first started with 24 mammograms with a known cancer diagnosis that were scanned. The images were then pre-processed using image segmentation computer vision algorithms to extract candidate objects from the mammogram images. Once segmented, the objects were then manually labeled by an experienced radiologist.<\/p>\n
A total of 29 features were extracted from the segmented objects thought to be most relevant to pattern recognition, which was reduced to 18, then finally to seven, as follows (taken directly from the paper):<\/p>\n
- \n
- Area of object (in pixels).<\/li>\n
- Average gray level of the object.<\/li>\n
- Gradient strength of the object’s perimeter pixels.<\/li>\n
- Root mean square noise fluctuation in the object.<\/li>\n
- Contrast, average gray level of the object minus the average of a two-pixel wide border surrounding the object.<\/li>\n
- A low order moment based on shape descriptor.<\/li>\n<\/ul>\n
There are two classes and the goal is to distinguish between microcalcifications and non-microcalcifications using the features for a given segmented object.<\/p>\n
- \n
- Non-microcalcifications<\/strong>: negative case, or majority class.<\/li>\n
- Microcalcifications<\/strong>: positive case, or minority class.<\/li>\n<\/ul>\n
A number of models were evaluated and compared in the original paper, such as neural networks, decision trees, and k-nearest neighbors. Models were evaluated using ROC Curves<\/a> and compared using the area under ROC Curve, or ROC AUC for short.<\/p>\n
ROC Curves and area under ROC Curves were chosen with the intent to minimize the false-positive rate (complement of the specificity) and maximize the true-positive rate (sensitivity), the two axes of the ROC Curve. The use of the ROC Curves also suggests the desire for a probabilistic model from which an operator can select a probability threshold as the cut-off between the acceptable false positive and true positive rates.<\/p>\n
Their results suggested a “linear classifier<\/em>” (seemingly a Gaussian Naive Bayes classifier<\/a>) performed the best with a ROC AUC of 0.936 averaged over 100 runs.<\/p>\n
Next, let’s take a closer look at the data.<\/p>\n<\/p>\n
<\/div>\n<\/p>\n Want to Get Started With Imbalance Classification?<\/h3>\n
Take my free 7-day email crash course now (with sample code).<\/p>\n
Click to sign-up and also get a free PDF Ebook version of the course.<\/p>\n
- Microcalcifications<\/strong>: positive case, or minority class.<\/li>\n<\/ul>\n
- Non-microcalcifications<\/strong>: negative case, or majority class.<\/li>\n