Many binary classification tasks do not have an equal number of examples from each class, e.g. the class distribution is skewed or imbalanced.<\/p>\n
Nevertheless, accuracy is equally important in both classes.<\/p>\n
An example is the classification of vowel sounds from European languages as either nasal or oral on speech recognition where there are many more examples of nasal than oral vowels. Classification accuracy is important for both classes, although accuracy as a metric cannot be used directly. Additionally, data sampling techniques may be required to transform the training dataset to make it more balanced when fitting machine learning algorithms.<\/p>\n
In this tutorial, you will discover how to develop and evaluate models for imbalanced binary classification of nasal and oral phonemes.<\/p>\n
After completing this tutorial, you will know:<\/p>\n
- \n
- How to load and explore the dataset and generate ideas for data preparation and model selection.<\/li>\n
- How to evaluate a suite of machine learning models and improve their performance with data oversampling techniques.<\/li>\n
- How to fit a final model and use it to predict class labels for specific cases.<\/li>\n<\/ul>\n
Discover SMOTE, one-class classification, cost-sensitive learning, threshold moving, and much more in my new book<\/a>, with 30 step-by-step tutorials and full Python source code.<\/p>\n
Let’s get started.<\/p>\n
![\"Predictive](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2020\/03\/Predictive-Model-for-the-Phoneme-Imbalanced-Classification-Dataset.jpg\")
<\/p>\nPredictive Model for the Phoneme Imbalanced Classification Dataset
Photo by Ed Dunens<\/a>, some rights reserved.<\/p>\n<\/div>\nTutorial Overview<\/h2>\n
This tutorial is divided into five parts; they are:<\/p>\n
- \n
- Phoneme Dataset<\/li>\n
- Explore the Dataset<\/li>\n
- Model Test and Baseline Result<\/li>\n
- Evaluate Models\n
- \n
- Evaluate Machine Learning Algorithms<\/li>\n
- Evaluate Data Oversampling Algorithms<\/li>\n<\/ol>\n<\/li>\n
- Make Predictions on New Data<\/li>\n<\/ol>\n
Phoneme Dataset<\/h2>\n
In this project, we will use a standard imbalanced machine learning dataset referred to as the “Phoneme<\/em>” dataset.<\/p>\n
This dataset is credited to the ESPRIT (European Strategic Program on Research in Information Technology<\/a>) project titled “ROARS<\/em>” (Robust Analytical Speech Recognition System) and described in progress reports and technical reports from that project.<\/p>\n
\n
The goal of the ROARS project is to increase the robustness of an existing analytical speech recognition system (i,e., one using knowledge about syllables, phonemes and phonetic features), and to use it as part of a speech understanding system with connected words and dialogue capability. This system will be evaluated for a specific application in two European languages<\/p>\n<\/blockquote>\n
— ESPRIT: The European Strategic Programme for Research and development in Information Technology<\/a>.<\/p>\n
The goal of the dataset was to distinguish between nasal and oral vowels.<\/p>\n
Vowel sounds were spoken and recorded to digital files. Then audio features were automatically extracted from each sound.<\/p>\n
\n
Five different attributes were chosen to characterize each vowel: they are the amplitudes of the five first harmonics AHi, normalised by the total energy Ene (integrated on all the frequencies): AHi\/Ene. Each harmonic is signed: positive when it corresponds to a local maximum of the spectrum and negative otherwise.<\/p>\n<\/blockquote>\n