Quick-reference guide to the 15 statistical hypothesis tests that you need in
\napplied machine learning, with sample code in Python.<\/h4>\n
Although there are hundreds of statistical hypothesis tests that you could use, there is only a small subset that you may need to use in a machine learning project.<\/p>\n
In this post, you will discover a cheat sheet for the most popular statistical hypothesis tests for a machine learning project with examples using the Python API.<\/p>\n
Each statistical test is presented in a consistent way, including:<\/p>\n
- \n
- The name of the test.<\/li>\n
- What the test is checking.<\/li>\n
- The key assumptions of the test.<\/li>\n
- How the test result is interpreted.<\/li>\n
- Python API for using the test.<\/li>\n<\/ul>\n
Note, when it comes to assumptions such as the expected distribution of data or sample size, the results of a given test are likely to degrade gracefully rather than become immediately unusable if an assumption is violated.<\/p>\n
Generally, data samples need to be representative of the domain and large enough to expose their distribution to analysis.<\/p>\n
In some cases, the data can be corrected to meet the assumptions, such as correcting a nearly normal distribution to be normal by removing outliers, or using a correction to the degrees of freedom in a statistical test when samples have differing variance, to name two examples.<\/p>\n
Finally, there may be multiple tests for a given concern, e.g. normality. We cannot get crisp answers to questions with statistics; instead, we get probabilistic answers. As such, we can arrive at different answers to the same question by considering the question in different ways. Hence the need for multiple different tests for some questions we may have about data.<\/p>\n
Let\u2019s get started.<\/p>\n
![\"Statistical](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2018\/08\/Statistical-Hypothesis-Tests-in-Python-Cheat-Sheet.jpg\")
<\/p>\nStatistical Hypothesis Tests in Python Cheat Sheet
Photo by davemichuda<\/a>, some rights reserved.<\/p>\n<\/div>\nTutorial Overview<\/h2>\n
This tutorial is divided into four parts; they are:<\/p>\n
- \n
- Normality Tests<\/li>\n
- Correlation Tests<\/li>\n
- Parametric Statistical Hypothesis Tests<\/li>\n
- Nonparametric Statistical Hypothesis Tests<\/li>\n<\/ol>\n
1. Normality Tests<\/h2>\n
This section lists statistical tests that you can use to check if your data has a Gaussian distribution.<\/p>\n
Shapiro-Wilk Test<\/h3>\n
Tests whether a data sample has a Gaussian distribution.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the sample has a Gaussian distribution.<\/li>\n
- H1: the sample does not have a Gaussian distribution.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import shapiro\r\ndata1 = ....\r\nstat, p = shapiro(data)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.shapiro<\/a><\/li>\n
- Shapiro-Wilk test on Wikipedia<\/a><\/li>\n<\/ul>\n
D\u2019Agostino\u2019s K^2 Test<\/h3>\n
Tests whether a data sample has a Gaussian distribution.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the sample has a Gaussian distribution.<\/li>\n
- H1: the sample does not have a Gaussian distribution.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import normaltest\r\ndata1 = ....\r\nstat, p = normaltest(data)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.normaltest<\/a><\/li>\n
- D\u2019Agostino\u2019s $K$-squared test on Wikipedia<\/a><\/li>\n<\/ul>\n
Anderson-Darling Test<\/h3>\n
Tests whether a data sample has a Gaussian distribution.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the sample has a Gaussian distribution.<\/li>\n
- H1: the sample does not have a Gaussian distribution.<\/li>\n<\/ul>\nPython Code<\/div>\n<\/p>\n
from scipy.stats import anderson\r\ndata1 = ....\r\nresult = anderson(data)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.anderson<\/a><\/li>\n
- Anderson-Darling test on Wikipedia<\/a><\/li>\n<\/ul>\n
2. Correlation Tests<\/h2>\n
This section lists statistical tests that you can use to check if two samples are related.<\/p>\n
Pearson\u2019s Correlation Coefficient<\/h3>\n
Tests whether two samples have a monotonic relationship.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n
- Observations in each sample are normally distributed.<\/li>\n
- Observations in each sample have the same variance.<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the two samples are independent.<\/li>\n
- H1: there is a dependency between the samples.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import pearsonr\r\ndata1, data2 = ...\r\ncorr, p = pearsonr(data1, data2)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.pearsonr<\/a><\/li>\n
- Pearson\u2019s correlation coefficient on Wikipedia<\/a><\/li>\n<\/ul>\n
Spearman\u2019s Rank Correlation<\/h3>\n
Tests whether two samples have a monotonic relationship.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n
- Observations in each sample can be ranked.<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the two samples are independent.<\/li>\n
- H1: there is a dependency between the samples.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import spearmanr\r\ndata1, data2 = ...\r\ncorr, p = spearmanr(data1, data2)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.spearmanr<\/a><\/li>\n
- Spearman\u2019s rank correlation coefficient on Wikipedia<\/a><\/li>\n<\/ul>\n
Kendall\u2019s Rank Correlation<\/h3>\n
Tests whether two samples have a monotonic relationship.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n
- Observations in each sample can be ranked.<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the two samples are independent.<\/li>\n
- H1: there is a dependency between the samples.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import kendalltau\r\ndata1, data2 = ...\r\ncorr, p = kendalltau(data1, data2)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.kendalltau<\/a><\/li>\n
- Kendall rank correlation coefficient on Wikipedia<\/a><\/li>\n<\/ul>\n
Chi-Squared Test<\/h3>\n
Tests whether two categorical variables are related or independent.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations used in the calculation of the contingency table are independent.<\/li>\n
- 25 or more examples in each cell of the contingency table.<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the two samples are independent.<\/li>\n
- H1: there is a dependency between the samples.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import chi2_contingency\r\ntable = ...\r\nstat, p, dof, expected = chi2_contingency(table)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.chi2_contingency<\/a><\/li>\n
- Chi-Squared test on Wikipedia<\/a><\/li>\n<\/ul>\n
3. Parametric Statistical Hypothesis Tests<\/h2>\n
This section lists statistical tests that you can use to compare data samples.<\/p>\n
Student\u2019s t-test<\/h3>\n
Tests whether the means of two independent samples are significantly different.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n
- Observations in each sample are normally distributed.<\/li>\n
- Observations in each sample have the same variance.<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the means of the samples are equal.<\/li>\n
- H1: the means of the samples are unequal.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import ttest_ind\r\ndata1, data2 = ...\r\nstat, p = ttest_ind(data1, data2)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.ttest_ind<\/a><\/li>\n
- Student\u2019s t-test on Wikipedia<\/a><\/li>\n<\/ul>\n
Paired Student\u2019s t-test<\/h3>\n
Tests whether the means of two paired samples are significantly different.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n
- Observations in each sample are normally distributed.<\/li>\n
- Observations in each sample have the same variance.<\/li>\n
- Observations across each sample are paired.<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the means of the samples are equal.<\/li>\n
- H1: the means of the samples are unequal.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import ttest_rel\r\ndata1, data2 = ...\r\nstat, p = ttest_rel(data1, data2)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.ttest_rel<\/a><\/li>\n
- Student\u2019s t-test on Wikipedia<\/a><\/li>\n<\/ul>\n
Analysis of Variance Test (ANOVA)<\/h3>\n
Tests whether the means of two or more independent samples are significantly different.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n
- Observations in each sample are normally distributed.<\/li>\n
- Observations in each sample have the same variance.<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the means of the samples are equal.<\/li>\n
- H1: one or more of the means of the samples are unequal.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import f_oneway\r\ndata1, data2, ... = ...\r\nstat, p = f_oneway(data1, data2, ...)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.f_oneway<\/a><\/li>\n
- Analysis of variance on Wikipedia<\/a><\/li>\n<\/ul>\n
Repeated Measures ANOVA Test<\/h3>\n
Tests whether the means of two or more paired samples are significantly different.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n
- Observations in each sample are normally distributed.<\/li>\n
- Observations in each sample have the same variance.<\/li>\n
- Observations across each sample are paired.<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the means of the samples are equal.<\/li>\n
- H1: one or more of the means of the samples are unequal.<\/li>\n<\/ul>\n
Python Code<\/p>\n
Currently not supported in Python.<\/p>\n
More Information<\/p>\n
- \n
- Analysis of variance on Wikipedia<\/a><\/li>\n<\/ul>\n
4. Nonparametric Statistical Hypothesis Tests<\/h2>\n
Mann-Whitney U Test<\/h3>\n
Tests whether the distributions of two independent samples are equal or not.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n
- Observations in each sample can be ranked.<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the distributions of both samples are equal.<\/li>\n
- H1: the distributions of both samples are not equal.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import mannwhitneyu\r\ndata1, data2 = ...\r\nstat, p = mannwhitneyu(data1, data2)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.mannwhitneyu<\/a><\/li>\n
- Mann-Whitney U test on Wikipedia<\/a><\/li>\n<\/ul>\n
Wilcoxon Signed-Rank Test<\/h3>\n
Tests whether the distributions of two paired samples are equal or not.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n
- Observations in each sample can be ranked.<\/li>\n
- Observations across each sample are paired.<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the distributions of both samples are equal.<\/li>\n
- H1: the distributions of both samples are not equal.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import wilcoxon\r\ndata1, data2 = ...\r\nstat, p = wilcoxon(data1, data2)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.wilcoxon<\/a><\/li>\n
- Wilcoxon signed-rank test on Wikipedia<\/a><\/li>\n<\/ul>\n
Kruskal-Wallis H Test<\/h3>\n
Tests whether the distributions of two or more independent samples are equal or not.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n
- Observations in each sample can be ranked.<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the distributions of all samples are equal.<\/li>\n
- H1: the distributions of one or more samples are not equal.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import kruskal\r\ndata1, data2, ... = ...\r\nstat, p = kruskal(data1, data2, ...)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.kruskal<\/a><\/li>\n
- Kruskal-Wallis one-way analysis of variance on Wikipedia<\/a><\/li>\n<\/ul>\n
Friedman Test<\/h3>\n
Tests whether the distributions of two or more paired samples are equal or not.<\/p>\n
Assumptions<\/p>\n
- \n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n
- Observations in each sample can be ranked.<\/li>\n
- Observations across each sample are paired.<\/li>\n<\/ul>\n
Interpretation<\/p>\n
- \n
- H0: the distributions of all samples are equal.<\/li>\n
- H1: the distributions of one or more samples are not equal.<\/li>\n<\/ul>\n
Python Code<\/p>\n
from scipy.stats import friedmanchisquare\r\ndata1, data2, ... = ...\r\nstat, p = friedmanchisquare(data1, data2, ...)<\/pre>\n
More Information<\/p>\n
- \n
- scipy.stats.friedmanchisquare<\/a><\/li>\n
- Friedman test on Wikipedia<\/a><\/li>\n<\/ul>\n
Further Reading<\/h2>\n
This section provides more resources on the topic if you are looking to go deeper.<\/p>\n
- \n
- A Gentle Introduction to Normality Tests in Python<\/a><\/li>\n
- How to Use Correlation to Understand the Relationship Between Variables<\/a><\/li>\n
- How to Use Parametric Statistical Significance Tests in Python<\/a><\/li>\n
- A Gentle Introduction to Statistical Hypothesis Tests<\/a><\/li>\n<\/ul>\n
Summary<\/h2>\n
In this tutorial, you discovered the key statistical hypothesis tests that you may need to use in a machine learning project.<\/p>\n
Specifically, you learned:<\/p>\n
- \n
- The types of tests to use in different circumstances, such as normality checking, relationships between variables, and differences between samples.<\/li>\n
- The key assumptions for each test and how to interpret the test result.<\/li>\n
- How to implement the test using the Python API.<\/li>\n<\/ul>\n
Do you have any questions?
\nAsk your questions in the comments below and I will do my best to answer.<\/p>\nDid I miss an important statistical test or key assumption for one of the listed tests?
\nLet me know in the comments below.<\/p>\nThe post 15 Statistical Hypothesis Tests in Python (Cheat Sheet)<\/a> appeared first on Machine Learning Mastery<\/a>.<\/p>\n<\/div>\n
Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
- How to Use Correlation to Understand the Relationship Between Variables<\/a><\/li>\n
- Friedman test on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.friedmanchisquare<\/a><\/li>\n
- Kruskal-Wallis one-way analysis of variance on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.kruskal<\/a><\/li>\n
- Wilcoxon signed-rank test on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.wilcoxon<\/a><\/li>\n
- Mann-Whitney U test on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.mannwhitneyu<\/a><\/li>\n
- Analysis of variance on Wikipedia<\/a><\/li>\n<\/ul>\n
- Analysis of variance on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.f_oneway<\/a><\/li>\n
- Student\u2019s t-test on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.ttest_rel<\/a><\/li>\n
- Student\u2019s t-test on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.ttest_ind<\/a><\/li>\n
- Chi-Squared test on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.chi2_contingency<\/a><\/li>\n
- Kendall rank correlation coefficient on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.kendalltau<\/a><\/li>\n
- Spearman\u2019s rank correlation coefficient on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.spearmanr<\/a><\/li>\n
- Pearson\u2019s correlation coefficient on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.pearsonr<\/a><\/li>\n
- Anderson-Darling test on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.anderson<\/a><\/li>\n
- D\u2019Agostino\u2019s $K$-squared test on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.normaltest<\/a><\/li>\n
- Shapiro-Wilk test on Wikipedia<\/a><\/li>\n<\/ul>\n
- scipy.stats.shapiro<\/a><\/li>\n
- Observations in each sample are independent and identically distributed (iid).<\/li>\n<\/ul>\n