![\"Dimensionality](\"https:\/\/editor.analyticsvidhya.com\/uploads\/78698SrDS.png\")
<\/p>\n
Hello! I like to share my interesting experience While I was working as a junior Data Scientist, I can even say I was a beginner during that time in this data science domain.<\/p>\n
One of the customers came to us for machine learning implementation for their problem statement in either way unsupervised and supervised forms, I thought it was going to be as usual mode of execution and process because based on my experience for small scale implementation or during my training period we use to have 25-30 features and we play around with that and we use to predict or classify or clustering the dataset and share the outcome.<\/p>\n
But this time they come up with thousands of features, But I was a little surprised and scared about the implementation and my head started spinning as anything. Same time my Senior Data Scientist brought everyone from the team into the meeting room.<\/p>\n
My\u00a0Senior Data Scientist (Sr. DS) coined the new word to us, that is nothing but\u00a0<\/span>Dimensionality Reduction<\/b><\/i>\u00a0<\/span>(OR)\u00a0<\/span>Dimension Reduction<\/b><\/i>\u00a0<\/span>(OR)\u00a0<\/span>Curse Of Dimensionality,\u00a0<\/span><\/b>all beginners thought that he is going to explain something in Physis, we had little remembrance that we had come across this term during our training programme. then he started to sketch on the board (Refer fig-1). When we started looking at 1-D, 2-D we are much comfortable but 3-D and above our heads started to spin.\u00a0<\/i><\/p>\n Sr. DS has continued his lecture, all these sample pictures are just notable\u00a0<\/span>features and we could play around<\/span>\u00a0<\/span>with these, in a real-time scenario, many Machine Learning(ML) problems involve thousands of features, so we end up training those models became extremely slow and will not give good solutions for business problem and we couldn\u2019t freeze the model, this situation is the so-called \u201cCurse Of Dimensionality\u201d\u00a0<\/b><\/i>working. Then we all started asking a question that how we should handle this.<\/p>\n He took a long breath and continue to share his experience in his own style.\u00a0 He started with a simple definition as follows.<\/p>\n \u00a0<\/p>\n We can say the number of features in our dataset is referred to as its dimensionality.<\/p>\n \u00a0<\/p>\n Dimensionality Reduction is the process of reducing the dimensions(features) of a given dataset. Let\u2019s say if your dataset with a hundred columns\/features and bringing the number of columns down to 20-25.\u00a0 In simple terms, you are converting the\u00a0<\/span>Cylinder<\/b>\/Sphere to a Circle<\/b><\/i>\u00a0<\/span>or\u00a0<\/span>Cube into a Plane<\/i>\u00a0<\/span><\/b>in the two-dimensional space as below figure.<\/p>\n He has drawn below the relationship clearly between\u00a0<\/span>Modle Performance<\/i>\u00a0<\/span>and\u00a0<\/span>Number of Features(Dimensions)<\/i>.\u00a0As the number of features increases, the number of data points also increases proportionally. the straight statement is that the more features will bring more data samples, So we have represented all combinations of features and their values.<\/p>\n Now everyone in the room got the feel of what is \u201cCurse Of Dimensionality\u201d\u00a0<\/b><\/span>at a very high level.<\/p>\n \u00a0<\/p>\n Suddenly, one of the team members asked can he tell us the benefits of doing\u00a0dimensionality reduction in the given dataset.<\/p>\n Our Sr. DS didn\u2019t stop sharing his extensive knowledge further. He has continued as below.<\/p>\n There are lots of benefits if we go with\u00a0dimensionality reduction.<\/p>\n Yes! it was an awe-inspiring spectacle, robustness, and dynamics of the \u201cDimensionality Reduction\u201d<\/b>. Now I can\u00a0<\/span>visualization the overall benefit as below. hope it could help you too\u00a0\u00a0<\/span><\/p>\n \u00a0<\/span><\/p>\n What is next, Of Course! We jump into the next major question that what are techniques available for Dimensionality Reduction.<\/p>\n \u00a0<\/p>\n Our Sr. DS very much interested continued his explanation on the\u00a0techniques whichever possible in Data Science domain, broadly classified into two approaches as mentioned earlier considering selecting the best-fit Feature(s) or removing less important Feature in the given high dimensional dataset. these high-level techniques use to be called\u00a0<\/span>Feature Selection\u00a0<\/span><\/b>or\u00a0<\/span>Feature Extraction,<\/b>\u00a0<\/span>and basically, this is part of\u00a0<\/span>Feature Engineering.<\/b>\u00a0<\/span>He has connected the dots perfectly.<\/p>\n He took us further in-depth concepts to understand the big picture of applied \u201cDimensionality Reduction\u201d<\/b><\/i>\u00a0on the high dimensional dataset. Once we saw the below figure we able to relate the Feature Engineering and Dimensionality Reduction. Look at this figure the essence of\u00a0Dimensionality Reduction well by our Sr. DS is in it!<\/p>\n Everyone was interested to know how to apply all these using Phyton libraries with the help of simple coding. our Sr. DS asked me to bring colorful markers and dusters<\/p>\n Sr. DS picked up the new blue marker and started explaining PCA with a simple example as follows, before that he explained what is PCA stuff for dimensionality reduction.<\/p>\n Principal Component Analysis(PCA):\u00a0<\/b>PCA is a technique for dimensionality reduction of a given dataset, by increasing interpretability with negligible information loss. Here the number of variables is decreasing, so it makes further analysis simpler.\u00a0<\/span>Which converts a set of correlated variables to a set of uncorrelated variables.\u00a0<\/span>Used for machine learning predictive modeling. And he advised us to go through\u00a0<\/span>Eigenvector,\u00a0<\/strong>Eigen Values<\/strong><\/p>\n He took familiar\u00a0<\/span>wines.csv for his quick analysis.<\/span><\/p>\n \u00a0<\/p>\n No Null value in the given data set, So great and we\u2019re lucky.<\/p>\n \u00a0<\/p>\n [\u2018fixed acidity\u2019, \u2018volatile acidity\u2019, \u2018citric acid\u2019, \u2018chlorides\u2019, \u2018total sulfur dioxide\u2019, \u2018density\u2019, \u2018sulphates\u2019, \u2018alcohol\u2019]<\/span><\/p>\n Training and Testing Shape<\/b><\/p>\n PCA implementation for Dimensionality reduction (with 2 columns)<\/b><\/span><\/p>\n Naming them as\u00a0principal component 1, principal component 2<\/b><\/i><\/p>\n New\u00a0principal components and their values.<\/span><\/p>\n We all surprised when looking at the above two columns with new column name and values, We asked what happen to\u00a0<\/i><\/span>\u2018fixed acidity\u2019, \u2018volatile acidity, \u2018citric acid\u2019, \u2018chlorides\u2019, \u2018total sulfur dioxide\u2019, \u2018density\u2019, \u2018sulphates\u2019, \u2018alcohol\u2019<\/i>\u00a0columns. Sr. DS said all gone, now we have just two columns after we applied PCA for dimensionality reduction on given data and we are going to implement few models and this is going to be the normal way.<\/span><\/p>\n He has mentioned one keyword\u00a0<\/span>\u201cvariation per principal component\u201d<\/b><\/p>\n this is the fraction of\u00a0<\/span>variance explained<\/b>\u00a0<\/span>by a\u00a0<\/span>principal component<\/b>\u00a0<\/span>is the ratio between the\u00a0<\/span>variance<\/b>\u00a0<\/span>of that\u00a0<\/span>principal component<\/b>\u00a0<\/span>and the total\u00a0<\/span>variance<\/b>.<\/p>\n Followed by this he was demonstrated the following models<\/p>\n Accuracy was better and little difference among each model, but he has mentioned this is for PCA implementation. Everyone in the room felt that we have completed an excellent\u00a0roller coaster. he has advised us to do hands-on other Dimensionality Reduction \u2013 Techniques.<\/p>\n Okay, Guys! Thanks for your time, hope I able to narrate my learning experience of\u00a0Dimensionality Reduction \u2013 Techniques in right ways here, I trust it would help to continue the journey to handle complex data set in machine learning problem statement. Cheers!<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 1-D and 2-D<\/div>\n
![\"Dimensionality](\"https:\/\/editor.analyticsvidhya.com\/uploads\/719093-D.png\")
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 3 \u2013 D<\/div>\nWhat is Dimensionality?\u00a0<\/b><\/h2>\n
\nWhat is\u00a0<\/b>Dimensionality Reduction?<\/b>\u00a0<\/b>
\n<\/h2>\n![\"Dimensionality](\"https:\/\/editor.analyticsvidhya.com\/uploads\/96012DR-Shape.png\")
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0Converting 3D- 2D<\/div>\n![\"Dimensionality](\"https:\/\/editor.analyticsvidhya.com\/uploads\/32524FVsP.png\")
Modle Performance<\/i>\u00a0<\/span>Vs\u00a0Number of Features<\/i>\n<\/div>\nBenefits of doing Dimensionality Reduction<\/h2>\n
\n
![\"Benefits](\"https:\/\/editor.analyticsvidhya.com\/uploads\/41402DR-benifts.png\")
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 Benefits of\u00a0Dimensionality Reduction.\n<\/div>\nDimensionality Reduction \u2013 Techniques<\/h2>\n
![\"Locating](\"https:\/\/editor.analyticsvidhya.com\/uploads\/64476DR-FE.png\")
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 Locating Dimensionality Reduction in\u00a0Feature Engineering family<\/b>\n<\/div>\n![\"Dimensionality](\"https:\/\/editor.analyticsvidhya.com\/uploads\/26473DR-FS_FE.png\")
<\/div>\n![\"marker\"](\"https:\/\/editor.analyticsvidhya.com\/uploads\/69343MD.png\")
<\/div>\n![\"PCA\"](\"https:\/\/editor.analyticsvidhya.com\/uploads\/55529Wins.png\")
\n<\/div>\n# Import all the necessary packages
import pandas as pd
\nimport numpy as np
\nimport matplotlib.pyplot as plt
\nimport seaborn as sns from sklearn.model_selection
\nimport train_test_split from sklearn.linear_model
\nimport LinearRegression from sklearn.metrics
\nimport confusion_matrix from sklearn.metrics
\nimport accuracy_score from sklearn
\nimport metrics %matplotlib inline
\nimport matplotlib.pyplot as plt
\n%matplotlib inline
\nwq_dataset = pd.read_csv('winequality.csv')<\/pre>\nEDA on a given data set<\/b><\/h5>\n
wq_dataset.head(5)<\/pre>\n
![\"dataset.head\"](\"https:\/\/editor.analyticsvidhya.com\/uploads\/45603wins-head.png\")
<\/div>\nwq_dataset.describe()<\/pre>\n
![\"describe\"](\"https:\/\/editor.analyticsvidhya.com\/uploads\/29591Describe.png\")
<\/div>\nwq_dataset.isnull().any()<\/pre>\n
<\/div>\n
Find correlations of each feature<\/h4>\n
correlations = wq_dataset.corr()['quality'].drop('quality') print(correlations)<\/pre>\n![\"correlations\"](\"https:\/\/editor.analyticsvidhya.com\/uploads\/20629correlations.png\")
<\/div>\nCorrelation Representation using Heatmap<\/h4>\n
sns.heatmap(wq_dataset.corr()) plt.show()<\/pre>\n
![\"correlation](\"https:\/\/editor.analyticsvidhya.com\/uploads\/77232cor-HM.png\")
\n<\/div>\nx = wq_dataset[features] y = wq_dataset['quality']<\/pre>\n
x_train, x_test, y_train, y_test = train_test_split(x, y, random_state=3)<\/pre>\n
print('Traning data shape:', x_train.shape) print('Testing data shape:', x_test.shape)<\/pre>\nTraning data shape: (1199, 8) Testing data shape: (400, 8)<\/pre>\n
from sklearn.decomposition import PCA pca_wins = PCA(n_components=2) principalComponents_wins = pca_wins.fit_transform(x)<\/pre>\n
pcs_wins_df = pd.DataFrame(data = principalComponents_wins, columns = ['principal component 1', 'principal component 2'])<\/pre>\n
pcs_wins_df.head()<\/pre>\n
![\"principal](\"https:\/\/editor.analyticsvidhya.com\/uploads\/75634pc12.png\")
<\/div>\nprint('Explained variation per principal component: {}'.format(pca_wins.explained_variance_ratio_))<\/pre>\nExplained variation per principal component: [0.99615166 0.00278501]<\/b><\/pre>\n
\n
<\/div>\n