With this article be ready to get your hands dirty with ML algorithms, concepts, Maths and coding.
To work with ML code, libraries play a very important role in Python which we will study in details but let see a very brief description of the most important ones :
- NumPy (Numerical Python) : It is one of the greatest Scientific and Mathematical computing library for Python. Platforms like Keras, Tensorflow have embedded Numpy operations on Tensors. The feature we are concerned with its power and easy to handle and perform operation on Array.
- Pandas : This package is very useful when it comes to handle data. This makes it very easier to manipulate, aggregate and visualize data.
- MatplotLib : This library facilitates the task of powerful and very simple visualizations.
There are many more libraries but they have no use right now. So, let’s begin.
Download the dataset :
Go to the link and download Data_for_Missing_Values.csv.
Anaconda :
I would suggest you guys to install Anaconda on your systems. Launch Spyder our Jupyter on your system. Reason behind suggesting is – Anaconda has all the basic Python Libraries pre installed in it.
Below is the Python code :
# Python code explaining How to # Handle Missing Value in Dataset """ PART 1 Importing Libraries """ import numpy as np import matplotlib.pyplot as plt import pandas as pd """ PART 2 Importing Data """ data_sets = pd.read_csv( 'C:\Users\Admin\Desktop\Data_for_Missing_Values.csv' ) print ( "Data Head :
" , data_sets.head()) print ( "
Data Describe :
" , data_sets.describe()) """ PART 3 Input and Output Data """ # All rows but all columns except last X = data_sets.iloc[:, : - 1 ].values # TES # All rows but only last column Y = data_sets.iloc[:, 3 ].values print ( "
Input :
" , X) print ( "
Output:
" , Y) """ PART 4 Handling the missing values """ # We will use sklearn library >> preprocessing package # Imputer class of that package from sklearn.preprocessing import Imputer # Using Imputer function to replace NaN # values with mean of that parameter value imputer = Imputer(missing_values = "NaN" , strategy = "mean" , axis = 0 ) # Fitting the data, function learns the stats imputer = imputer.fit(X[:, 1 : 3 ]) # fit_transform() will execute those # stats on the input ie. X[:, 1:3] X[:, 1 : 3 ] = imputer.fit_transform(X[:, 1 : 3 ]) # filling the missing value with mean print ( "
New Input with Mean Value for NaN :
" , X) |
Ouput :
Data Head : Country Age Salary Purchased 0 France 44.0 72000.0 No 1 Spain 27.0 48000.0 Yes 2 Germany 30.0 54000.0 No 3 Spain 38.0 61000.0 No 4 Germany 40.0 NaN Yes Data Describe : Age Salary count 9.000000 9.000000 mean 38.777778 63777.777778 std 7.693793 12265.579662 min 27.000000 48000.000000 25% 35.000000 54000.000000 50% 38.000000 61000.000000 75% 44.000000 72000.000000 max 50.000000 83000.000000 Input : [['France' 44.0 72000.0] ['Spain' 27.0 48000.0] ['Germany' 30.0 54000.0] ['Spain' 38.0 61000.0] ['Germany' 40.0 nan] ['France' 35.0 58000.0] ['Spain' nan 52000.0] ['France' 48.0 79000.0] ['Germany' 50.0 83000.0] ['France' 37.0 67000.0]] Output: ['No' 'Yes' 'No' 'No' 'Yes' 'Yes' 'No' 'Yes' 'No' 'Yes'] New Input with Mean Value for NaN : [['France' 44.0 72000.0] ['Spain' 27.0 48000.0] ['Germany' 30.0 54000.0] ['Spain' 38.0 61000.0] ['Germany' 40.0 63777.77777777778] ['France' 35.0 58000.0] ['Spain' 38.77777777777778 52000.0] ['France' 48.0 79000.0] ['Germany' 50.0 83000.0] ['France' 37.0 67000.0]]
CODE EXPLANATION :
- Part 1 – Importing Libraries : In the above code, imported numpy, pandas and matplotlib but we have used pandas only.
- PART 2 – Importing Data :
- Import
Data_for_Missing_Values.csv
by giving the path to pandas read_csv function. Now “data_sets” is a DataFrame(Two-dimensional tabular data structure with labeled rows and columns). - Then print first 5 data-entries of the dataframe using head() function. Number of entries can be changed for e.g. for first 3 values we can use dataframe.head(3). Similarly, last values can also be gotten using tail() function.
- Then used describe() function. It gives statistical summary of data which includes min, max, percentile (.25, .5, .75), mean and standard deviation for each parameter values.
- Import
- PART 3 – Input and Output Data : We split our dataframe to input and output.
- PART 4 – Handling the missing values : Using Imputer() function from sklearn.preprocessing package.
IMPUTER :
Imputer(missing_values=’NaN’, strategy=’mean’, axis=0, verbose=0, copy=True)
is a function from Imputer class of sklearn.preprocessing package. It’s role is to transformer parameter value from missing values(NaN) to set strategic value.
Syntax : sklearn.preprocessing.Imputer() Parameters : -> missing_values : integer or “NaN” -> strategy : What to impute - mean, median or most_frequent along axis -> axis(default=0) : 0 means along column and 1 means along row
leave a comment
0 Comments