count missing values by column in pandas
df.isna().sum()check for missing/ nan values in pandas dataframe
In [27]: df
Out[27]:
A B C
1 NaN -2.027325 1.533582
2 NaN NaN 0.461821
3 -0.788073 NaN NaN
4 -0.916080 -0.612343 NaN
5 -0.887858 1.033826 NaN
In [28]: df.isnull().sum() # Returns the sum of NaN values in each column.
Out[28]:
A 2
B 2
C 3
In [29]: df.isnull().sum().sum # Returns the total NaN values in the dataframe
Out[29]:
7filling the missing data in pandas
note:to fill a specific value
varable = 1
def fill_mod_acc(most_related_coloum_name,missing_data_coloum):
if np.isnan(missing_data_coloum):
return varable[most_related_coloum_name]
else:
return missing_data_coloum
df['missing_data_coloum'] = df.apply(lambda x:fill_mod_acc(x['most_related_coloum_name'],x['missing_data_coloum']),axis=1)
Note:to fill mean from existing closley related coloum
varable = df.groupby('most_related_coloum_name').mean()['missing_data_coloum']
def fill_mod_acc(most_related_coloum_name,missing_data_coloum):
if np.isnan(missing_data_coloum):
return varable[most_related_coloum_name]
else:
return missing_data_coloum
df['missing_data_coloum'] = df.apply(lambda x:fill_mod_acc(x['most_related_coloum_name'],x['missing_data_coloum']),axis=1)how to check missing values in python
# Total missing values for each featureprint df.isnull().sum()Out:ST_NUM 2ST_NAME 0OWN_OCCUPIED 2NUM_BEDROOMS 4Copyright © 2021 Codeinu
Forgot your account's password or having trouble logging into your Account? Don't worry, we'll help you to get back your account. Enter your email address and we'll send you a recovery link to reset your password. If you are experiencing problems resetting your password contact us
