Answers for "handling missing or NaN values in pandas dataframe"

count nan pandas

#Python, pandas
#Count missing values for each column of the dataframe df

df.isnull().sum()

find nan values in a column pandas

df.isnull().values.any()

find nan values in a column pandas

df['your column name'].isnull().values.any()

handling missing or NaN values in pandas dataframe

# Six(6) ways to handle NaN values

# 1. Drop/delete any rows with NaN values
df.dropna(axis = 0) #row is axis = 0
# 2. Drop/delete any columns with NaN values
df.dropna(axis = 1) #column is axis = 1
# 3. Replace all NaN values with 0
df.fillna(0) 
# 4. Replace NaN values with the previous value in the column, Fill Forward
df.fillna(method = 'ffill', axis = 0) #OR axis = 1 for rows
# 5. Replace NaN values with the next value in the column, Fill Backward
df.fillna(method = 'backfill', axis = 0) #OR axis =1 for rows
# 6. replace NaN values by using linear interpolation using column values
df.interpolate(method = 'linear', axis = 0) #OR axis = 1 for rows

#NB: 1. For the last three options, depending on the method, changes to NaN 
# in the first row, last row, first column or last column may not be effected.
# 2. Remember to include inplace = True if you want the original dataframe to
#be modified, else the changes will revert back to the original when you 
#reference the dataframe again. Eg.
df.dropna(axis = 0, inplace = True)

check for missing/ nan values in pandas dataframe

In [27]: df 
Out[27]: 
          A         B         C
1       NaN -2.027325  1.533582
2       NaN       NaN  0.461821
3 -0.788073       NaN       NaN
4 -0.916080 -0.612343       NaN
5 -0.887858  1.033826       NaN
 
In [28]: df.isnull().sum() # Returns the sum of NaN values in each column.
Out[28]: 
A   2
B   2
C   3

In [29]: df.isnull().sum().sum # Returns the total NaN values in the dataframe
Out[29]: 
7

check for missing values in pandas

df.isna()