Answers for "how to remove duplicated data in pandas dataframe"

remove duplicate row in df

df = df.drop_duplicates()

dataframe delete duplicate rows with same column value

df = df.drop_duplicates(subset=['Column1', 'Column2'], keep='first')

# Exemple
import pandas as pd
df = pd.DataFrame({"A":["foo", "foo", "foo", "bar"], "B":[0,1,1,1], "C":["A","A","B","A"]})
df.drop_duplicates(subset=['A', 'C'], keep=False)

pandas merge two dataframes remove duplicates

concat = pd.merge(data_1, data_2, how='inner')