Answers for "pandas remove duplicates based on two columns"

python: remove duplicate in a specific column

df = df.drop_duplicates(subset=['Column1', 'Column2'], keep='first')

remove duplicates based on two columns in dataframe

df.drop_duplicates(['A','B'],keep= 'last')

remove duplicate columns python dataframe

df = df.loc[:,~df.columns.duplicated()]

pd.merge duplicate columns remove

#Create test data
df1 = pd.DataFrame(np.random.randint(100,size=(1000, 3)),columns=['A','B','C'])
df2 = pd.DataFrame(np.random.randint(100,size=(1000, 3)),columns=['B','C','D'])

pd.merge(df1, df2, how='inner', left_on=['B','C'], right_on=['B','C'])