Answers for "remove duplicate rows pandas give a column"

5

python: remove duplicate in a specific column

df = df.drop_duplicates(subset=['Column1', 'Column2'], keep='first')
Posted by: Guest on July-22-2020
8

remove duplicate row in df

df = df.drop_duplicates()
Posted by: Guest on August-19-2020
0

remove duplicate columns python dataframe

df = df.loc[:,~df.columns.duplicated()]
Posted by: Guest on May-28-2020
0

Return a new DataFrame with duplicate rows removed

# Return a new DataFrame with duplicate rows removed

from pyspark.sql import Row
df = sc.parallelize([
  Row(name='Alice', age=5, height=80),
  Row(name='Alice', age=5, height=80),
  Row(name='Alice', age=10, height=80)]).toDF()
df.dropDuplicates().show()
# +---+------+-----+
# |age|height| name|
# +---+------+-----+
# |  5|    80|Alice|
# | 10|    80|Alice|
# +---+------+-----+

df.dropDuplicates(['name', 'height']).show()
# +---+------+-----+
# |age|height| name|
# +---+------+-----+
# |  5|    80|Alice|
# +---+------+-----+
Posted by: Guest on April-08-2020

Code answers related to "remove duplicate rows pandas give a column"

Python Answers by Framework

Browse Popular Code Answers by Language