Answers for "add new column to spark df"

2

spark add column to dataframe

from pyspark.sql.functions import lit

df = sqlContext.createDataFrame(
    [(1, "a", 23.0), (3, "B", -23.0)], ("x1", "x2", "x3"))

df_with_x4 = df.withColumn("x4", lit(0))
df_with_x4.show()
Posted by: Guest on June-09-2020
0

spark dataframe add column with function

val myDF = sqlContext.parquetFile("hdfs:/to/my/file.parquet")

myDF.withColumn("Code", coder(myDF("Amt")))
Posted by: Guest on July-29-2020

Python Answers by Framework

Browse Popular Code Answers by Language