Answers for "how to convert dataframe to spark table"

convert pandas dataframe to spark dataframe

import pandas as pd
from pyspark.sql import SparkSession

filename = <'path to file'>
spark = SparkSession.build.appName('pandasToSpark').getOrCreate()
# Assuming file is csv
pandas_df = pd.read_csv(filename)
spark_df = spark.CreateDataFrame(pandas_df)

Posted by: Guest on March-14-2021

Code answers related to "how to convert dataframe to spark table"

Code answers related to "Python"

Python Answers by Framework

Browse Popular Code Answers by Language

Answers for "how to convert dataframe to spark table"

Code answers related to "how to convert dataframe to spark table"

Code answers related to "Python"

Python Answers by Framework

Browse Popular Code Answers by Language

Answers for "how to convert dataframe to spark table"

Code answers related to "how to convert dataframe to spark table"

Code answers related to "Python"

Python Answers by Framework

Browse Popular Code Answers by Language

Popular Programming Languages

Advertisements

Company

Compilers

Help

Connect with us