pyspark session
from pyspark.sql import SparkSession
spark = SparkSession.builder.master("yarn").appName("my app").enableHiveSupport().getOrCreate()pyspark session
from pyspark.sql import SparkSession
spark = SparkSession.builder.master("yarn").appName("my app").enableHiveSupport().getOrCreate()Gets an existing SparkSession or, if there is no existing one, creates a new
# Gets an existing SparkSession or, if there is no one, creates a new
s1 = SparkSession.builder.config("k1", "v1").getOrCreate()
s1.conf.get("k1") == s1.sparkContext.getConf().get("k1") == "v1"
# True
s2 = SparkSession.builder.config("k2", "v2").getOrCreate()
s1.conf.get("k1") == s2.conf.get("k1")
# True
s1.conf.get("k2") == s2.conf.get("k2")
# TrueCopyright © 2021 Codeinu
Forgot your account's password or having trouble logging into your Account? Don't worry, we'll help you to get back your account. Enter your email address and we'll send you a recovery link to reset your password. If you are experiencing problems resetting your password contact us
