We were using spark in Scala environment for a long time and it did its job well. Now we need to query spark in python3 scripts and this post is a simple how-to guide for pyspark.
pyspark is in your
SPARK_HOME/python, you need to put it into you python path. You can
ln the directory to you python3 path, another method is use
sys.path.append in your scripts.
from pyspark import SparkContext, SparkConf