Question 1

Which of the following describes a difference between Spark's cluster and client execution modes?

Options :

A : In cluster mode, the cluster manager resides on a worker node, while it resides on an edge node in client mode.

B : In cluster mode, executor processes run on worker nodes, while they run on gateway nodes in client mode.

C : In cluster mode, the driver resides on a worker node, while it resides on an edge node in client mode.

D : In cluster mode, a gateway machine hosts the driver, while it is co-located with the executor in client mode.

E : In cluster mode, the Spark driver is not co-located with the cluster manager, while it is co-located in client mode.

Answer: C

Question 2

The code block shown below should return the number of columns in the CSV file stored at location filePath. From the CSV file, only lines should be read that do not start with a # character. Choose the answer that correctly fills the blanks in the code block to accomplish this

Code block:

__1__(__2__.__3__.csv(filePath, __4__).__5__)

Options :

A :

1. size

2. spark

3. read()

4. escape='#'

5. columns

B :

1. DataFrame

2. spark

3. read()

4. escape='#'

5. shape[0]

C :

1. len

2. pyspark

3. DataFrameReader

4. comment='#'

5. columns

D :

1. size

2. pyspark

3. DataFrameReader

4. comment='#'

5. columns

E : 1. len

2. spark

3. read

4. comment='#'

5. columns

Answer: A

Question 3

Given the following statements regarding caching:

1. 1: The default storage level for a DataFrame is StorageLevel.MEMORY

2.

3. 2: The DataFrame class does have an unpersist() operation

4.

5. 3: The persist() method needs an action to load data from its source to materialize the DataFrame in cache

Which one is NOT TRUE ?

Options :

A : 2

B : 1,2,3

C : 1,3

D : 1

E : 1,2

Answer: D

Question 4

The code block shown below should return a new DataFrame with a new column named “casted” who’s value is the long equivalent of column “a” which is a integer column also this dataframe should contain all the previously existing columns from DataFrame df. Choose the response that correclty fills in the numbered blanks within the code block to complete this task. Code block: df._1_(_2_)

Options :

A :

1. withColumn

2. “casted”

3. cast(a)

B :

1. withColumnRenamed

2. casted

3. col(“a”).cast(“long”)

C :

1. withColumn

2. “casted”

3. cast(col(“a”)

D :

1. withColumnRenamed

2. “casted”

3. col(“a”).cast(“long”)

E :

1. withColumn

2. “casted”

3. col(“a”).cast(“long”)

F :

1. withColumn

2. casted

3. col(a).cast(long)

Answer: A

Question 5

The code block shown below should return a new DataFrame with 25 percent of random records from dataframe df without replacement. Choose the response that

correctly fills in the numbered blanks within the code block to complete this task.

Code block:

df._1_(_2_, _3_, _4_)

Options :

A :

1. sample

2. False

3. 25

4. 5

B :

1. sample

2. False

3. 0.25

4. 5

C :

1. sample

2. True

3. 0.25

4. 5

D :

1. take

2. False

3. 0.25

4. 5

E :

1. random

2. False

3. 0.25

4. 5

F :

1. sample

2. False

3. 0.5

4. 25

Answer: B