Question 1

Which of the following code blocks immediately removes the previously cached DataFrame transactionsDf from memory and disk?

Options :

A : array_remove(transactionsDf, "*")

B : transactionsDf.unpersist()

C : del transactionsDf

D : transactionsDf.clearCache()

E : transactionsDf.persist()

Answer: B

Question 2

Which of the following is correct for the cluster manager ?

Options :

A : Executes code assigned to it.

B : Keeps track of the resources available.

C :

It is interchangeable with job.

D : Reports the state of the computation back to the driver.

E : All of the answers are correct.

Answer: B

Question 3

How to make sure that dataframe df has 12 partitions given that df has 4 partitions ?

Options :

A : df.repartition(12)

B : df.setPartitition(12)

C : df.repartition()

D : df.setPartitition()

Answer: A

Question 4

The code block displayed below contains an error. The code block should return DataFrame transactionsDf,

but with the column storeId renamed to storeNumber. Find the error.

Code block:

transactionsDf.withColumn("storeNumber", "storeId")

Options :

A : Instead of withColumn, the withColumnRenamed method should be used.

B : Arguments "storeNumber" and "storeId" each need to be wrapped in a col() operator.

C : Argument "storeId" should be the first and argument "storeNumber" should be the second argument to the withColumn method

D : The withColumn operator should be replaced with the copyDataFrame operator.

E : Instead of withColumn, the withColumnRenamed method should be used and argument "storeId" should be the first and argument "storeNumber" should be the second argument to that method.

Answer: A

Question 5

Which of the following code blocks creates a new DataFrame with 3 columns, productId, highest, and lowest,

that shows the biggest and smallest values of column value per value in column

productId from DataFrame transactionsDf?

Sample of DataFrame transactionsDf:

1.+-------------+---------+-----+-------+---------+----+

3.+-------------+---------+-----+-------+---------+----+

4.| 1| 3| 4| 25| 1|null|

5.| 2| 6| 7| 2| 2|null|

6.| 3| 3| null| 25| 3|null|

7.| 4| null| null| 3| 2|null|

8.| 5| null| null| null| 2|null|

9.| 6| 3| 2| 25| 2|null|

10.+-------------+---------+-----+-------+---------+----+

Options :

A : transactionsDf.max('value').min('value')

B : transactionsDf.agg(max('value').alias('highest'), min('value').alias('lowest'))

C : transactionsDf.groupby(col(productId)).agg(max(col(value)).alias("highest"), min(col(value)).alias("lowest"))

D : transactionsDf.groupby('productId').agg(max('value').alias('highest'), min('value').alias('lowest'))

E : transactionsDf.groupby("productId").agg({"highest": max("value"), "lowest": min("value")})

Answer: D