Question 1

If spark is running in client mode, which of the following statement about is correct ?

Options :

A : Spark driver is attributed to the machine that has the most resources

B : Spark driver is randomly attributed to a machine in the cluster

C : The entire spark application is run on a single machine

D : Spark driver remainse on the client machine that submitted the application.

Answer: D

Question 2

Given the code block down below, a database test and a dataframe containing nulls, identify the error.

1. spark.udf.register("strlen", (s: String) => s.length)

2. Spark.sql("select s from test where s is not null and strlen(s) > 1")

Options :

A : This WHERE clause does not guarantee the strlen UDF to be invoked after filtering out nulls. So we will have null pointer

B : We need to create the function first and then pass it to udf.register

C : We need to use function ‘query’ instead of ‘sql’ to query table test.

Answer: A

Question 3

When joining two dataframes, if there is a need to evaluate the keys in both of the DataFrames or tables and include all rows from the right DataFrame as well as any rows

in the left DataFrame that have a match in the righ DataFrame also If there is no equivalent row in the left DataFrame, we want to instert null: which join type we should

select ? df1.join(person, joinExpression, joinType)

Options :

A :

joinType = “right_outer”

B : joinType = “rightSemi”

C :

joinType = “rightAnti”

D :

joinType = “rightOuter”

Answer: A

Question 4

Which of the followings are useful use cases of spark ?

Options :

A : Analyzing graph data sets and social networks

B : Building, training, and evaluating machine learning models using MLlib

C : Processing in parallel large data sets distributed across a cluster

D : Performing ad hoc or interactive queries to explore and visualize data sets

E : All of the answers are correct.

Answer: A

Question 5

Which of the following code blocks immediately removes the previously cached DataFrame transactionsDf from memory and disk?

Options :

A : array_remove(transactionsDf, "*")

B : transactionsDf.unpersist()

C : del transactionsDf

D : transactionsDf.clearCache()

E : transactionsDf.persist()

Answer: B