Question 1

A Spark engineer must select an appropriate deployment mode for the Spark jobs. What is the benefit of using cluster mode in Apache Spark™?

Options :

A : In cluster mode, resources are allocated from a resource manager on the cluster, enabling better performance and scalability for large jobs

B : In cluster mode, the driver is responsible for executing all tasks locally without distributing them across the worker nodes.

C : In cluster mode, the driver runs on the client machine, which can limit the application's ability to handle large datasets efficiently.

D : In cluster mode, the driver program runs on one of the worker nodes, allowing the application to fully utilize the distributed resources of the cluster.

Answer: D

Question 2

A data engineer observes that an upstream streaming source sends duplicate records, where duplicates share the same key and have at most a 30-minute difference inevent_timestamp. The engineer adds: dropDuplicatesWithinWatermark("event_timestamp", "30 minutes") What is the result?

Options :

A : It is not able to handle deduplication in this scenario

B : It removes duplicates that arrive within the 30-minute window specified by the watermark

C : It removes all duplicates regardless of when they arrive

D : It accepts watermarks in seconds and the code results in an error

Answer: B

Question 3

Which Spark configuration controls the number of tasks that can run in parallel on the executor? Options:

Options :

A : spark.executor.cores

B : spark.task.maxFailures

C : spark.driver.cores

D : spark.executor.memory

Answer: A

Question 4

Given a DataFramedfthat has 10 partitions, after running the code: result = df.coalesce(20) How many partitions will the result DataFrame have?

Options :

A : 10

B : Same number as the cluster executors

C : 1

D : 20

Answer: A

Question 5

A data engineer needs to write a Streaming DataFrame as Parquet files. Given the code:Given the code:

Options :

A : format("parquet") .option("location", "path/to/destination/dir")

B : CopyEdit .option("format", "parquet") .option("destination", "path/to/destination/dir")

C : .option("format", "parquet") .option("location", "path/to/destination/dir")

D : format("parquet") .option("path", "path/to/destination/dir")

Answer: D