Databricks Associate-Developer-Apache-Spark 問題集

質問 1：
Which of the following is not a feature of Adaptive Query Execution?
A. Coalesce partitions to accelerate data processing.
B. Reroute a query in case of an executor failure.
C. Collect runtime statistics during query execution.
D. Replace a sort merge join with a broadcast join, where appropriate.
E. Split skewed partitions into smaller partitions to avoid differences in partition processing time.
正解：B
解説: (Topexam メンバーにのみ表示されます)

質問 2：
Which of the following code blocks returns the number of unique values in column storeId of DataFrame transactionsDf?
A. transactionsDf.dropDuplicates().agg(count("storeId"))
B. transactionsDf.select(distinct("storeId")).count()
C. transactionsDf.select("storeId").dropDuplicates().count()
D. transactionsDf.select(count("storeId")).dropDuplicates()
E. transactionsDf.distinct().select("storeId").count()
正解：C
解説: (Topexam メンバーにのみ表示されます)

質問 3：
The code block displayed below contains an error. The code block below is intended to add a column itemNameElements to DataFrame itemsDf that includes an array of all words in column itemName. Find the error.
Sample of DataFrame itemsDf:
1.+------+----------------------------------+-------------------+
2.|itemId|itemName |supplier |
3.+------+----------------------------------+-------------------+
4.|1 |Thick Coat for Walking in the Snow|Sports Company Inc.|
5.|2 |Elegant Outdoors Summer Dress |YetiX |
6.|3 |Outdoors Backpack |Sports Company Inc.|
7.+------+----------------------------------+-------------------+
Code block:
itemsDf.withColumnRenamed("itemNameElements", split("itemName"))
itemsDf.withColumnRenamed("itemNameElements", split("itemName"))
A. All column names need to be wrapped in the col() operator.
B. Operator withColumnRenamed needs to be replaced with operator withColumn and a second argument
"," needs to be passed to the split method.
C. The expressions "itemNameElements" and split("itemName") need to be swapped.
D. Operator withColumnRenamed needs to be replaced with operator withColumn and the split method needs to be replaced by the splitString method.
E. Operator withColumnRenamed needs to be replaced with operator withColumn and a second argument "
" needs to be passed to the split method.
正解：E
解説: (Topexam メンバーにのみ表示されます)

質問 4：
Which of the following code blocks creates a new DataFrame with 3 columns, productId, highest, and lowest, that shows the biggest and smallest values of column value per value in column productId from DataFrame transactionsDf?
Sample of DataFrame transactionsDf:
1.+-------------+---------+-----+-------+---------+----+
2.|transactionId|predError|value|storeId|productId| f|
3.+-------------+---------+-----+-------+---------+----+
4.| 1| 3| 4| 25| 1|null|
5.| 2| 6| 7| 2| 2|null|
6.| 3| 3| null| 25| 3|null|
7.| 4| null| null| 3| 2|null|
8.| 5| null| null| null| 2|null|
9.| 6| 3| 2| 25| 2|null|
10.+-------------+---------+-----+-------+---------+----+
A. transactionsDf.groupby("productId").agg({"highest": max("value"), "lowest": min("value")})
B. transactionsDf.groupby('productId').agg(max('value').alias('highest'), min('value').alias('lowest'))
C. transactionsDf.groupby(col(productId)).agg(max(col(value)).alias("highest"), min(col(value)).alias("lowest"))
D. transactionsDf.max('value').min('value')
E. transactionsDf.agg(max('value').alias('highest'), min('value').alias('lowest'))
正解：B
解説: (Topexam メンバーにのみ表示されます)

質問 5：
The code block displayed below contains an error. The code block should trigger Spark to cache DataFrame transactionsDf in executor memory where available, writing to disk where insufficient executor memory is available, in a fault-tolerant way. Find the error.
Code block:
transactionsDf.persist(StorageLevel.MEMORY_AND_DISK)
A. Caching is not supported in Spark, data are always recomputed.
B. The code block uses the wrong operator for caching.
C. The storage level is inappropriate for fault-tolerant storage.
D. Data caching capabilities can be accessed through the spark object, but not through the DataFrame API.
E. The DataFrameWriter needs to be invoked.
正解：C
解説: (Topexam メンバーにのみ表示されます)

質問 6：
The code block shown below should read all files with the file ending .png in directory path into Spark.
Choose the answer that correctly fills the blanks in the code block to accomplish this.
spark.__1__.__2__(__3__).option(__4__, "*.png").__5__(path)
A. 1. read()
2. format
3. "binaryFile"
4. "recursiveFileLookup"
5. load
B. 1. read
2. format
3. "binaryFile"
4. "pathGlobFilter"
5. load
C. 1. open
2. as
3. "binaryFile"
4. "pathGlobFilter"
5. load
D. 1. open
2. format
3. "image"
4. "fileType"
5. open
E. 1. read
2. format
3. binaryFile
4. pathGlobFilter
5. load
正解：B
解説: (Topexam メンバーにのみ表示されます)