Question 1

When preparing to score a new dataset with a predictive model you have previously built, you notice that one of the predictor variables has a higher rate of missing values than in the training set used to build the model. What potential issue should you be wary of?

Options :

A :

The scoring process will automatically become faster due to fewer cases to process.

B :

Predictions made by the model may be less reliable due to the difference in data quality.

C :

The model will automatically discard any new records with missing values in the scoring dataset.

D :

There will be no issues as predictive models are robust to changes in the rate of missing data.

Answer: B

Question 2

You have built a logistic regression model using PROC LOGISTIC to predict the probability of customer churn based on several predictors. You want to evaluate the performance of the model by analyzing the Receiver Operating Characteristic (ROC) curve. To score a validation dataset and produce the ROC curve, which statement correctly implements the OUTROC option in the SCORE statement?

Options :

A :

SCORE DATA=validation OUT=scored_results OUTROC=roc_curve;

B :

PROC LOGISTIC DATA=training; SCORE OUTROC=roc_curve; RUN;

C :

PROC LOGISTIC DATA=training; SCORE DATA=validation OUTROC=roc_curve; RUN;

D :

SCORE DATA=validation OUT=scored_results; MODEL OUTROC=roc_curve;

Answer: C

Question 3

You have developed a predictive model using a very large set of independent variables. Despite achieving outstanding performance on your training dataset, when you apply your model to new, unseen data, the performance significantly deteriorates. Which of the following is the most likely explanation for this occurrence?

Options :

A :

Underfitting due to insufficient model complexity

B :

Overfitting due to the model being too complex and capturing noise in the training data

C :

Data leakage, where information from outside the training dataset has been incorrectly included

D :

The new data has a different distribution than the training data, which is an instance of dataset shift

Answer: B

Question 4

A data scientist is evaluating the performance of a neural network model for a regression problem. After training, the scientist wants to assess the model's ability to generalize to new data. Which of the following metrics is the most appropriate for evaluating the performance of the neural network on a validation dataset?

Options :

A :

Accuracy

B :

Mean Squared Error (MSE)

C :

Area Under the ROC Curve (AUC)

D :

Precision

Answer: B

Question 5

You are developing a predictive model using SAS and have completed the model training phase. You are now preparing to deploy the model for scoring a new dataset. Which mode should you operate in to score the new dataset and why?

Options :

A :

Training mode, because the model needs to be retrained on the new dataset before it can be scored.

B :

Training mode with cross-validation, to ensure that the model generalizes well to the new dataset.

C :

Output mode, as the model is already trained and the goal now is to use the model to predict new outcomes.

D : Output mode, but with added regularization parameters to avoid overfitting on the new dataset.

Answer: C