Compare Learners

Compare learners using the specified loss_metric

Usage

compare_learners(sl_output, y_variable, loss_metric)

Arguments

sl_output

Output from running super_learner() with verbose_output = TRUE.

y_variable

A character vector indicating the outcome variable. y_variable will be automatically inferred if it is missing and can be inferred from the sl_output.

loss_metric

A loss metric, like the mean-squared-error or negative-log-loss to be used in comparing the learners. A loss metric should take two (vector) arguments: predictions, and true outcomes, and produce a single statistic summarizing the performance of each learner.

Value

A data.frame with the loss-metric on the held-out data for each learner.

Examples

if (FALSE) { # \dontrun{
sl_model <- super_learner(
  data = mtcars,
  learners = list(lm = lnr_lm, rf = lnr_rf, mean = lnr_mean),
  formula = mpg ~ .,
  verbose = TRUE)

compare_learners(sl_model)

sl_model <- super_learner(
  data = mtcars,
  learners = list(lnr_logistic, lnr_rf_binary, mean = lnr_mean),
  formula = am ~ mpg,
  outcome_type = 'binary',
  verbose = TRUE)
compare_learners(sl_model)
} # }