Skip to contents

mlr3 with Guild AI

Source: vignettes/examples/mlr3.Rmd
mlr3.Rmd

Here is an example showing how to use guildai with ml3r in R.

If this is your first exposure to guildai, we recommend starting with the framework agnostic “Getting Started” guide. This example assumes familiarity with guild concepts like flags and scalars.

We’ll start with an example R script, named “train-mlr3.R”, that trains, evaluates, and saves an tidymodels model using the “Palmer Penguins” dataset. Here is what it looks like:

library(mlr3)

# create learning task
task_penguins = as_task_classif(species ~ ., data = palmerpenguins::penguins)
task_penguins

cp <- 0.01

# load learner and set hyperparameter
learner = lrn("classif.rpart", cp = cp)

# train/test split
split = partition(task_penguins, ratio = 0.67)

# train the model
learner$train(task_penguins, split$train_set)

# predict data
prediction = learner$predict(task_penguins, split$test_set)

# calculate performance
prediction$confusion

measure = msr("classif.acc")
score <- prediction$score(measure)

# output scalars for guild
writeLines(sprintf(
  "%s: %f",
  names(score), score[[1]]))

A few things to note about the script:

  • There is no ‘guildai’ specific configuration in the script code. It’s a regular R script, that you can safely source() or work with interactively at the REPL.
  • We define one hyperparameter flag: cp
  • At the end of the run, we print metric scalars.

With our script defined, we can launch a guild run and view it:

library(guildai)
guild_run("train-mlr3.R")
## > library(mlr3)
## > # create learning task
## > task_penguins = as_task_classif(species ~ ., data = palmerpenguins::penguins)
## > task_penguins
## <TaskClassif:palmerpenguins::penguins> (344 x 8)
## * Target: species
## * Properties: multiclass
## * Features (7):
##   - int (3): body_mass_g, flipper_length_mm, year
##   - dbl (2): bill_depth_mm, bill_length_mm
##   - fct (2): island, sex
## > cp <- 0.01
## > # load learner and set hyperparameter
## > learner = lrn("classif.rpart", cp = cp)
## > # train/test split
## > split = partition(task_penguins, ratio = 0.67)
## > # train the model
## > learner$train(task_penguins, split$train_set)
## > # predict data
## > prediction = learner$predict(task_penguins, split$test_set)
## > # calculate performance
## > prediction$confusion
##            truth
## response    Adelie Chinstrap Gentoo
##   Adelie       146         5      0
##   Chinstrap      6        63      1
##   Gentoo         0         0    123
## > measure = msr("classif.acc")
## > score <- prediction$score(measure)
## > # output scalars for guild
## > writeLines(sprintf(
## +   "%s: %f",
## +   names(score), score[[1]]))
## classif.acc: 0.965116

Launch a batch of runs with different flag values:

guild_run("train-mlr3.R", echo = FALSE, flags = list(
  cp = c(0.01, 0.05, 0.1)
))

Retrieve run results:

library(dplyr, warn.conflicts = FALSE)
runs_info() %>%
  select(flags, scalars)
## # A tibble: 4 × 2
##   flags$cp scalars$classif.acc
##      <dbl>               <dbl>
## 1     0.1                0.951
## 2     0.05               0.951
## 3     0.01               0.965
## 4     0.01               0.965