Skip to contents

Interface to OpenML Runs

Source: R/OMLRun.R
oml_run.Rd

This is the class for OpenML Runs, which are conceptually similar to mlr3::ResampleResults. This object can also be constructed using the sugar function oml_run().

OpenML Integration

  • A OMLTask is returned by accessing the active field $task.

  • A OMLData is returned by accessing the active field $data (short for $task$data)

  • A OMLFlow is returned by accessing the active field $flow.

  • The raw predictions are returned by accessing the active field $prediction.

mlr3 Integration

References

Vanschoren J, van Rijn JN, Bischl B, Torgo L (2014). “OpenML.” ACM SIGKDD Explorations Newsletter, 15(2), 49--60. doi:10.1145/2641190.2641198 .

Super class

mlr3oml::OMLObject -> OMLRun

Active bindings

flow_id

(integer(1))
The id of the flow.

flow

(OMLFlow)
The OpenML Flow.

tags

(character())
Returns all tags of the object.

parquet

(logical(1))
Whether to use parquet.

task_id

(character(1))
The id of the task solved by this run.

task

(OMLTask)
The task solved by this run.

data_id

(integer(1))
The id of the dataset.

data

(OMLData)
The data used in this run.

task_type

(character())
The task type.

parameter_setting

data.table())
The parameter setting for this run.

prediction

(data.table())
The raw predictions of the run as returned by OpenML, not in standard mlr3 format. Formatted predictions are accessible after converting to a mlr3::ResampleResult via as_resample_result().

evaluation

(data.table())
The evaluations calculated by the OpenML server.

Methods

Public methods

Inherited methods

Method new()

Creates a new instance of this R6 class.

Usage

OMLRun$new(
  id,
  cache = cache_default(),
  parquet = parquet_default(),
  test_server = test_server_default()
)

Arguments

id

(integer(1))
OpenML id for the object.

cache

(logical(1) | character(1))
See field cache for an explanation of possible values. Defaults to value of option "mlr3oml.cache", or FALSE if not set.

parquet

(logical(1))
Whether to use parquet instead of arff. If parquet is not available, it will fall back to arff. Defaults to value of option "mlr3oml.parquet" or FALSE if not set.

test_server

(character(1))
Whether to use the OpenML test server or public server. Defaults to value of option "mlr3oml.test_server", or FALSE if not set.

Method print()

Prints the object.

Usage

OMLRun$print()

Method clone()

The objects of this class are cloneable with this method.

Usage

OMLRun$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

try({
  library("mlr3")
  orun = OMLRun$new(id = 10587724)
  # sugar
  orun = orn(id = 10587724)
  print(orun)
  print(orun$task) # OMLTask
  print(orun$data) # OMLData
  print(orun$flow) # OMLFlow
  print(orun$prediction)
  as_task(orun)
  as_resampling(orun)
  as_data_backend(orun)
  rr = as_resample_result(orun)
  rr$score(msr("classif.ce"))
  }, silent = TRUE)
#> <OMLRun:10587724>
#>  * Task: credit-g (id: 31)
#>  * Flow: mlr3.classif.rpart (id: 19068)
#>  * Estimation: crossvalidation (id: 1; repeats: 1, folds: 10)
#> <OMLTask:31>
#>  * Type: Supervised Classification
#>  * Data: credit-g (id: 31; dim: 1000x21)
#>  * Target: class
#>  * Estimation: crossvalidation (id: 1; repeats: 1, folds: 10)
#> <OMLData:31:credit-g> (1000x21)
#>  * Default target: class
#> <OMLFlow:19068>
#>  * Name: mlr3.classif.rpart
#>  * Dependencies: mlr3_0.13.1-9000, rpart_4.1-15
#>       repeat. fold row_id prediction truth confidence.good confidence.bad
#>    1:       0    0    490       good  good       0.8644068      0.1355932
#>    2:       0    0    406       good  good       0.8644068      0.1355932
#>    3:       0    0    139       good  good       0.8644068      0.1355932
#>    4:       0    0    482        bad  good       0.3846154      0.6153846
#>    5:       0    0    178       good  good       0.8644068      0.1355932
#>   ---                                                                    
#>  996:       0    9    320        bad   bad       0.3333333      0.6666667
#>  997:       0    9    503       good   bad       0.6954733      0.3045267
#>  998:       0    9    822        bad   bad       0.2435897      0.7564103
#>  999:       0    9    558       good   bad       0.6954733      0.3045267
#> 1000:       0    9    714        bad   bad       0.2435897      0.7564103
#>                  task  task_id                      learner learner_id
#>  1: <TaskClassif[50]> credit-g <LearnerClassifOML19068[36]>  oml.19068
#>  2: <TaskClassif[50]> credit-g <LearnerClassifOML19068[36]>  oml.19068
#>  3: <TaskClassif[50]> credit-g <LearnerClassifOML19068[36]>  oml.19068
#>  4: <TaskClassif[50]> credit-g <LearnerClassifOML19068[36]>  oml.19068
#>  5: <TaskClassif[50]> credit-g <LearnerClassifOML19068[36]>  oml.19068
#>  6: <TaskClassif[50]> credit-g <LearnerClassifOML19068[36]>  oml.19068
#>  7: <TaskClassif[50]> credit-g <LearnerClassifOML19068[36]>  oml.19068
#>  8: <TaskClassif[50]> credit-g <LearnerClassifOML19068[36]>  oml.19068
#>  9: <TaskClassif[50]> credit-g <LearnerClassifOML19068[36]>  oml.19068
#> 10: <TaskClassif[50]> credit-g <LearnerClassifOML19068[36]>  oml.19068
#>                 resampling resampling_id iteration              prediction
#>  1: <ResamplingCustom[20]>        custom         1 <PredictionClassif[20]>
#>  2: <ResamplingCustom[20]>        custom         2 <PredictionClassif[20]>
#>  3: <ResamplingCustom[20]>        custom         3 <PredictionClassif[20]>
#>  4: <ResamplingCustom[20]>        custom         4 <PredictionClassif[20]>
#>  5: <ResamplingCustom[20]>        custom         5 <PredictionClassif[20]>
#>  6: <ResamplingCustom[20]>        custom         6 <PredictionClassif[20]>
#>  7: <ResamplingCustom[20]>        custom         7 <PredictionClassif[20]>
#>  8: <ResamplingCustom[20]>        custom         8 <PredictionClassif[20]>
#>  9: <ResamplingCustom[20]>        custom         9 <PredictionClassif[20]>
#> 10: <ResamplingCustom[20]>        custom        10 <PredictionClassif[20]>
#>     classif.ce
#>  1:  0.2000000
#>  2:  0.3200000
#>  3:  0.2500000
#>  4:  0.2525253
#>  5:  0.2900000
#>  6:  0.3100000
#>  7:  0.2800000
#>  8:  0.2100000
#>  9:  0.2500000
#> 10:  0.2100000