Extreme Gradient Boosting AFT Survival Learner
Source:R/learner_xgboost_surv_xgboost_aft.R
mlr_learners_surv.xgboost.aft.RdeXtreme Gradient Boosting regression using an Accelerated Failure Time
objective.
Calls xgboost::xgb.train() from package xgboost with objective
set to survival:aft and eval_metric to aft-nloglik.
Note
To compute on GPUs, you first need to compile xgboost yourself and link against CUDA. See https://xgboost.readthedocs.io/en/stable/build.html#building-with-gpu-support.
Prediction types
This learner returns three prediction types:
response: the estimated survival time \(T\) for each test observation.lp: a vector of linear predictors (relative risk scores), one per observation, estimated as \(-log(T)\). Higher survival time denotes lower risk.crank: same aslp.
Early stopping
Early stopping can be used to find the optimal number of boosting rounds.
The early_stopping_set parameter controls which set is used to monitor the
performance.
By default, early_stopping_set = "none" which disables early stopping.
Set early_stopping_set = "test" to monitor the performance of the model on
the test set while training.
The test set for early stopping can be set with the "test" row role in the
mlr3::Task.
Additionally, the range must be set in which the performance must increase
with early_stopping_rounds and the maximum number of boosting rounds with
nrounds.
While resampling, the test set is automatically applied from the mlr3::Resampling.
Not that using the test set for early stopping can potentially bias the
performance scores.
Meta Information
Task type: “surv”
Predict Types: “crank”, “lp”, “response”
Feature Types: “integer”, “numeric”
Required Packages: mlr3, mlr3proba, mlr3extralearners, xgboost
Parameters
| Id | Type | Default | Levels | Range |
| aft_loss_distribution | character | normal | normal, logistic, extreme | - |
| aft_loss_distribution_scale | numeric | - | \((-\infty, \infty)\) | |
| alpha | numeric | 0 | \([0, \infty)\) | |
| base_score | numeric | 0.5 | \((-\infty, \infty)\) | |
| booster | character | gbtree | gbtree, gblinear, dart | - |
| callbacks | untyped | NULL | - | |
| seed | integer | - | \((-\infty, \infty)\) | |
| colsample_bylevel | numeric | 1 | \([0, 1]\) | |
| colsample_bynode | numeric | 1 | \([0, 1]\) | |
| colsample_bytree | numeric | 1 | \([0, 1]\) | |
| disable_default_eval_metric | logical | FALSE | TRUE, FALSE | - |
| evals | untyped | NULL | - | |
| early_stopping_rounds | integer | NULL | \([1, \infty)\) | |
| learning_rate | numeric | 0.3 | \([0, 1]\) | |
| feature_selector | character | cyclic | cyclic, shuffle, random, greedy, thrifty | - |
| gamma | numeric | 0 | \([0, \infty)\) | |
| grow_policy | character | depthwise | depthwise, lossguide | - |
| interaction_constraints | untyped | - | - | |
| iterationrange | untyped | - | - | |
| lambda | numeric | 1 | \([0, \infty)\) | |
| max_bin | integer | 256 | \([2, \infty)\) | |
| max_delta_step | numeric | 0 | \([0, \infty)\) | |
| max_depth | integer | 6 | \([0, \infty)\) | |
| max_leaves | integer | 0 | \([0, \infty)\) | |
| maximize | logical | NULL | TRUE, FALSE | - |
| min_child_weight | numeric | 1 | \([0, \infty)\) | |
| monotone_constraints | integer | 0 | \([-1, 1]\) | |
| normalize_type | character | tree | tree, forest | - |
| nrounds | integer | - | \([1, \infty)\) | |
| nthread | integer | - | \([1, \infty)\) | |
| num_parallel_tree | integer | 1 | \([1, \infty)\) | |
| one_drop | logical | FALSE | TRUE, FALSE | - |
| print_every_n | integer | 1 | \([1, \infty)\) | |
| rate_drop | numeric | 0 | \([0, 1]\) | |
| refresh_leaf | logical | TRUE | TRUE, FALSE | - |
| sampling_method | character | uniform | uniform, gradient_based | - |
| sample_type | character | uniform | uniform, weighted | - |
| save_name | untyped | - | - | |
| save_period | integer | - | \([0, \infty)\) | |
| scale_pos_weight | numeric | 1 | \((-\infty, \infty)\) | |
| seed_per_iteration | logical | FALSE | TRUE, FALSE | - |
| skip_drop | numeric | 0 | \([0, 1]\) | |
| use_rmm | logical | FALSE | TRUE, FALSE | - |
| max_cached_hist_node | integer | NULL | \((-\infty, \infty)\) | |
| extmem_single_page | logical | FALSE | TRUE, FALSE | - |
| max_cat_to_onehot | integer | 4 | \((-\infty, \infty)\) | |
| max_cat_threshold | integer | 64 | \((-\infty, \infty)\) | |
| subsample | numeric | 1 | \([0, 1]\) | |
| top_k | integer | 0 | \([0, \infty)\) | |
| tree_method | character | auto | auto, exact, approx, hist, gpu_hist | - |
| updater | untyped | - | - | |
| verbose | integer | - | \([0, 2]\) | |
| verbosity | integer | - | \([0, 2]\) | |
| xgb_model | untyped | - | - | |
| device | untyped | - | - | |
| missing | numeric | NA | \((-\infty, \infty)\) | |
| validate_features | logical | TRUE | TRUE, FALSE | - |
Initial parameter values
nroundsis initialized to 1000.nthreadis initialized to 1 to avoid conflicts with parallelization via future.verboseandverbosityare both initialized to 0.
References
Chen, Tianqi, Guestrin, Carlos (2016). “Xgboost: A scalable tree boosting system.” In Proceedings of the 22nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 785–794. ACM. doi:10.1145/2939672.2939785 .
Avinash B, Hyunsu C, Toby H (2022). “Survival Regression with Accelerated Failure Time Model in XGBoost.” Journal of Computational and Graphical Statistics. ISSN 15372715, doi:10.1080/10618600.2022.2067548 .
See also
as.data.table(mlr_learners)for a table of available Learners in the running session (depending on the loaded packages).Chapter in the mlr3book: https://mlr3book.mlr-org.com/basics.html#learners
mlr3learners for a selection of recommended learners.
mlr3cluster for unsupervised clustering learners.
mlr3pipelines to combine learners with pre- and postprocessing steps.
mlr3tuning for tuning of hyperparameters, mlr3tuningspaces for established default tuning spaces.
Super classes
mlr3::Learner -> mlr3proba::LearnerSurv -> LearnerSurvXgboostAFT
Active bindings
internal_valid_scoresThe last observation of the validation scores for all metrics. Extracted from
model$evaluation_loginternal_tuned_valuesReturns the early stopped iterations if
early_stopping_roundswas set during training.validateHow to construct the internal validation data. This parameter can be either
NULL, a ratio,"test", or"predefined".model(any)
The fitted model. Only available after$train()has been called.
Methods
Method importance()
The importance scores are calculated with xgboost::xgb.importance().
Returns
Named numeric().
Examples
# Define the Learner
learner = lrn("surv.xgboost.aft")
print(learner)
#>
#> ── <LearnerSurvXgboostAFT> (surv.xgboost.aft): Extreme Gradient Boosting AFT ───
#> • Model: -
#> • Parameters: nrounds=1000, nthread=1, verbose=0, verbosity=0
#> • Validate: NULL
#> • Packages: mlr3, mlr3proba, mlr3extralearners, and xgboost
#> • Predict Types: [crank], lp, and response
#> • Feature Types: integer and numeric
#> • Encapsulation: none (fallback: -)
#> • Properties: importance, internal_tuning, missings, validation, and weights
#> • Other settings: use_weights = 'use'
# Define a Task
task = tsk("grace")
# Create train and test set
ids = partition(task)
# Train the learner on the training ids
learner$train(task, row_ids = ids$train)
print(learner$model)
#> ##### xgb.Booster
#> call:
#> xgboost::xgb.train(params = pv[names(pv) %in% formalArgs(xgboost::xgb.params)],
#> data = data, nrounds = pv$nrounds, evals = pv$evals, verbose = pv$verbose,
#> print_every_n = pv$print_every_n, early_stopping_rounds = pv$early_stopping_rounds,
#> maximize = pv$maximize, save_period = pv$save_period, save_name = pv$save_name,
#> callbacks = pv$callbacks %??% list())
#> # of features: 6
#> # of rounds: 1000
print(learner$importance())
#> revascdays revasc los sysbp age stchange
#> 0.333152092 0.319069397 0.138087356 0.114430717 0.087400950 0.007859488
# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)
# Score the predictions
predictions$score()
#> surv.cindex
#> 0.8323826