Gradient Boosted Decision Trees Classification Learner
Source:R/learner_catboost_classif_catboost.R
      mlr_learners_classif.catboost.RdGradient boosting algorithm that also supports categorical data.
Calls catboost::catboost.train() from package 'catboost'.
Meta Information
- Task type: “classif” 
- Predict Types: “response”, “prob” 
- Feature Types: “numeric”, “factor”, “ordered” 
- Required Packages: mlr3, mlr3extralearners, catboost 
Parameters
| Id | Type | Default | Levels | Range | 
| loss_function_twoclass | character | Logloss | Logloss, CrossEntropy | - | 
| loss_function_multiclass | character | MultiClass | MultiClass, MultiClassOneVsAll | - | 
| learning_rate | numeric | 0.03 | \([0.001, 1]\) | |
| random_seed | integer | 0 | \([0, \infty)\) | |
| l2_leaf_reg | numeric | 3 | \([0, \infty)\) | |
| bootstrap_type | character | - | Bayesian, Bernoulli, MVS, Poisson, No | - | 
| bagging_temperature | numeric | 1 | \([0, \infty)\) | |
| subsample | numeric | - | \([0, 1]\) | |
| sampling_frequency | character | PerTreeLevel | PerTree, PerTreeLevel | - | 
| sampling_unit | character | Object | Object, Group | - | 
| mvs_reg | numeric | - | \([0, \infty)\) | |
| random_strength | numeric | 1 | \([0, \infty)\) | |
| depth | integer | 6 | \([1, 16]\) | |
| grow_policy | character | SymmetricTree | SymmetricTree, Depthwise, Lossguide | - | 
| min_data_in_leaf | integer | 1 | \([1, \infty)\) | |
| max_leaves | integer | 31 | \([1, \infty)\) | |
| ignored_features | untyped | NULL | - | |
| one_hot_max_size | untyped | FALSE | - | |
| has_time | logical | FALSE | TRUE, FALSE | - | 
| rsm | numeric | 1 | \([0.001, 1]\) | |
| nan_mode | character | Min | Min, Max | - | 
| fold_permutation_block | integer | - | \([1, 256]\) | |
| leaf_estimation_method | character | - | Newton, Gradient, Exact | - | 
| leaf_estimation_iterations | integer | - | \([1, \infty)\) | |
| leaf_estimation_backtracking | character | AnyImprovement | No, AnyImprovement, Armijo | - | 
| fold_len_multiplier | numeric | 2 | \([1.001, \infty)\) | |
| approx_on_full_history | logical | TRUE | TRUE, FALSE | - | 
| class_weights | untyped | - | - | |
| auto_class_weights | character | None | None, Balanced, SqrtBalanced | - | 
| boosting_type | character | - | Ordered, Plain | - | 
| boost_from_average | logical | - | TRUE, FALSE | - | 
| langevin | logical | FALSE | TRUE, FALSE | - | 
| diffusion_temperature | numeric | 10000 | \([0, \infty)\) | |
| score_function | character | Cosine | Cosine, L2, NewtonCosine, NewtonL2 | - | 
| monotone_constraints | untyped | - | - | |
| feature_weights | untyped | - | - | |
| first_feature_use_penalties | untyped | - | - | |
| penalties_coefficient | numeric | 1 | \([0, \infty)\) | |
| per_object_feature_penalties | untyped | - | - | |
| model_shrink_rate | numeric | - | \((-\infty, \infty)\) | |
| model_shrink_mode | character | - | Constant, Decreasing | - | 
| target_border | numeric | - | \((-\infty, \infty)\) | |
| border_count | integer | - | \([1, 65535]\) | |
| feature_border_type | character | GreedyLogSum | Median, Uniform, UniformAndQuantiles, MaxLogSum, MinEntropy, GreedyLogSum | - | 
| per_float_feature_quantization | untyped | - | - | |
| classes_count | integer | - | \([1, \infty)\) | |
| thread_count | integer | 1 | \([-1, \infty)\) | |
| task_type | character | CPU | CPU, GPU | - | 
| devices | untyped | - | - | |
| logging_level | character | Silent | Silent, Verbose, Info, Debug | - | 
| metric_period | integer | 1 | \([1, \infty)\) | |
| train_dir | untyped | "catboost_info" | - | |
| model_size_reg | numeric | 0.5 | \([0, 1]\) | |
| allow_writing_files | logical | FALSE | TRUE, FALSE | - | 
| save_snapshot | logical | FALSE | TRUE, FALSE | - | 
| snapshot_file | untyped | - | - | |
| snapshot_interval | integer | 600 | \([1, \infty)\) | |
| simple_ctr | untyped | - | - | |
| combinations_ctr | untyped | - | - | |
| ctr_target_border_count | integer | - | \([1, 255]\) | |
| counter_calc_method | character | Full | SkipTest, Full | - | 
| max_ctr_complexity | integer | - | \([1, \infty)\) | |
| ctr_leaf_count_limit | integer | - | \([1, \infty)\) | |
| store_all_simple_ctr | logical | FALSE | TRUE, FALSE | - | 
| final_ctr_computation_mode | character | Default | Default, Skip | - | 
| verbose | logical | FALSE | TRUE, FALSE | - | 
| ntree_start | integer | 0 | \([0, \infty)\) | |
| ntree_end | integer | 0 | \([0, \infty)\) | |
| early_stopping_rounds | integer | - | \([1, \infty)\) | |
| eval_metric | untyped | - | - | |
| use_best_model | logical | - | TRUE, FALSE | - | 
| iterations | integer | 1000 | \([1, \infty)\) | 
Initial parameter values
- logging_level:- Actual default: "Verbose" 
- Adjusted default: "Silent" 
- Reason for change: consistent with other mlr3 learners 
 
- thread_count:- Actual default: -1 
- Adjusted default: 1 
- Reason for change: consistent with other mlr3 learners 
 
- allow_writing_files:- Actual default: TRUE 
- Adjusted default: FALSE 
- Reason for change: consistent with other mlr3 learners 
 
- save_snapshot:- Actual default: TRUE 
- Adjusted default: FALSE 
- Reason for change: consistent with other mlr3 learners 
 
Early stopping
Early stopping can be used to find the optimal number of boosting rounds.
Set early_stopping_rounds to an integer value to monitor the performance of the model on the validation set while training.
For information on how to configure the validation set, see the Validation section of mlr3::Learner.
References
Dorogush, Veronika A, Ershov, Vasily, Gulin, Andrey (2018). “CatBoost: gradient boosting with categorical features support.” arXiv preprint arXiv:1810.11363.
See also
- as.data.table(mlr_learners)for a table of available Learners in the running session (depending on the loaded packages).
- Chapter in the mlr3book: https://mlr3book.mlr-org.com/basics.html#learners 
- mlr3learners for a selection of recommended learners. 
- mlr3cluster for unsupervised clustering learners. 
- mlr3pipelines to combine learners with pre- and postprocessing steps. 
- mlr3tuning for tuning of hyperparameters, mlr3tuningspaces for established default tuning spaces. 
Super classes
mlr3::Learner -> mlr3::LearnerClassif -> LearnerClassifCatboost
Active bindings
- internal_valid_scores
- The last observation of the validation scores for all metrics. Extracted from - model$evaluation_log
- internal_tuned_values
- Returns the early stopped iterations if - early_stopping_roundswas set during training.
- validate
- How to construct the internal validation data. This parameter can be either - NULL, a ratio,- "test", or- "predefined".
Methods
Inherited methods
mlr3::Learner$base_learner()mlr3::Learner$configure()mlr3::Learner$encapsulate()mlr3::Learner$format()mlr3::Learner$help()mlr3::Learner$predict()mlr3::Learner$predict_newdata()mlr3::Learner$print()mlr3::Learner$reset()mlr3::Learner$selected_features()mlr3::Learner$train()mlr3::LearnerClassif$predict_newdata_fast()
Method importance()
The importance scores are calculated using
catboost.get_feature_importance,
setting type = "FeatureImportance", returned for 'all'.
Returns
Named numeric().
Examples
# Define the Learner
learner = lrn("classif.catboost",
  iterations = 100)
print(learner)
#> 
#> ── <LearnerClassifCatboost> (classif.catboost): Gradient Boosting ──────────────
#> • Model: -
#> • Parameters: loss_function_twoclass=Logloss,
#> loss_function_multiclass=MultiClass, thread_count=1, logging_level=Silent,
#> allow_writing_files=FALSE, save_snapshot=FALSE, iterations=100
#> • Validate: NULL
#> • Packages: mlr3, mlr3extralearners, and catboost
#> • Predict Types: [response] and prob
#> • Feature Types: numeric, factor, and ordered
#> • Encapsulation: none (fallback: -)
#> • Properties: importance, internal_tuning, missings, multiclass, twoclass,
#> validation, and weights
#> • Other settings: use_weights = 'use'
# Define a Task
task = tsk("sonar")
# Create train and test set
ids = partition(task)
# Train the learner on the training ids
learner$train(task, row_ids = ids$train)
print(learner$model)
#> CatBoost model (100 trees)
#> Loss function: Logloss
#> Fit to 60 feature(s)
print(learner$importance())
#>        V11        V52         V9         V4        V48        V28        V49 
#> 13.4068467  4.7043504  4.4769022  3.9568505  3.9420870  3.4950025  3.3289798 
#>        V21        V16        V12        V47        V37        V17        V36 
#>  3.3141801  3.0563047  3.0202089  2.8416233  2.7474523  2.6578850  2.3810853 
#>        V51        V29        V26        V45        V27        V23        V44 
#>  2.1153259  1.8725492  1.7771297  1.6911578  1.6488590  1.6158658  1.5936330 
#>        V46        V15        V31        V14        V59        V13        V50 
#>  1.5314877  1.4320813  1.3193179  1.2806567  1.2540768  1.2528946  1.1978170 
#>        V39        V22        V38        V19        V41        V10        V58 
#>  1.1328927  1.1301651  1.0543565  1.0409318  0.9597303  0.9491879  0.9329649 
#>        V54        V20         V6        V57        V53        V34        V32 
#>  0.8846069  0.8806102  0.8570039  0.7719507  0.7675676  0.7396317  0.7105359 
#>         V2         V5        V60         V7        V55        V40         V1 
#>  0.7101249  0.6986390  0.6595744  0.6413829  0.6393825  0.5345535  0.5292179 
#>        V43        V25         V3        V24        V56        V33        V30 
#>  0.4644390  0.4208437  0.4065955  0.3792747  0.3741328  0.3713442  0.3692405 
#>         V8        V35        V18        V42 
#>  0.3573546  0.3374753  0.2061220  0.1755548 
# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)
# Score the predictions
predictions$score()
#> classif.ce 
#>  0.1594203