Gradient Boosted Decision Trees Classification Learner

Gradient boosting algorithm that also supports categorical data. Calls catboost::catboost.train() from package 'catboost'.

Dictionary

This Learner can be instantiated via the dictionary mlr_learners or with the associated sugar function lrn():

mlr_learners$get("classif.catboost")
lrn("classif.catboost")

Meta Information

• Task type: “classif”

• Predict Types: “response”, “prob”

• Feature Types: “numeric”, “factor”, “ordered”

• Required Packages: mlr3, mlr3extralearners, catboost

Parameters

Id	Type	Default	Levels	Range
loss_function_twoclass	character	Logloss	Logloss, CrossEntropy	-
loss_function_multiclass	character	MultiClass	MultiClass, MultiClassOneVsAll	-
iterations	integer	1000		\([1, \infty)\)
learning_rate	numeric	0.03		\([0.001, 1]\)
random_seed	integer	0		\([0, \infty)\)
l2_leaf_reg	numeric	3		\([0, \infty)\)
bootstrap_type	character	-	Bayesian, Bernoulli, MVS, Poisson, No	-
bagging_temperature	numeric	1		\([0, \infty)\)
subsample	numeric	-		\([0, 1]\)
sampling_frequency	character	PerTreeLevel	PerTree, PerTreeLevel	-
sampling_unit	character	Object	Object, Group	-
mvs_reg	numeric	-		\([0, \infty)\)
random_strength	numeric	1		\([0, \infty)\)
depth	integer	6		\([1, 16]\)
grow_policy	character	SymmetricTree	SymmetricTree, Depthwise, Lossguide	-
min_data_in_leaf	integer	1		\([1, \infty)\)
max_leaves	integer	31		\([1, \infty)\)
ignored_features	untyped			-
one_hot_max_size	untyped	FALSE		-
has_time	logical	FALSE	TRUE, FALSE	-
rsm	numeric	1		\([0.001, 1]\)
nan_mode	character	Min	Min, Max	-
fold_permutation_block	integer	-		\([1, 256]\)
leaf_estimation_method	character	-	Newton, Gradient, Exact	-
leaf_estimation_iterations	integer	-		\([1, \infty)\)
leaf_estimation_backtracking	character	AnyImprovement	No, AnyImprovement, Armijo	-
fold_len_multiplier	numeric	2		\([1.001, \infty)\)
approx_on_full_history	logical	TRUE	TRUE, FALSE	-
class_weights	untyped	-		-
auto_class_weights	character	None	None, Balanced, SqrtBalanced	-
boosting_type	character	-	Ordered, Plain	-
boost_from_average	logical	-	TRUE, FALSE	-
langevin	logical	FALSE	TRUE, FALSE	-
diffusion_temperature	numeric	10000		\([0, \infty)\)
score_function	character	Cosine	Cosine, L2, NewtonCosine, NewtonL2	-
monotone_constraints	untyped	-		-
feature_weights	untyped	-		-
first_feature_use_penalties	untyped	-		-
penalties_coefficient	numeric	1		\([0, \infty)\)
per_object_feature_penalties	untyped	-		-
model_shrink_rate	numeric	-		\((-\infty, \infty)\)
model_shrink_mode	character	-	Constant, Decreasing	-
target_border	numeric	-		\((-\infty, \infty)\)
border_count	integer	-		\([1, 65535]\)
feature_border_type	character	GreedyLogSum	Median, Uniform, UniformAndQuantiles, MaxLogSum, MinEntropy, GreedyLogSum	-
per_float_feature_quantization	untyped	-		-
classes_count	integer	-		\([1, \infty)\)
thread_count	integer	1		\([-1, \infty)\)
task_type	character	CPU	CPU, GPU	-
devices	untyped	-		-
logging_level	character	Silent	Silent, Verbose, Info, Debug	-
metric_period	integer	1		\([1, \infty)\)
train_dir	untyped	catboost_info		-
model_size_reg	numeric	0.5		\([0, 1]\)
allow_writing_files	logical	FALSE	TRUE, FALSE	-
save_snapshot	logical	FALSE	TRUE, FALSE	-
snapshot_file	untyped	-		-
snapshot_interval	integer	600		\([1, \infty)\)
simple_ctr	untyped	-		-
combinations_ctr	untyped	-		-
ctr_target_border_count	integer	-		\([1, 255]\)
counter_calc_method	character	Full	SkipTest, Full	-
max_ctr_complexity	integer	-		\([1, \infty)\)
ctr_leaf_count_limit	integer	-		\([1, \infty)\)
store_all_simple_ctr	logical	FALSE	TRUE, FALSE	-
final_ctr_computation_mode	character	Default	Default, Skip	-
verbose	logical	FALSE	TRUE, FALSE	-
ntree_start	integer	0		\([0, \infty)\)
ntree_end	integer	0		\([0, \infty)\)

Installation

The easiest way to install catboost is with the helper function install_catboost.

Custom mlr3 defaults

• logging_level:

  • Actual default: "Verbose"

  • Adjusted default: "Silent"

  • Reason for change: consistent with other mlr3 learners

• thread_count:

  • Actual default: -1

  • Adjusted default: 1

  • Reason for change: consistent with other mlr3 learners

• allow_writing_files:

  • Actual default: TRUE

  • Adjusted default: FALSE

  • Reason for change: consistent with other mlr3 learners

• save_snapshot:

  • Actual default: TRUE

  • Adjusted default: FALSE

  • Reason for change: consistent with other mlr3 learners

References

Dorogush, Veronika A, Ershov, Vasily, Gulin, Andrey (2018). “CatBoost: gradient boosting with categorical features support.” arXiv preprint arXiv:1810.11363.

See also

• Dictionary of Learners: mlr3::mlr_learners.

• as.data.table(mlr_learners) for a table of available Learners in the running session (depending on the loaded packages).

• Chapter in the mlr3book: https://mlr3book.mlr-org.com/basics.html#learners

• mlr3learners for a selection of recommended learners.

• mlr3cluster for unsupervised clustering learners.

• mlr3pipelines to combine learners with pre- and postprocessing steps.

• mlr3tuning for tuning of hyperparameters, mlr3tuningspaces for established default tuning spaces.

Author

sumny

Super classes

mlr3::Learner -> mlr3::LearnerClassif -> LearnerClassifCatboost

Methods

Public methods

• LearnerClassifCatboost$new()

• LearnerClassifCatboost$importance()

• LearnerClassifCatboost$clone()

Inherited methods

• mlr3::Learner$base_learner()
• mlr3::Learner$format()
• mlr3::Learner$help()
• mlr3::Learner$predict()
• mlr3::Learner$predict_newdata()
• mlr3::Learner$print()
• mlr3::Learner$reset()
• mlr3::Learner$train()

---

Method new()

Create a LearnerClassifCatboost object.

Usage

LearnerClassifCatboost$new()

---

Method importance()

The importance scores are calculated using catboost.get_feature_importance, setting type = "FeatureImportance", returned for 'all'.

Usage

LearnerClassifCatboost$importance()

Returns

Named numeric().

---

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerClassifCatboost$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

learner = mlr3::lrn("classif.catboost")
#> Warning: Package 'catboost' required but not installed for Learner 'classif.catboost'
print(learner)
#> <LearnerClassifCatboost:classif.catboost>: Gradient Boosting
#> * Model: -
#> * Parameters: loss_function_twoclass=Logloss,
#>   loss_function_multiclass=MultiClass, logging_level=Silent,
#>   thread_count=1, allow_writing_files=FALSE, save_snapshot=FALSE
#> * Packages: mlr3, mlr3extralearners, catboost
#> * Predict Types:  [response], prob
#> * Feature Types: numeric, factor, ordered
#> * Properties: importance, missings, multiclass, twoclass, weights

# available parameters:
learner$param_set$ids()
#>  [1] "loss_function_twoclass"         "loss_function_multiclass"      
#>  [3] "iterations"                     "learning_rate"                 
#>  [5] "random_seed"                    "l2_leaf_reg"                   
#>  [7] "bootstrap_type"                 "bagging_temperature"           
#>  [9] "subsample"                      "sampling_frequency"            
#> [11] "sampling_unit"                  "mvs_reg"                       
#> [13] "random_strength"                "depth"                         
#> [15] "grow_policy"                    "min_data_in_leaf"              
#> [17] "max_leaves"                     "ignored_features"              
#> [19] "one_hot_max_size"               "has_time"                      
#> [21] "rsm"                            "nan_mode"                      
#> [23] "fold_permutation_block"         "leaf_estimation_method"        
#> [25] "leaf_estimation_iterations"     "leaf_estimation_backtracking"  
#> [27] "fold_len_multiplier"            "approx_on_full_history"        
#> [29] "class_weights"                  "auto_class_weights"            
#> [31] "boosting_type"                  "boost_from_average"            
#> [33] "langevin"                       "diffusion_temperature"         
#> [35] "score_function"                 "monotone_constraints"          
#> [37] "feature_weights"                "first_feature_use_penalties"   
#> [39] "penalties_coefficient"          "per_object_feature_penalties"  
#> [41] "model_shrink_rate"              "model_shrink_mode"             
#> [43] "target_border"                  "border_count"                  
#> [45] "feature_border_type"            "per_float_feature_quantization"
#> [47] "classes_count"                  "thread_count"                  
#> [49] "task_type"                      "devices"                       
#> [51] "logging_level"                  "metric_period"                 
#> [53] "train_dir"                      "model_size_reg"                
#> [55] "allow_writing_files"            "save_snapshot"                 
#> [57] "snapshot_file"                  "snapshot_interval"             
#> [59] "simple_ctr"                     "combinations_ctr"              
#> [61] "ctr_target_border_count"        "counter_calc_method"           
#> [63] "max_ctr_complexity"             "ctr_leaf_count_limit"          
#> [65] "store_all_simple_ctr"           "final_ctr_computation_mode"    
#> [67] "verbose"                        "ntree_start"                   
#> [69] "ntree_end"