Classification LightGBM Learner

Gradient boosting algorithm. Calls lightgbm::lightgbm() from lightgbm. The list of parameters can be found here and in the documentation of lightgbm::lgb.train(). Note that lightgbm models have to be saved using lightgbm::lgb.save, so you cannot simpliy save the learner using saveRDS. This will change in future versions of lightgbm.

Dictionary

This Learner can be instantiated via the dictionary mlr_learners or with the associated sugar function lrn():

mlr_learners$get("classif.lightgbm")
lrn("classif.lightgbm")

Meta Information

• Task type: “classif”

• Predict Types: “response”, “prob”

• Feature Types: “logical”, “integer”, “numeric”, “factor”

• Required Packages: mlr3, mlr3extralearners, lightgbm

Parameters

Id	Type	Default	Levels	Range
num_iterations	integer	100		\([0, \infty)\)
objective	character	-	binary, multiclass, multiclassova	-
eval	untyped	-		-
verbose	integer	1		\((-\infty, \infty)\)
record	logical	TRUE	TRUE, FALSE	-
eval_freq	integer	1		\([1, \infty)\)
early_stopping_rounds	integer	-		\([1, \infty)\)
early_stopping_split	numeric	0		\([0, 1]\)
callbacks	untyped	-		-
reset_data	logical	FALSE	TRUE, FALSE	-
categorical_feature	untyped			-
convert_categorical	logical	TRUE	TRUE, FALSE	-
boosting	character	gbdt	gbdt, rf, dart, goss	-
linear_tree	logical	FALSE	TRUE, FALSE	-
learning_rate	numeric	0.1		\([0, \infty)\)
num_leaves	integer	31		\([1, 131072]\)
tree_learner	character	serial	serial, feature, data, voting	-
num_threads	integer	0		\([0, \infty)\)
device_type	character	cpu	cpu, gpu	-
seed	integer	-		\((-\infty, \infty)\)
deterministic	logical	FALSE	TRUE, FALSE	-
force_col_wise	logical	FALSE	TRUE, FALSE	-
force_row_wise	logical	FALSE	TRUE, FALSE	-
histogram_pool_size	numeric	-1		\((-\infty, \infty)\)
max_depth	integer	-1		\((-\infty, \infty)\)
min_data_in_leaf	integer	20		\([0, \infty)\)
min_sum_hessian_in_leaf	numeric	0.001		\([0, \infty)\)
bagging_fraction	numeric	1		\([0, 1]\)
pos_bagging_fraction	numeric	1		\([0, 1]\)
neg_bagging_fraction	numeric	1		\([0, 1]\)
bagging_freq	integer	0		\([0, \infty)\)
bagging_seed	integer	3		\((-\infty, \infty)\)
feature_fraction	numeric	1		\([0, 1]\)
feature_fraction_bynode	numeric	1		\([0, 1]\)
feature_fraction_seed	integer	2		\((-\infty, \infty)\)
extra_trees	logical	FALSE	TRUE, FALSE	-
extra_seed	integer	6		\((-\infty, \infty)\)
first_metric_only	logical	FALSE	TRUE, FALSE	-
max_delta_step	numeric	0		\((-\infty, \infty)\)
lambda_l1	numeric	0		\([0, \infty)\)
lambda_l2	numeric	0		\([0, \infty)\)
linear_lambda	numeric	0		\([0, \infty)\)
min_gain_to_split	numeric	0		\([0, \infty)\)
drop_rate	numeric	0.1		\([0, 1]\)
max_drop	integer	50		\((-\infty, \infty)\)
skip_drop	numeric	0.5		\([0, 1]\)
xgboost_dart_mode	logical	FALSE	TRUE, FALSE	-
uniform_drop	logical	FALSE	TRUE, FALSE	-
drop_seed	integer	4		\((-\infty, \infty)\)
top_rate	numeric	0.2		\([0, 1]\)
other_rate	numeric	0.1		\([0, 1]\)
min_data_per_group	integer	100		\([1, \infty)\)
max_cat_threshold	integer	32		\([1, \infty)\)
cat_l2	numeric	10		\([0, \infty)\)
cat_smooth	numeric	10		\([0, \infty)\)
max_cat_to_onehot	integer	4		\([1, \infty)\)
top_k	integer	20		\([1, \infty)\)
monotone_constraints	untyped			-
monotone_constraints_method	character	basic	basic, intermediate, advanced	-
monotone_penalty	numeric	0		\([0, \infty)\)
feature_contri	untyped			-
forcedsplits_filename	untyped			-
refit_decay_rate	numeric	0.9		\([0, 1]\)
cegb_tradeoff	numeric	1		\([0, \infty)\)
cegb_penalty_split	numeric	0		\([0, \infty)\)
cegb_penalty_feature_lazy	untyped	-		-
cegb_penalty_feature_coupled	untyped	-		-
path_smooth	numeric	0		\([0, \infty)\)
interaction_constraints	untyped	-		-
max_bin	integer	255		\([2, \infty)\)
max_bin_by_feature	untyped			-
min_data_in_bin	integer	3		\([1, \infty)\)
bin_construct_sample_cnt	integer	200000		\([1, \infty)\)
data_random_seed	integer	1		\((-\infty, \infty)\)
is_enable_sparse	logical	TRUE	TRUE, FALSE	-
enable_bundle	logical	TRUE	TRUE, FALSE	-
use_missing	logical	TRUE	TRUE, FALSE	-
zero_as_missing	logical	FALSE	TRUE, FALSE	-
feature_pre_filter	logical	TRUE	TRUE, FALSE	-
pre_partition	logical	FALSE	TRUE, FALSE	-
two_round	logical	FALSE	TRUE, FALSE	-
forcedbins_filename	untyped			-
is_unbalance	logical	FALSE	TRUE, FALSE	-
scale_pos_weight	numeric	1		\([0, \infty)\)
sigmoid	numeric	1		\([0, \infty)\)
boost_from_average	logical	TRUE	TRUE, FALSE	-
eval_at	untyped	:, 1, 5		-
multi_error_top_k	integer	1		\([1, \infty)\)
auc_mu_weights	untyped			-
num_machines	integer	1		\([1, \infty)\)
local_listen_port	integer	12400		\([1, \infty)\)
time_out	integer	120		\([1, \infty)\)
machines	untyped			-
gpu_platform_id	integer	-1		\((-\infty, \infty)\)
gpu_device_id	integer	-1		\((-\infty, \infty)\)
gpu_use_dp	logical	FALSE	TRUE, FALSE	-
num_gpu	integer	1		\([1, \infty)\)
start_iteration_predict	integer	0		\((-\infty, \infty)\)
num_iteration_predict	integer	-1		\((-\infty, \infty)\)
pred_early_stop	logical	FALSE	TRUE, FALSE	-
pred_early_stop_freq	integer	10		\((-\infty, \infty)\)
pred_early_stop_margin	numeric	10		\((-\infty, \infty)\)

Initial parameter values

• convert_categorical: Additional parameter. If this parameter is set to TRUE (default), all factor and logical columns are converted to integers and the parameter categorical_feature of lightgbm is set to those columns.

• early_stopping_split: Additional parameter. Instead of providing the data that is used for early stopping explicitly, the parameter early_stopping_split determines the proportion of the training data that is used for early stopping. Here, stratification on the target variable is used if there is no grouping variable, as one cannot simultaneously stratify and group.

• num_class: This parameter is automatically inferred for multiclass tasks and does not have to be set.

Custom mlr3 defaults

• num_threads:

  • Actual default: 0L

  • Adjusted default: 1L

  • Reason for change: Prevents accidental conflicts with future.

• verbose:

  • Actual default: 1L

  • Adjusted default: -1L

  • Reason for change: Prevents accidental conflicts with mlr messaging system.

• objective: Depending if the task is binary / multiclass, the default is set to "binary" or "multiclasss".

References

Ke, Guolin, Meng, Qi, Finley, Thomas, Wang, Taifeng, Chen, Wei, Ma, Weidong, Ye, Qiwei, Liu, Tie-Yan (2017). “Lightgbm: A highly efficient gradient boosting decision tree.” Advances in neural information processing systems, 30.

See also

• Dictionary of Learners: mlr3::mlr_learners.

• as.data.table(mlr_learners) for a table of available Learners in the running session (depending on the loaded packages).

• Chapter in the mlr3book: https://mlr3book.mlr-org.com/basics.html#learners

• mlr3learners for a selection of recommended learners.

• mlr3cluster for unsupervised clustering learners.

• mlr3pipelines to combine learners with pre- and postprocessing steps.

• mlr3tuning for tuning of hyperparameters, mlr3tuningspaces for established default tuning spaces.

Author

kapsner

Super classes

mlr3::Learner -> mlr3::LearnerClassif -> LearnerClassifLightGBM

Methods

Public methods

• LearnerClassifLightGBM$new()

• LearnerClassifLightGBM$importance()

• LearnerClassifLightGBM$clone()

Inherited methods

• mlr3::Learner$base_learner()
• mlr3::Learner$format()
• mlr3::Learner$help()
• mlr3::Learner$predict()
• mlr3::Learner$predict_newdata()
• mlr3::Learner$print()
• mlr3::Learner$reset()
• mlr3::Learner$train()

---

Method new()

Creates a new instance of this R6 class.

Usage

LearnerClassifLightGBM$new()

---

Method importance()

The importance scores are extracted from lbg.importance.

Usage

LearnerClassifLightGBM$importance()

Returns

Named numeric().

---

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerClassifLightGBM$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

learner = mlr3::lrn("classif.lightgbm")
print(learner)
#> <LearnerClassifLightGBM:classif.lightgbm>: Gradient Boosting
#> * Model: -
#> * Parameters: num_threads=1, verbose=-1, convert_categorical=TRUE
#> * Packages: mlr3, mlr3extralearners, lightgbm
#> * Predict Types:  response, [prob]
#> * Feature Types: logical, integer, numeric, factor
#> * Properties: hotstart_forward, importance, missings, multiclass,
#>   twoclass, weights

# available parameters:
learner$param_set$ids()
#>   [1] "num_iterations"               "objective"                   
#>   [3] "eval"                         "verbose"                     
#>   [5] "record"                       "eval_freq"                   
#>   [7] "early_stopping_rounds"        "early_stopping_split"        
#>   [9] "callbacks"                    "reset_data"                  
#>  [11] "categorical_feature"          "convert_categorical"         
#>  [13] "boosting"                     "linear_tree"                 
#>  [15] "learning_rate"                "num_leaves"                  
#>  [17] "tree_learner"                 "num_threads"                 
#>  [19] "device_type"                  "seed"                        
#>  [21] "deterministic"                "force_col_wise"              
#>  [23] "force_row_wise"               "histogram_pool_size"         
#>  [25] "max_depth"                    "min_data_in_leaf"            
#>  [27] "min_sum_hessian_in_leaf"      "bagging_fraction"            
#>  [29] "pos_bagging_fraction"         "neg_bagging_fraction"        
#>  [31] "bagging_freq"                 "bagging_seed"                
#>  [33] "feature_fraction"             "feature_fraction_bynode"     
#>  [35] "feature_fraction_seed"        "extra_trees"                 
#>  [37] "extra_seed"                   "first_metric_only"           
#>  [39] "max_delta_step"               "lambda_l1"                   
#>  [41] "lambda_l2"                    "linear_lambda"               
#>  [43] "min_gain_to_split"            "drop_rate"                   
#>  [45] "max_drop"                     "skip_drop"                   
#>  [47] "xgboost_dart_mode"            "uniform_drop"                
#>  [49] "drop_seed"                    "top_rate"                    
#>  [51] "other_rate"                   "min_data_per_group"          
#>  [53] "max_cat_threshold"            "cat_l2"                      
#>  [55] "cat_smooth"                   "max_cat_to_onehot"           
#>  [57] "top_k"                        "monotone_constraints"        
#>  [59] "monotone_constraints_method"  "monotone_penalty"            
#>  [61] "feature_contri"               "forcedsplits_filename"       
#>  [63] "refit_decay_rate"             "cegb_tradeoff"               
#>  [65] "cegb_penalty_split"           "cegb_penalty_feature_lazy"   
#>  [67] "cegb_penalty_feature_coupled" "path_smooth"                 
#>  [69] "interaction_constraints"      "max_bin"                     
#>  [71] "max_bin_by_feature"           "min_data_in_bin"             
#>  [73] "bin_construct_sample_cnt"     "data_random_seed"            
#>  [75] "is_enable_sparse"             "enable_bundle"               
#>  [77] "use_missing"                  "zero_as_missing"             
#>  [79] "feature_pre_filter"           "pre_partition"               
#>  [81] "two_round"                    "forcedbins_filename"         
#>  [83] "is_unbalance"                 "scale_pos_weight"            
#>  [85] "sigmoid"                      "boost_from_average"          
#>  [87] "eval_at"                      "multi_error_top_k"           
#>  [89] "auc_mu_weights"               "num_machines"                
#>  [91] "local_listen_port"            "time_out"                    
#>  [93] "machines"                     "gpu_platform_id"             
#>  [95] "gpu_device_id"                "gpu_use_dp"                  
#>  [97] "num_gpu"                      "start_iteration_predict"     
#>  [99] "num_iteration_predict"        "pred_early_stop"             
#> [101] "pred_early_stop_freq"         "pred_early_stop_margin"