Regression LightGBM Learner

Gradient boosting algorithm. Calls lightgbm::lightgbm() from lightgbm. The list of parameters can be found here and in the documentation of lightgbm::lgb.train(). Note that lightgbm models have to be saved using lightgbm::lgb.save, so you cannot simpliy save the learner using saveRDS. This will change in future versions of lightgbm.

Dictionary

This Learner can be instantiated via the dictionary mlr_learners or with the associated sugar function lrn():

mlr_learners$get("regr.lightgbm")
lrn("regr.lightgbm")

Meta Information

• Task type: “regr”

• Predict Types: “response”

• Feature Types: “logical”, “integer”, “numeric”, “factor”

• Required Packages: mlr3, mlr3extralearners, lightgbm

Parameters

Id	Type	Default	Levels	Range
num_iterations	integer	100		\([0, \infty)\)
objective	character	regression	regression, regression_l1, huber, fair, poisson, quantile, mape, gamma, tweedie	-
eval	untyped	-		-
verbose	integer	1		\((-\infty, \infty)\)
record	logical	TRUE	TRUE, FALSE	-
eval_freq	integer	1		\([1, \infty)\)
early_stopping_rounds	integer	-		\([1, \infty)\)
early_stopping	logical	FALSE	TRUE, FALSE	-
callbacks	untyped	-		-
reset_data	logical	FALSE	TRUE, FALSE	-
categorical_feature	untyped			-
convert_categorical	logical	TRUE	TRUE, FALSE	-
boosting	character	gbdt	gbdt, rf, dart, goss	-
linear_tree	logical	FALSE	TRUE, FALSE	-
learning_rate	numeric	0.1		\([0, \infty)\)
num_leaves	integer	31		\([1, 131072]\)
tree_learner	character	serial	serial, feature, data, voting	-
num_threads	integer	0		\([0, \infty)\)
device_type	character	cpu	cpu, gpu	-
seed	integer	-		\((-\infty, \infty)\)
deterministic	logical	FALSE	TRUE, FALSE	-
data_sample_strategy	character	bagging	bagging, goss	-
force_col_wise	logical	FALSE	TRUE, FALSE	-
force_row_wise	logical	FALSE	TRUE, FALSE	-
histogram_pool_size	integer	-1		\((-\infty, \infty)\)
max_depth	integer	-1		\((-\infty, \infty)\)
min_data_in_leaf	integer	20		\([0, \infty)\)
min_sum_hessian_in_leaf	numeric	0.001		\([0, \infty)\)
bagging_fraction	numeric	1		\([0, 1]\)
bagging_freq	integer	0		\([0, \infty)\)
bagging_seed	integer	3		\((-\infty, \infty)\)
feature_fraction	numeric	1		\([0, 1]\)
feature_fraction_bynode	numeric	1		\([0, 1]\)
feature_fraction_seed	integer	2		\((-\infty, \infty)\)
extra_trees	logical	FALSE	TRUE, FALSE	-
extra_seed	integer	6		\((-\infty, \infty)\)
first_metric_only	logical	FALSE	TRUE, FALSE	-
max_delta_step	numeric	0		\((-\infty, \infty)\)
lambda_l1	numeric	0		\([0, \infty)\)
lambda_l2	numeric	0		\([0, \infty)\)
linear_lambda	numeric	0		\([0, \infty)\)
min_gain_to_split	numeric	0		\([0, \infty)\)
drop_rate	numeric	0.1		\([0, 1]\)
max_drop	integer	50		\((-\infty, \infty)\)
skip_drop	numeric	0.5		\([0, 1]\)
xgboost_dart_mode	logical	FALSE	TRUE, FALSE	-
uniform_drop	logical	FALSE	TRUE, FALSE	-
drop_seed	integer	4		\((-\infty, \infty)\)
top_rate	numeric	0.2		\([0, 1]\)
other_rate	numeric	0.1		\([0, 1]\)
min_data_per_group	integer	100		\([1, \infty)\)
max_cat_threshold	integer	32		\([1, \infty)\)
cat_l2	numeric	10		\([0, \infty)\)
cat_smooth	numeric	10		\([0, \infty)\)
max_cat_to_onehot	integer	4		\([1, \infty)\)
top_k	integer	20		\([1, \infty)\)
monotone_constraints	untyped			-
monotone_constraints_method	character	basic	basic, intermediate, advanced	-
monotone_penalty	numeric	0		\([0, \infty)\)
feature_contri	untyped			-
forcedsplits_filename	untyped			-
refit_decay_rate	numeric	0.9		\([0, 1]\)
cegb_tradeoff	numeric	1		\([0, \infty)\)
cegb_penalty_split	numeric	0		\([0, \infty)\)
cegb_penalty_feature_lazy	untyped	-		-
cegb_penalty_feature_coupled	untyped	-		-
path_smooth	numeric	0		\([0, \infty)\)
interaction_constraints	untyped	-		-
max_bin	integer	255		\([2, \infty)\)
max_bin_by_feature	untyped			-
min_data_in_bin	integer	3		\([1, \infty)\)
bin_construct_sample_cnt	integer	200000		\([1, \infty)\)
data_random_seed	integer	1		\((-\infty, \infty)\)
is_enable_sparse	logical	TRUE	TRUE, FALSE	-
enable_bundle	logical	TRUE	TRUE, FALSE	-
use_missing	logical	TRUE	TRUE, FALSE	-
zero_as_missing	logical	FALSE	TRUE, FALSE	-
feature_pre_filter	logical	TRUE	TRUE, FALSE	-
pre_partition	logical	FALSE	TRUE, FALSE	-
two_round	logical	FALSE	TRUE, FALSE	-
forcedbins_filename	untyped			-
boost_from_average	logical	TRUE	TRUE, FALSE	-
reg_sqrt	logical	FALSE	TRUE, FALSE	-
alpha	numeric	0.9		\([0, \infty)\)
fair_c	numeric	1		\([0, \infty)\)
poisson_max_delta_step	numeric	0.7		\([0, \infty)\)
tweedie_variance_power	numeric	1.5		\([1, 2]\)
metric_freq	integer	1		\([1, \infty)\)
num_machines	integer	1		\([1, \infty)\)
local_listen_port	integer	12400		\([1, \infty)\)
time_out	integer	120		\([1, \infty)\)
machines	untyped			-
gpu_platform_id	integer	-1		\((-\infty, \infty)\)
gpu_device_id	integer	-1		\((-\infty, \infty)\)
gpu_use_dp	logical	FALSE	TRUE, FALSE	-
num_gpu	integer	1		\([1, \infty)\)
start_iteration_predict	integer	0		\((-\infty, \infty)\)
num_iteration_predict	integer	-1		\((-\infty, \infty)\)
pred_early_stop	logical	FALSE	TRUE, FALSE	-
pred_early_stop_freq	integer	10		\((-\infty, \infty)\)
pred_early_stop_margin	numeric	10		\((-\infty, \infty)\)

Initial parameter values

• convert_categorical: Additional parameter. If this parameter is set to TRUE (default), all factor and logical columns are converted to integers and the parameter categorical_feature of lightgbm is set to those columns.

Custom mlr3 defaults

• num_threads:

  • Actual default: 0L

  • Adjusted default: 1L

  • Reason for change: Prevents accidental conflicts with future.

• verbose:

  • Actual default: 1L

  • Adjusted default: -1L

  • Reason for change: Prevents accidental conflicts with mlr messaging system.

Custom mlr3 parameters

• early_stopping Whether to use the test set for early stopping. Default is FALSE.

References

Ke, Guolin, Meng, Qi, Finley, Thomas, Wang, Taifeng, Chen, Wei, Ma, Weidong, Ye, Qiwei, Liu, Tie-Yan (2017). “Lightgbm: A highly efficient gradient boosting decision tree.” Advances in neural information processing systems, 30.

See also

• Dictionary of Learners: mlr3::mlr_learners.

• as.data.table(mlr_learners) for a table of available Learners in the running session (depending on the loaded packages).

• Chapter in the mlr3book: https://mlr3book.mlr-org.com/basics.html#learners

• mlr3learners for a selection of recommended learners.

• mlr3cluster for unsupervised clustering learners.

• mlr3pipelines to combine learners with pre- and postprocessing steps.

• mlr3tuning for tuning of hyperparameters, mlr3tuningspaces for established default tuning spaces.

Author

kapsner

Super classes

mlr3::Learner -> mlr3::LearnerRegr -> LearnerRegrLightGBM

Methods

Public methods

• LearnerRegrLightGBM$new()

• LearnerRegrLightGBM$importance()

• LearnerRegrLightGBM$clone()

Inherited methods

• mlr3::Learner$base_learner()
• mlr3::Learner$format()
• mlr3::Learner$help()
• mlr3::Learner$predict()
• mlr3::Learner$predict_newdata()
• mlr3::Learner$print()
• mlr3::Learner$reset()
• mlr3::Learner$train()

---

Method new()

Creates a new instance of this R6 class.

Usage

LearnerRegrLightGBM$new()

---

Method importance()

The importance scores are extracted from lbg.importance.

Usage

LearnerRegrLightGBM$importance()

Returns

Named numeric().

---

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerRegrLightGBM$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

learner = mlr3::lrn("regr.lightgbm")
print(learner)
#> <LearnerRegrLightGBM:regr.lightgbm>: Gradient Boosting
#> * Model: -
#> * Parameters: num_threads=1, verbose=-1, objective=regression,
#>   convert_categorical=TRUE
#> * Packages: mlr3, mlr3extralearners, lightgbm
#> * Predict Types:  [response]
#> * Feature Types: logical, integer, numeric, factor
#> * Properties: hotstart_forward, importance, missings, weights

# available parameters:
learner$param_set$ids()
#>   [1] "num_iterations"               "objective"                   
#>   [3] "eval"                         "verbose"                     
#>   [5] "record"                       "eval_freq"                   
#>   [7] "early_stopping_rounds"        "early_stopping"              
#>   [9] "callbacks"                    "reset_data"                  
#>  [11] "categorical_feature"          "convert_categorical"         
#>  [13] "boosting"                     "linear_tree"                 
#>  [15] "learning_rate"                "num_leaves"                  
#>  [17] "tree_learner"                 "num_threads"                 
#>  [19] "device_type"                  "seed"                        
#>  [21] "deterministic"                "data_sample_strategy"        
#>  [23] "force_col_wise"               "force_row_wise"              
#>  [25] "histogram_pool_size"          "max_depth"                   
#>  [27] "min_data_in_leaf"             "min_sum_hessian_in_leaf"     
#>  [29] "bagging_fraction"             "bagging_freq"                
#>  [31] "bagging_seed"                 "feature_fraction"            
#>  [33] "feature_fraction_bynode"      "feature_fraction_seed"       
#>  [35] "extra_trees"                  "extra_seed"                  
#>  [37] "first_metric_only"            "max_delta_step"              
#>  [39] "lambda_l1"                    "lambda_l2"                   
#>  [41] "linear_lambda"                "min_gain_to_split"           
#>  [43] "drop_rate"                    "max_drop"                    
#>  [45] "skip_drop"                    "xgboost_dart_mode"           
#>  [47] "uniform_drop"                 "drop_seed"                   
#>  [49] "top_rate"                     "other_rate"                  
#>  [51] "min_data_per_group"           "max_cat_threshold"           
#>  [53] "cat_l2"                       "cat_smooth"                  
#>  [55] "max_cat_to_onehot"            "top_k"                       
#>  [57] "monotone_constraints"         "monotone_constraints_method" 
#>  [59] "monotone_penalty"             "feature_contri"              
#>  [61] "forcedsplits_filename"        "refit_decay_rate"            
#>  [63] "cegb_tradeoff"                "cegb_penalty_split"          
#>  [65] "cegb_penalty_feature_lazy"    "cegb_penalty_feature_coupled"
#>  [67] "path_smooth"                  "interaction_constraints"     
#>  [69] "max_bin"                      "max_bin_by_feature"          
#>  [71] "min_data_in_bin"              "bin_construct_sample_cnt"    
#>  [73] "data_random_seed"             "is_enable_sparse"            
#>  [75] "enable_bundle"                "use_missing"                 
#>  [77] "zero_as_missing"              "feature_pre_filter"          
#>  [79] "pre_partition"                "two_round"                   
#>  [81] "forcedbins_filename"          "boost_from_average"          
#>  [83] "reg_sqrt"                     "alpha"                       
#>  [85] "fair_c"                       "poisson_max_delta_step"      
#>  [87] "tweedie_variance_power"       "metric_freq"                 
#>  [89] "num_machines"                 "local_listen_port"           
#>  [91] "time_out"                     "machines"                    
#>  [93] "gpu_platform_id"              "gpu_device_id"               
#>  [95] "gpu_use_dp"                   "num_gpu"                     
#>  [97] "start_iteration_predict"      "num_iteration_predict"       
#>  [99] "pred_early_stop"              "pred_early_stop_freq"        
#> [101] "pred_early_stop_margin"