Scikit-learn Learners¶
LearnM8 provides five scikit-learn based learners optimized for molecular property prediction. These models offer proven performance, computational efficiency, and robust handling of different dataset characteristics.
RandomForestLearner¶
Overview¶
Random Forest uses an ensemble of decision trees trained on bootstrapped samples of the data. Each tree votes on the prediction, with the average serving as the final output. This learner provides a fast, robust baseline with good performance across diverse molecular datasets.
When to use:
- Fast prototyping and baseline establishment
- Datasets of any size (particularly effective for 100-10,000 compounds)
- As a component in ensemble models
Key characteristics:
- No uncertainty quantification (base implementation)
- Parallel training across all CPU cores
- Out-of-bag (OOB) scoring for model validation
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
n_estimators |
int | 100 | Number of trees in the forest. More trees generally improve performance but increase training time linearly. 100 provides good balance. |
max_depth |
int | None | Maximum tree depth. None allows unlimited depth. Limiting depth (e.g., 10-20) prevents overfitting on small datasets. |
min_samples_split |
int | 2 | Minimum samples required to split a node. Higher values (5-10) prevent overfitting. |
min_samples_leaf |
int | 1 | Minimum samples required at leaf nodes. Higher values (2-5) create smoother decision boundaries. |
max_features |
str | 'sqrt' | Features considered per split. 'sqrt' uses sqrt(n_features), 'log2' is more conservative, None uses all features. |
random_state |
int | 42 | Random seed for reproducibility. |
n_jobs |
int | -1 | Parallel jobs. -1 uses all CPU cores for 5-10x training speedup. |
Performance notes:
- Training time scales linearly with
n_estimators - Memory scales with
n_estimators× tree size - Prediction is fast (logarithmic in tree depth)
- OOB scoring adds negligible overhead
Uncertainty Support¶
No - Base RandomForestLearner does not provide uncertainty estimates. Use ensemble variants for uncertainty quantification.
Performance Characteristics¶
Speed: Fast (trains in seconds for typical molecular datasets)
Scalability:
- Excellent for 100-100,000 compounds
- Linear scaling with dataset size
- Parallel training utilizes all CPU cores
Memory: Moderate (trees stored in memory, ~1-10 MB per 100 trees)
Best for: Quick prototyping, baseline models
Example (CLI)¶
Example (API)¶
from learnm8 import run_active_learning
from learnm8.learners.sklearn import RandomForestLearner
learner = RandomForestLearner(
n_estimators=200,
max_depth=15,
min_samples_split=5,
random_state=42
)
results = run_active_learning(
compound_pool='compounds.csv',
oracle='oracle.csv',
learner=learner,
target_col='Activity',
featurizer='morgan',
n_cycles=10
)
oob_score = learner.get_oob_score()
GaussianProcessLearner¶
Overview¶
Gaussian Process (GP) regression provides the gold standard for uncertainty quantification in active learning. It models predictions as a Gaussian distribution, naturally providing both mean predictions and standard deviation estimates. GP excels on small to medium datasets where principled uncertainty is crucial.
When to use:
- Small to medium datasets (<5,000 compounds recommended)
- Uncertainty-based acquisition strategies (UCB, EI, Thompson sampling)
- When accurate uncertainty quantification is critical
- Benchmark studies requiring rigorous uncertainty estimates
Key characteristics:
- Native, principled uncertainty quantification
- Scales cubically with dataset size (O(n³))
- Hyperparameter optimization via marginal likelihood
- Smooth, continuous prediction surfaces
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
kernel |
Kernel | None | GP kernel function. None uses RBF with automatic hyperparameter learning: C(1.0) * RBF(1.0). Custom kernels supported. |
alpha |
float | 1e-10 | Noise regularization parameter. Larger values (1e-6 to 1e-3) add stability for noisy data. Smaller values for low-noise targets. |
n_restarts_optimizer |
int | 5 | Optimizer restarts for hyperparameter optimization. More restarts (10-20) improve convergence but increase training time. |
normalize_y |
bool | True | Normalize target values. Highly recommended for GP stability and performance. |
random_state |
int | 42 | Random seed for reproducibility. |
Performance notes:
- Training time: O(n³) - becomes slow for >5,000 compounds
- Prediction time: O(n²) - scales quadratically with training set size
- Memory: O(n²) - stores full covariance matrix
- Hyperparameter optimization adds 5-10x overhead but critical for performance
Uncertainty Support¶
Yes - GaussianProcessLearner provides principled uncertainty estimates through posterior standard deviation. This is the most theoretically sound uncertainty available in LearnM8.
How it works:
- GP models predictions as Gaussian distributions
- Returns mean (prediction) and standard deviation (uncertainty)
- Uncertainty increases in unexplored regions
- Uncertainty decreases near training data
Performance Characteristics¶
Speed: Slow for large datasets (cubic scaling)
Scalability:
- Excellent for <1,000 compounds
- Acceptable for 1,000-5,000 compounds
- Not recommended for >5,000 compounds
Memory: High (stores n×n covariance matrix)
Best for: Small datasets requiring principled uncertainty, uncertainty-based acquisition strategies
Example (CLI)¶
learnm8 run compounds.csv --target Activity --learner gp --featurizer descriptors --cycles "random:0.02 ucb:0.01*8"
Example (API)¶
from learnm8 import run_active_learning
from learnm8.learners.sklearn import GaussianProcessLearner
from sklearn.gaussian_process.kernels import Matern, ConstantKernel as C
learner = GaussianProcessLearner(
kernel=C(1.0) * Matern(length_scale=1.0, nu=2.5),
alpha=1e-6,
n_restarts_optimizer=10,
normalize_y=True,
random_state=42
)
results = run_active_learning(
compound_pool='compounds.csv',
oracle='oracle.csv',
learner=learner,
target_col='Activity',
featurizer='descriptors',
cycles=[('random', 0.02), ('ucb', 0.01)],
acquisition_params={'beta': 2.0}
)
XGBoostLearner¶
Overview¶
XGBoost provides high-performance gradient boosting optimized for speed and accuracy. It builds an ensemble of decision trees sequentially, with each tree correcting errors from previous trees. XGBoost excels on medium to large molecular datasets where prediction quality and computational efficiency are both priorities.
When to use:
- Medium to large datasets (1,000-100,000+ compounds)
- When prediction accuracy is the top priority
- Production systems requiring fast predictions
- Datasets with complex non-linear relationships
Key characteristics:
- State-of-the-art tabular data performance
- Efficient parallel training
- Regularization prevents overfitting
- No native uncertainty quantification (use XGBEnsemble for uncertainty)
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
n_estimators |
int | 100 | Number of boosting rounds. More rounds improve performance but risk overfitting. 100-300 typical. |
learning_rate |
float | 0.1 | Step size shrinkage. Lower values (0.01-0.05) require more estimators but often improve generalization. |
max_depth |
int | 6 | Maximum tree depth. Controls model complexity. 3-10 typical for molecular data. |
min_child_weight |
int | 1 | Minimum sum of instance weight in child. Higher values (5-10) prevent overfitting on small datasets. |
subsample |
float | 0.8 | Row subsampling fraction. 0.6-0.9 adds regularization through randomness. |
colsample_bytree |
float | 0.8 | Column subsampling per tree. 0.6-0.9 prevents feature dominance. |
reg_alpha |
float | 0.0 | L1 regularization. Increase (0.1-1.0) for sparse feature importance. |
reg_lambda |
float | 1.0 | L2 regularization. Increase (1.0-10.0) for smoother predictions. |
random_state |
int | 42 | Random seed for reproducibility. |
n_jobs |
int | -1 | Parallel jobs. -1 uses all CPU cores for fast training. |
Performance notes:
- Training time scales with
n_estimators×max_depth - Histogram-based algorithm (
tree_method='hist') for efficiency - Memory efficient compared to Random Forest
Uncertainty Support¶
No - Base XGBoostLearner does not provide uncertainty estimates. Use XGBEnsemble for uncertainty quantification through model disagreement.
Performance Characteristics¶
Speed: Very fast (optimized C++ implementation)
Scalability:
- Excellent for 1,000-100,000+ compounds
- Sub-linear scaling with efficient algorithms
- Parallel training utilizes all CPU cores
Memory: Low (gradient statistics stored efficiently)
Best for: Large datasets, production systems, maximum prediction accuracy
Example (CLI)¶
Example (API)¶
from learnm8 import run_active_learning
from learnm8.learners.sklearn import XGBoostLearner
learner = XGBoostLearner(
n_estimators=200,
learning_rate=0.05,
max_depth=8,
min_child_weight=3,
subsample=0.7,
colsample_bytree=0.7,
reg_alpha=0.1,
reg_lambda=2.0,
random_state=42
)
results = run_active_learning(
compound_pool='compounds.csv',
oracle='oracle.csv',
learner=learner,
target_col='Activity',
featurizer='morgan',
n_cycles=15
)
DecisionTreeLearner¶
Overview¶
Decision Tree provides a single interpretable tree structure for predictions. Unlike ensemble methods, a single tree offers complete transparency in decision-making, making it valuable for understanding model logic and identifying key features. However, single trees are prone to overfitting and generally provide lower accuracy than ensemble methods.
When to use:
- Model interpretability is paramount
- Understanding decision logic for domain insights
- Debugging feature extraction or data quality issues
- Teaching active learning concepts with simple models
Key characteristics:
- Fully interpretable decision paths
- Fast training and prediction
- Prone to overfitting without regularization
- No uncertainty quantification
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
max_depth |
int | 10 | Maximum tree depth. Limits model complexity. 5-15 typical for molecular data. Lower prevents overfitting. |
min_samples_split |
int | 10 | Minimum samples required to split. Higher values (10-20) create simpler trees. |
min_samples_leaf |
int | 5 | Minimum samples at leaf nodes. Higher values (5-10) prevent overfitting. |
max_features |
str | None | Features considered per split. None uses all features. 'sqrt' or 'log2' add regularization. |
random_state |
int | 42 | Random seed for reproducibility. |
Performance notes:
- Defaults designed to prevent overfitting (limited depth, higher min_samples)
- Training time logarithmic in dataset size
- Prediction extremely fast (single path through tree)
- Tree structure can be exported and visualized
Uncertainty Support¶
No - DecisionTreeLearner does not provide uncertainty estimates. Each leaf node has a single prediction value.
Performance Characteristics¶
Speed: Very fast (fastest learner in LearnM8)
Scalability:
- Excellent for any dataset size
- Logarithmic prediction time
- Minimal memory footprint
Memory: Very low (single tree structure)
Best for: Model interpretation, debugging, educational purposes
Example (CLI)¶
Example (API)¶
from learnm8 import run_active_learning
from learnm8.learners.sklearn import DecisionTreeLearner
learner = DecisionTreeLearner(
max_depth=15,
min_samples_split=20,
min_samples_leaf=10,
max_features='sqrt',
random_state=42
)
results = run_active_learning(
compound_pool='compounds.csv',
oracle='oracle.csv',
learner=learner,
target_col='Activity',
featurizer='morgan',
n_cycles=10
)
LinearRegressionLearner¶
Overview¶
Linear Regression models predictions as a linear combination of features. This learner provides the simplest baseline and works well when molecular properties have approximately linear relationships with features. Supports both standard linear regression and Ridge regression (L2 regularization) for handling feature collinearity.
When to use:
- Simple baseline establishment
- Linear relationships between features and targets
- Small datasets where complex models overfit
- Fast predictions required in production
Key characteristics:
- Analytical solution (no iterative optimization)
- Interpretable coefficients
- Very fast training and prediction
- Ridge variant handles collinear features
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
alpha |
float | None | Regularization strength. None uses standard LinearRegression. Values like 0.1-10.0 enable Ridge regression with L2 regularization. |
fit_intercept |
bool | True | Whether to fit intercept term. Usually True unless features are centered. |
n_jobs |
int | -1 | Parallel jobs for LinearRegression only. -1 uses all CPU cores. Ridge doesn't parallelize. |
random_state |
int | 42 | Random seed for Ridge regression reproducibility. |
Performance notes:
- LinearRegression: Parallel computation via BLAS/LAPACK
- Ridge: Sequential but still very fast
- Training time: O(n² × d) for n samples, d features
- Analytical solution means no hyperparameter tuning needed
Uncertainty Support¶
No - LinearRegressionLearner does not provide uncertainty estimates. Use LREnsemble for uncertainty through model disagreement.
Performance Characteristics¶
Speed: Very fast (analytical solution)
Scalability:
- Excellent for any dataset size
- Quadratic scaling with features
- Parallel computation for LinearRegression
Memory: Very low (stores only coefficients)
Best for: Simple baselines, linear relationships, fast predictions
Example (CLI)¶
Example (API)¶
from learnm8 import run_active_learning
from learnm8.learners.sklearn import LinearRegressionLearner
learner = LinearRegressionLearner(
alpha=1.0,
fit_intercept=True,
random_state=42
)
results = run_active_learning(
compound_pool='compounds.csv',
oracle='oracle.csv',
learner=learner,
target_col='Activity',
featurizer='descriptors',
n_cycles=10
)
Learner Comparison¶
Performance Summary¶
| Learner | Speed | Scalability | Uncertainty | Memory | Best Use Case |
|---|---|---|---|---|---|
| RandomForest | Fast | 100-100k | No | Moderate | Fast baseline |
| GaussianProcess | Slow | <5k | Yes (best) | High | Small datasets, principled uncertainty |
| XGBoost | Very fast | 1k-100k+ | No | Low | Large datasets, max accuracy |
| DecisionTree | Very fast | Any | No | Very low | Interpretability |
| LinearRegression | Very fast | Any | No | Very low | Simple baseline |
Featurizer Recommendations¶
Best featurizer by learner type:
| Learner | Recommended Featurizer | Rationale |
|---|---|---|
| RandomForest | morgan or ecfp6 |
Tree methods work well with binary fingerprints |
| GaussianProcess | descriptors |
Continuous features better for kernel methods |
| XGBoost | morgan or ecfp6 |
Tree methods optimized for binary features |
| DecisionTree | morgan |
Simpler fingerprints easier to interpret |
| LinearRegression | descriptors |
Continuous features for linear relationships |
When to Use Each Learner¶
Choose RandomForestLearner when:
- Establishing baselines quickly
- Dataset size is 100-10,000 compounds
- Computational resources limited
Choose GaussianProcessLearner when:
- Dataset size <5,000 compounds
- Uncertainty-based acquisition critical (UCB, EI, Thompson)
- Principled uncertainty quantification required
- Computational time available for cubic scaling
Choose XGBoostLearner when:
- Dataset size >1,000 compounds
- Maximum prediction accuracy required
- Fast training and prediction needed
- Production deployment planned
Choose DecisionTreeLearner when:
- Model interpretability is paramount
- Understanding decision logic required
- Debugging data quality issues
- Educational/demonstration purposes
Choose LinearRegressionLearner when:
- Simple baseline needed
- Linear relationships suspected
- Extremely fast predictions required
- Interpretable coefficients valuable