Uncertainty-Based Acquisition Strategies¶
Uncertainty-based acquisition strategies leverage model confidence estimates to balance exploration (sampling uncertain regions) with exploitation (sampling predicted high-value compounds). These strategies are particularly powerful when paired with learners that provide well-calibrated uncertainty quantification.
All strategies in this section require learners that support uncertainty estimation (Gaussian Process, ensembles, or MC Dropout).
Upper Confidence Bound (UCB)¶
Overview¶
Upper Confidence Bound balances exploitation and exploration by selecting compounds with high upper confidence bounds. The acquisition score is computed as:
Formula: score = prediction + β × uncertainty
For maximization problems, UCB selects compounds with high predicted values plus a bonus for high uncertainty. The parameter β controls the exploration-exploitation tradeoff.
Exploration/Exploitation Balance¶
- β = 0: Pure exploitation (greedy selection on predictions only)
- β = 1-2: Balanced exploration and exploitation (recommended starting point)
- β > 3: Aggressive exploration (prioritizes uncertain regions)
Higher β values encourage sampling in uncertain regions of chemical space, which can be valuable early in campaigns when building model confidence.
When to Use¶
- Early-stage screening: When you want to explore diverse chemical space
- Model uncertainty: When predictions are unreliable and exploration is needed
- Balanced campaigns: Default choice for general-purpose active learning
- Large compound pools: Scales well to millions of compounds
Avoid UCB when:
- Pure exploitation is desired (use greedy instead)
- Uncertainty estimates are poorly calibrated
- Computational budget is extremely limited (fewer cycles mean less time for exploration)
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
beta |
float | 2.0 | Exploration parameter. Higher values favor exploration over exploitation. |
score_direction |
str | 'higher' | Optimization direction ('higher' or 'lower'). |
Tuning Guidance:
- Start with β = 2.0 for balanced behavior
- Increase to β = 3-5 for more exploration (useful in first few cycles)
- Decrease to β = 1.0 for more exploitation (useful in later cycles)
- Can vary β across cycles using
CycleConfigwith differentacquisition_params
Requires Uncertainty¶
UCB requires uncertainty estimates. Compatible learners:
- GaussianProcessLearner - Best calibrated uncertainty
- Ensemble learners - Uncertainty from model disagreement (MixedEnsemble, RFEnsemble, etc.)
- MCDropoutLearner - Uncertainty from dropout sampling
Examples¶
CLI with default β:
learnm8 run compounds.csv --target Activity \
--learner gp --featurizer morgan \
--cycles "random:0.02 ucb:0.005*10"
CLI with custom β (more exploration):
learnm8 run compounds.csv --target Activity \
--learner ensemble --featurizer morgan \
--cycles "random:0.02 ucb:0.005*5" \
--acquisition-params '{"beta": 3.0}'
Python API with varying β across cycles:
from learnm8 import run_active_learning, CycleConfig
results = run_active_learning(
compound_pool='compounds.csv',
oracle='oracle.csv',
learner='gp',
target_col='Activity',
featurizer='morgan',
cycles=[
CycleConfig('random', batch_fraction=0.02),
CycleConfig('ucb', n_cycles=5, batch_fraction=0.005,
acquisition_params={'beta': 3.0}),
CycleConfig('ucb', n_cycles=5, batch_fraction=0.005,
acquisition_params={'beta': 1.5})
]
)
Expected Improvement (EI)¶
Overview¶
Expected Improvement calculates the expected improvement over the current best observed value from labeled training data. It provides a principled probabilistic framework for balancing exploration and exploitation.
Formula: EI = E[max(0, improvement)]
Where improvement is calculated relative to the best value seen in training data. EI uses the predictive distribution (mean and uncertainty) to compute the probability-weighted expected gain.
Requires: SciPy for normal distribution calculations (pip install scipy)
Exploration/Exploitation Balance¶
- ξ = 0: Exploitation-focused (only sample if improvement is expected)
- ξ = 0.01: Balanced (default, small exploration bonus)
- ξ = 0.1: Exploration-focused (accepts candidates with lower expected improvement)
The ξ parameter adds a small buffer to encourage exploration even when improvement probability is modest.
When to Use¶
- Iterative optimization: When you want to systematically improve beyond current best
- Well-calibrated uncertainty: When learner uncertainty is reliable (GP recommended)
- Sequential screening: Works well with greedy follow-up phases
- Bayesian optimization: Standard choice in Bayesian optimization workflows
Avoid EI when:
- Current best value is unknown or unreliable
- Uncertainty calibration is poor
- Goal is exploration rather than optimization
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
xi |
float | 0.01 | Exploration parameter. Small positive values encourage exploration. |
score_direction |
str | 'higher' | Optimization direction ('higher' or 'lower'). |
current_best |
float | Required | Best observed value from labeled training data. |
Tuning Guidance:
- Keep ξ small (0.001-0.1) for optimization-focused campaigns
- Increase ξ to 0.1-0.5 for more exploration
current_bestis automatically computed from training data in each cycle- Pass via
acquisition_paramsat cycle level:{'current_best': 0.95}
Requires Uncertainty¶
EI requires well-calibrated uncertainty estimates. Compatible learners:
- GaussianProcessLearner - Gold standard for EI (best calibration)
- Ensemble learners - Reasonable uncertainty estimates
- MCDropoutLearner - Requires sufficient dropout samples (100+)
Examples¶
CLI with default ξ:
learnm8 run compounds.csv --target Activity \
--learner gp --featurizer morgan \
--cycles "random:0.02 ei:0.005*10"
Python API with current best:
from learnm8 import run_active_learning
import polars as pl
df = pl.read_csv('compounds.csv')
current_best = df['Activity'].max()
results = run_active_learning(
compound_pool=df,
oracle='oracle.csv',
learner='gp',
target_col='Activity',
featurizer='morgan',
cycles=[
('random', 0.02),
('ei', 0.005)
],
acquisition_params={'xi': 0.01, 'current_best': current_best}
)
Probability of Improvement (PI)¶
Overview¶
Probability of Improvement calculates the probability that a compound will improve over the current best observed value. Unlike EI, PI focuses solely on the probability of improvement rather than the magnitude.
Formula: PI = P(f(x) > current_best + ξ)
PI is simpler and more conservative than EI, selecting compounds with high probability of any improvement.
Requires: SciPy for normal distribution calculations (pip install scipy)
Exploration/Exploitation Balance¶
- ξ = 0: Pure optimization (select if any improvement probability)
- ξ = 0.01: Balanced (default, requires slight improvement)
- ξ = 0.1: Conservative (requires more substantial improvement)
Similar to EI, ξ controls the exploration bonus but with different interpretation.
When to Use¶
- Conservative optimization: When you prefer high-probability improvements
- Risk-averse screening: When measurement costs are high
- Complement to EI: Alternative probabilistic approach
- Simple optimization: When EI's complexity is unnecessary
Avoid PI when:
- Magnitude of improvement matters (use EI instead)
- Goal is broad exploration (use UCB or entropy instead)
- Uncertainty estimates are unreliable
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
xi |
float | 0.01 | Exploration parameter. Controls improvement threshold. |
score_direction |
str | 'higher' | Optimization direction ('higher' or 'lower'). |
current_best |
float | Required | Best observed value from labeled training data. |
Tuning Guidance:
- Keep ξ = 0.01 for balanced probability of improvement
- Increase ξ to 0.1-0.5 for more conservative selection (higher improvement required)
- Decrease ξ to 0.001 for more aggressive selection
- Like EI,
current_bestshould be computed from training data each cycle
Requires Uncertainty¶
PI requires uncertainty estimates. Compatible learners:
- GaussianProcessLearner - Best choice for calibrated probabilities
- Ensemble learners - Reasonable probability estimates
- MCDropoutLearner - Requires sufficient samples
Examples¶
CLI:
learnm8 run compounds.csv --target Activity \
--learner gp --featurizer morgan \
--cycles "random:0.02 pi:0.005*10"
Python API with conservative threshold:
from learnm8 import run_active_learning
results = run_active_learning(
compound_pool='compounds.csv',
oracle='oracle.csv',
learner='gp',
target_col='Activity',
featurizer='morgan',
cycles=[
('random', 0.02),
('pi', 0.005)
],
acquisition_params={'xi': 0.1}
)
Thompson Sampling¶
Overview¶
Thompson Sampling is a stochastic exploration strategy that samples from the posterior predictive distribution and selects compounds based on sampled values. This Bayesian approach naturally balances exploration and exploitation through the posterior uncertainty.
Algorithm:
- For each compound, sample a value from its predictive distribution:
sample ~ N(prediction, uncertainty) - Select compounds with highest (or lowest) sampled values
- Stochasticity ensures exploration in uncertain regions
Thompson Sampling provides probability matching to the optimal strategy in the long run.
Exploration/Exploitation Balance¶
Thompson Sampling automatically balances exploration and exploitation without requiring parameter tuning:
- Compounds with high uncertainty have wider sampling distributions (more exploration)
- Compounds with high predicted values are more likely to be sampled (exploitation)
- Stochasticity provides natural diversity across cycles
The only parameter is random_state for reproducibility, not exploration control.
When to Use¶
- Parameter-free exploration: When you want automatic exploration without tuning
- Stochastic sampling: When diversity across runs is acceptable or desired
- Bayesian workflows: Natural choice in Bayesian optimization contexts
- Long campaigns: Provably optimal in the long run (multi-armed bandit theory)
Avoid Thompson Sampling when:
- Deterministic selection is required
- Short campaigns where stochasticity introduces too much variance
- Interpretability is important (behavior is less intuitive than UCB/EI)
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
random_state |
int | 42 | Random seed for reproducible sampling. |
score_direction |
str | 'higher' | Optimization direction ('higher' or 'lower'). |
Tuning Guidance:
- Set
random_statefor reproducibility in benchmark experiments - Vary
random_stateacross independent runs for exploration diversity - No other tuning required (automatic exploration)
Requires Uncertainty¶
Thompson Sampling requires uncertainty estimates. Compatible learners:
- GaussianProcessLearner - Best calibrated sampling
- Ensemble learners - Samples from model disagreement
- MCDropoutLearner - Samples from dropout distribution
Examples¶
CLI:
learnm8 run compounds.csv --target Activity \
--learner gp --featurizer morgan \
--cycles "random:0.02 thompson:0.005*10" \
--random-state 42
Python API with custom random state:
from learnm8 import run_active_learning
results = run_active_learning(
compound_pool='compounds.csv',
oracle='oracle.csv',
learner='ensemble',
target_col='Activity',
featurizer='morgan',
cycles=[
('random', 0.02),
('thompson', 0.005)
],
acquisition_params={'random_state': 123}
)
Multiple independent runs with different seeds:
from learnm8 import run_active_learning
for seed in [42, 43, 44, 45, 46]:
results = run_active_learning(
compound_pool='compounds.csv',
oracle='oracle.csv',
learner='gp',
target_col='Activity',
featurizer='morgan',
cycles=[('random', 0.02), ('thompson', 0.005)],
acquisition_params={'random_state': seed},
output_dir=f'results/thompson_seed_{seed}'
)
Entropy¶
Overview¶
Entropy-based acquisition selects compounds that maximize information gain by choosing samples with highest predictive entropy (uncertainty). This is a pure exploration strategy focused on learning model parameters rather than finding high-value compounds.
Formula: Entropy = uncertainty or Entropy = variance
Entropy acquisition prioritizes reducing model uncertainty uniformly across chemical space.
Exploration/Exploitation Balance¶
Entropy is a pure exploration strategy with no exploitation component:
- Selects compounds with maximum uncertainty regardless of predicted value
- Useful for building robust models before exploitation
- Often combined with greedy phases in multi-stage cycles
The entropy_type parameter controls whether raw uncertainty (standard deviation) or variance is used.
When to Use¶
- Model building: Early cycles to establish model confidence
- Exploration phase: Before exploitation in multi-stage workflows
- Uncertain landscapes: When predictions are unreliable everywhere
- Diversity complement: Combines well with diversity-based strategies
Avoid Entropy when:
- Goal is optimization rather than exploration
- Later cycles when exploitation is needed
- Uncertainty estimates are poorly calibrated
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
entropy_type |
str | 'uncertainty' | Type of entropy measure ('uncertainty' or 'variance'). |
score_direction |
str | 'higher' | Optimization direction (ignored by entropy, but required for consistency). |
Tuning Guidance:
- Use
entropy_type='uncertainty'(default) for linear uncertainty scoring - Use
entropy_type='variance'to emphasize high-uncertainty compounds more strongly - Variance heavily penalizes low-uncertainty compounds (quadratic relationship)
Requires Uncertainty¶
Entropy requires uncertainty estimates. Compatible learners:
- GaussianProcessLearner - Best calibrated entropy
- Ensemble learners - Entropy from model disagreement
- MCDropoutLearner - Entropy from dropout variability
Examples¶
CLI with uncertainty-based entropy:
learnm8 run compounds.csv --target Activity \
--learner gp --featurizer morgan \
--cycles "random:0.02 entropy:0.01*3 greedy:0.005*7"
CLI with variance-based entropy:
learnm8 run compounds.csv --target Activity \
--learner ensemble --featurizer morgan \
--cycles "random:0.02 entropy:0.01*5" \
--acquisition-params '{"entropy_type": "variance"}'
Python API with exploration-exploitation sequence:
from learnm8 import run_active_learning, CycleConfig
results = run_active_learning(
compound_pool='compounds.csv',
oracle='oracle.csv',
learner='gp',
target_col='Activity',
featurizer='morgan',
cycles=[
CycleConfig('random', batch_fraction=0.02),
CycleConfig('entropy', n_cycles=3, batch_fraction=0.01,
acquisition_params={'entropy_type': 'uncertainty'}),
CycleConfig('greedy', n_cycles=7, batch_fraction=0.005)
]
)
Comparison Table¶
| Strategy | Exploration | Exploitation | Requires | Key Parameter | Best Use Case |
|---|---|---|---|---|---|
| UCB | High (tunable) | High (tunable) | Uncertainty | β (2.0) | General-purpose balanced |
| EI | Medium | High | Uncertainty + current_best | ξ (0.01) | Iterative optimization |
| PI | Low | High | Uncertainty + current_best | ξ (0.01) | Conservative optimization |
| Thompson | Medium | Medium | Uncertainty | random_state | Parameter-free exploration |
| Entropy | Very High | None | Uncertainty | entropy_type | Pure exploration phase |
Performance Considerations¶
All uncertainty-based strategies have similar computational complexity:
- Training time: Dominated by learner (GP is slowest, ensembles 3x cost)
- Selection time: O(n) scoring + O(n log k) selection where n = pool size, k = batch size
- Memory: Minimal additional memory beyond predictions/uncertainties
Optimization tips:
- EI and PI require SciPy (normal distribution CDF/PDF)
- Thompson Sampling uses numpy random sampling (faster than EI/PI)
- UCB is fastest (simple arithmetic)
- All scale well to millions of compounds (vectorized operations)
Combining with Other Strategies¶
Uncertainty-based strategies work well in multi-stage cycles:
Exploration → Exploitation:
Balanced throughout:
Optimization-focused: