Estimation Methods in Population Modeling

Understand how population models are actually estimated, including FO, FOCE, SAEM, and Bayesian approaches.
Tip

What you’ll build today: a clear conceptual understanding of how model parameters are estimated and why different estimation methods exist.

Learning Objectives

By the end of this lesson, you will be able to:

  • Explain what it means to estimate a population model
  • Understand the role of likelihood in estimation
  • Connect likelihood to objective function values
  • Distinguish FO, FOCE, FOCEI, SAEM, and Bayesian methods
  • Recognize when different methods are appropriate

Key Ideas

All estimation methods try to answer the same question:

What parameter values best explain the observed data?

This is formalized using the likelihood:

\[ L(\theta) = P(\text{data} \mid \theta) \]

Insight: Estimation is about finding parameters that make the observed data most plausible.

Likelihood and Log-Likelihood

In practice, we often maximize the log-likelihood:

\[ \log L(\theta)=\log P(\text{data}\mid\theta) \]

because it is:

  • easier numerically
  • more stable for computation
  • useful because products become sums

You do not need to calculate log-likelihood manually.

The key idea is:

Better Parameter Values
↓
Better Agreement with Data
↓
Higher Likelihood

From Likelihood to Objective Function Values

Most pharmacometric software does not report likelihood directly.

Instead, it often reports an objective function value (OFV):

\[ OFV = -2\log L \]

where:

  • \(L\) is the likelihood
  • lower OFV indicates better agreement between the model and the data

Conceptually:

Higher Likelihood

↓

Lower OFV

↓

Better Agreement

Likelihood and OFV contain the same information expressed on different scales.

Note

You do not need to calculate OFV manually.

For now, remember:

higher likelihood = lower OFV

The Challenge

Population models include:

  • random effects (η)
  • nonlinear relationships

This makes the likelihood difficult to compute exactly.

Different methods exist to approximate or solve this problem.

Visualizing Estimation Methods

All methods start from the same basic ingredients.

flowchart TD

DATA["Data + Model"]

-->

LIK["Likelihood"]

LIK -->

OBJ["OFV or Posterior"]

OBJ -->

FO["FO"]

OBJ -->

FOCE["FOCE / FOCEI"]

OBJ -->

SAEM["SAEM"]

OBJ -->

BAYES["Bayesian"]

Different methods use different strategies to work with the same underlying estimation problem.

  • FO / FOCE → simplify likelihood mathematically
  • SAEM → approximate likelihood through simulation
  • Bayesian → combine likelihood with prior information to obtain a posterior distribution

First-Order (FO)

Idea

  • Linearizes the model around typical values
  • Simplifies computation

Strengths

  • Fast
  • Historically important

Limitations

  • Can be inaccurate with strong nonlinearity
  • Poor handling of variability

First-Order Conditional Estimation (FOCE)

Idea

  • Linearizes around individual estimates, not just typical values

Strengths

  • More accurate than FO
  • Better handling of variability

FOCEI (with interaction)

  • Includes interaction between random effects and residual error
  • More realistic modeling

Insight: FOCE improves accuracy by accounting for individual-level behavior.

SAEM (Stochastic Approximation EM)

Idea

  • Uses simulation (Monte Carlo) to approximate likelihood
  • Iteratively improves parameter estimates

Strengths

  • Handles complex nonlinear models well
  • Robust for sparse data

Limitations

  • Computationally heavier

Insight: SAEM avoids linearization by using simulation.

Bayesian Methods

Idea

  • Treat parameters as random variables
  • Combine likelihood with prior information
  • Estimate the full posterior distribution

\[ P(\theta \mid \text{data}) \propto P(\text{data} \mid \theta) \times P(\theta) \]

Strengths

  • Full uncertainty quantification
  • Flexible

Limitations

  • Computationally intensive
  • Requires prior specification

Worked Example: Same Model, Different Estimation

Imagine fitting a sparse PK dataset.

Different methods may behave differently:

Method Behavior
FO Fast but may oversimplify
FOCE Better individual handling
SAEM More robust with sparse data
Bayesian Provides uncertainty distribution

The underlying model is identical.

Only the estimation strategy changes.

This means:

model structure and estimation method are separate decisions.

Worked Comparison

Think of estimation methods as different ways to solve the same estimation problem.

  • FO → simplify first, estimate second
  • FOCE → personalize then estimate
  • SAEM → simulate then estimate
  • Bayesian → estimate distributions rather than single values

Each method trades:

  • speed
  • stability
  • approximation quality
  • uncertainty representation

Insight

A critical idea:

All methods solve the same problem, but with different approximations.

Note

The choice of method affects accuracy, stability, and interpretation.

When Methods Matter

Differences become important when:

  • data are sparse
  • models are nonlinear
  • variability is high

In simple cases:

  • methods may give similar results

In complex cases:

  • method choice can change conclusions

Strategies

  • Match method to model complexity
  • Use robust methods such as FOCE or SAEM for nonlinear models
  • Check convergence diagnostics
  • Compare results across methods when possible

Common Mistakes

  • Treating all estimation methods as equivalent
  • Ignoring convergence warnings
  • Over-relying on default settings
  • Misinterpreting unstable results
  • Thinking lower OFV automatically proves the model is scientifically correct

Practice Problems

  1. What does the likelihood represent?
  2. Why are different estimation methods needed?
  3. How is OFV related to likelihood?
  4. What is the main advantage of SAEM?

  1. Likelihood measures how plausible the observed data are under a set of parameter values.

  2. Different methods are needed because population model likelihoods can be difficult to compute exactly, especially with random effects and nonlinear models.

  3. OFV is related to likelihood by:

\[ OFV = -2\log L \]

Higher likelihood corresponds to lower OFV.

  1. SAEM can handle nonlinear and complex models without relying on the same linearization strategy used by FO and FOCE.

Summary

Estimation methods:

  • attempt to maximize likelihood
  • often work with OFV instead of likelihood directly
  • differ in how they approximate or explore the estimation problem

Method choice changes not only computation, but also what conclusions are trustworthy.

Key methods:

  • FO → simple but limited
  • FOCE → improved accuracy
  • SAEM → robust and flexible
  • Bayesian → full probabilistic framework
  • Estimation = finding parameters that make the data plausible
  • Higher likelihood means lower OFV
  • Methods differ in approximation strategy
  • FOCE and SAEM are commonly preferred
  • Always check convergence
  • Method choice affects results