Algorithms¶

Detailed documentation for each optimisation and sampling algorithm in Diffid.

Optimisers¶

Gradient-free and gradient-based algorithms for finding optimal parameters.

Samplers¶

MCMC and nested sampling for uncertainty quantification and model comparison.

Algorithm Comparison¶

Choosing an Algorithm¶

graph TD
    A[Start] --> B{Need uncertainty?}
    B -->|No| C{Gradients available?}
    B -->|Yes| D{Need evidence?}
    C -->|Yes| E[Adam]
    C -->|No| F{Problem size?}
    D -->|Yes| G[Nested Sampling]
    D -->|No| H[Metropolis-Hastings]
    F -->|< 10 params| I[Nelder-Mead]
    F -->|> 10 params| J[CMA-ES]

Performance Characteristics¶

Convergence Speed¶

Fast → Slow:

1. Adam (with gradients)
2. Nelder-Mead (small problems)
3. CMA-ES (large problems)
4. MCMC samplers (many evaluations)
5. Nested sampling (most evaluations)

Robustness to Local Minima¶

Least → Most robust:

1. Adam (gradient descent)
2. Nelder-Mead (local search)
3. CMA-ES (global search)
4. MCMC (explores posterior)
5. Nested sampling (explores full space)

Implementation Details¶

All algorithms are implemented in Rust for performance:

• Zero-copy: Efficient memory usage
• Parallel: Where applicable (CMA-ES, nested sampling)
• Numerically stable: Careful floating-point handling
• Well-tested: Comprehensive test suite

Source code: rust/src/optimisers/

References¶

Each algorithm page includes references to original papers and implementation details.

See Also¶

• Choosing an Optimiser
• Tuning Optimisers
• API Reference
• Tutorials