Some problems are easy to describe and hard to calculate.

What is the expected delay when five uncertain processes interact? What proportion of possible futures exceed a capacity limit? How much uncertainty reaches the final forecast when every input is uncertain? What does a posterior distribution look like when its normalising constant cannot be calculated in closed form?

Monte Carlo methods answer difficult numerical questions by replacing impossible or inconvenient exact calculations with repeated samples from a defined probabilistic model. The repeated draws create a computational approximation. As the number of suitable draws grows, many estimates stabilise around the model-defined quantity of interest.

That power creates a dangerous visual illusion. Ten million simulated rows can look like ten million observations. They are not. The information content still comes from the model, inputs and evidence that generated the simulated world.

This article owns the Monte Carlo computation job. Models and Simulations owns the broader question of representation, verification and validation. Sensitivity Analysis and Robustness Checks owns dependence on assumptions. Bayesian Inference and Updating owns probabilistic updating, including Bayesian uses of MCMC.

The basic Monte Carlo move

Suppose X is uniformly distributed between 0 and 1 and we want E[X²]. Calculus gives the exact answer 1/3. Monte Carlo gives another route:

1. Draw many values of X from Uniform(0,1).
2. Square each draw.
3. Average the squared values.

Using a fixed modern pseudorandom stream with seed 20260905, 10,000 draws give an illustrative estimate of approximately 0.33485, close to 1/3 ≈ 0.33333. The estimated Monte Carlo standard error of that sample average is about 0.00298.

The exact integral and the Monte Carlo estimate are different kinds of objects. The first is exact under the mathematical model. The second contains numerical sampling error because only finitely many random draws were used.

This simple example is deliberately solvable without simulation. It gives us a reference answer against which the computational method can be checked. In harder problems, that reference may be unavailable.

Why random sampling can approximate an integral

Many expectations are integrals. If X follows density p(x), then the expected value of a function g(X) is:

E[g(X)] = ∫ g(x) p(x) dx

If we can draw X₁, X₂, …, Xₙ from p, the Monte Carlo estimate is:

ḡ = (1/n) Σ g(Xᵢ)

Under suitable conditions, the law of large numbers makes this sample average converge toward the expectation. A central-limit approximation often makes the numerical error shrink at order 1/√n.

That square-root rate is both powerful and sobering. To reduce Monte Carlo standard error by a factor of ten using brute force, we may need roughly one hundred times as many independent draws.

Simulation size and evidence size are not the same thing

Imagine a model fitted from twenty measured observations. We then generate one million simulations from it. The simulated file has one million rows, but the evidence supporting the fitted model did not become one million observations.

The large simulation may give a numerically precise answer to the question “what does this fitted model imply?” It does not automatically give a precise answer to “what is true in the world?”

Keep three uncertainty layers separate:

• World uncertainty: genuine variability or unknown future states.
• Model uncertainty: uncertainty about parameters, structure, inputs or assumptions.
• Monte Carlo uncertainty: numerical error from using a finite simulation sample.

Running more simulations reduces the third. It may do nothing to the first two.

A second worked example: expected maximum

Draw two independent Uniform(0,1) values and take their maximum. Mathematics gives an expected maximum of 2/3.

With 10,000 simulated pairs under seed 20260905, the sample mean of the maxima is approximately 0.66966. The estimated Monte Carlo standard error is about 0.00234.

If we reported 0.6696644626769711 without its computational uncertainty, the extra digits would create false precision. The simulation does not support fourteen meaningful decimal places merely because the programming language prints them.

Reporting precision should reflect both the numerical error and the much larger uncertainty in whatever real-world model the computation represents.

Random-number generators are deterministic machines producing pseudorandom sequences

Most scientific Monte Carlo uses pseudorandom number generators. Given an internal state, the algorithm deterministically produces a sequence designed to have useful statistical properties.

The current NumPy random-sampling documentation describes a Generator that owns a BitGenerator and transforms generated bits into draws from probability distributions. As of the current documentation checked for this edition, default_rng uses PCG64 by default.

A seed lets another analyst reconstruct a stream. It is useful for debugging, tests and reproducibility. It does not make a model more realistic.

For parallel simulations, random streams need deliberate management. Accidentally reusing identical streams across workers can create dependence and give the appearance of more independent simulation than actually occurred.

Do not confuse pseudorandomness with real-world randomness

A random-number generator may sample a Normal distribution perfectly while the real process is not Normal. The numerical randomness belongs to the computational experiment. It does not validate the probability distribution being sampled.

Similarly, assigning a Uniform distribution to an unknown parameter does not mean the world is uniformly uncertain. It means the analyst has chosen that distribution to represent uncertainty for the calculation.

A Monte Carlo model therefore needs the same evidence discipline as any other model: source each input distribution, declare which parts are measured, estimated, elicited or assumed, and test whether the result changes under defensible alternatives.

Uncertainty propagation is one of Monte Carlo’s great strengths

Consider an output Y = f(A,B,C), where A, B and C are uncertain. An analytic formula for the distribution of Y may be difficult or impossible. Monte Carlo can draw A, B and C jointly, compute Y for each draw and inspect the resulting distribution.

The word jointly matters. If A and B are correlated, sampling them independently can create impossible or implausible combinations. The output distribution will then represent the analyst’s broken dependence model, not the system.

Use observed dependence, a justified copula, structural relationship or other appropriate joint model where needed. Independence is an assumption, not the default state of reality.

Monte Carlo is especially useful for nonlinear systems

When a system is nonlinear, averages do not simply pass through the model. In general, f(E[X]) is not equal to E[f(X)].

A threshold provides a simple example. Suppose a service fails whenever total demand exceeds capacity. The mean demand can remain below capacity while a substantial tail probability still exceeds it. Plugging only the mean into the system would miss the risk entirely.

Monte Carlo retains the distribution of inputs long enough for nonlinearities, thresholds and interactions to shape the output distribution.

Variance reduction asks how to learn more from the same simulation budget

Brute-force simulation wastes information when samples are poorly placed or the target event is rare. Variance-reduction techniques aim to reduce Monte Carlo error without simply multiplying the number of evaluations.

Common ideas include antithetic variates, control variates, stratified sampling, importance sampling and conditional Monte Carlo. Each exploits additional structure.

A control variate uses a correlated quantity with known expectation. Importance sampling draws more often from regions important to the target and reweights the draws appropriately. Stratification ensures deliberate coverage of parts of the input space.

Variance reduction is not free. A bad proposal distribution for importance sampling can produce unstable weights. A poorly chosen control variate can add complexity with little benefit. Efficiency should be measured, not assumed.

Rare-event simulation needs special treatment

Suppose the event of interest has probability one in a million. A simulation of ten thousand ordinary draws will usually see zero events. Reporting the estimated probability as zero would confuse “not observed in the simulation” with “impossible under the model”.

To study rare events, analysts may need importance sampling, splitting methods, extreme-value modelling or other specialised techniques. The appropriate method depends on the system.

This is analogous to the warning in Bootstrap and Resampling: repeated computation cannot recover a tail that the computational design almost never visits.

Quasi-Monte Carlo uses deliberately even point sets rather than ordinary randomness

Quasi-Monte Carlo methods use low-discrepancy sequences designed to cover the integration space more evenly than independent pseudorandom draws.

Sobol’ and Halton sequences are common examples. The SciPy quasi-Monte Carlo module provides current implementations of low-discrepancy samplers and helpers.

For suitable smooth low-dimensional or moderate-dimensional integrals, quasi-Monte Carlo can converge faster than ordinary Monte Carlo. Randomised quasi-Monte Carlo adds randomisation that helps uncertainty estimation while preserving favourable coverage properties.

These methods do not abolish dimensionality problems. Effective dimension, discontinuities and dependence structure still matter.

MCMC solves a different sampling problem

Ordinary Monte Carlo often assumes we can draw independent samples from the target distribution. In many Bayesian models, the posterior is known only up to a proportionality constant and direct sampling is difficult.

Markov chain Monte Carlo constructs a chain whose stationary distribution is the desired target. The draws are dependent, so ordinary independent-sample standard-error formulas need adjustment.

The Stan Reference Manual version 2.39 documents Hamiltonian Monte Carlo and the no-U-turn sampler for posterior sampling. These methods exploit gradients to move efficiently through complex continuous parameter spaces.

MCMC is therefore Monte Carlo because it estimates posterior quantities using samples, but the sample-generation mechanism and diagnostics are specialised.

Effective sample size matters more than raw MCMC length

Ten thousand highly autocorrelated MCMC draws can contain much less information than ten thousand independent draws. Effective sample size estimates the approximate number of independent draws that would provide comparable Monte Carlo precision for a quantity.

Different posterior functions can have different effective sample sizes. A chain that estimates a central mean well may still explore a tail probability poorly.

Convergence diagnostics, divergent transitions, rank-normalised split R-hat and Monte Carlo standard errors should be interpreted as computational diagnostics. Passing them does not prove the probabilistic model is scientifically valid.

Simulation-based inference can compare models to data when likelihoods are difficult

Some mechanistic simulators can generate synthetic data for a parameter setting but do not offer a tractable likelihood. Simulation-based inference methods attempt to learn about parameters by comparing simulated and observed data.

Approximate Bayesian computation, synthetic likelihoods and newer neural simulation-based methods belong to this broad family. They differ substantially in how they compare simulations to observations and approximate the inferential target.

The central risk is the same: if the simulator cannot reproduce features of the real data that matter for inference, a sophisticated inference engine may confidently learn the wrong parameter relationship.

Simulation-based inference therefore requires simulator validation, summary-statistic scrutiny where summaries are used, calibration tests and coverage checks under known simulated truths.

Nested simulation can quietly multiply error and cost

Some analyses contain simulation inside simulation. An outer loop samples uncertain parameter values; an inner loop estimates expected outcomes for each parameter draw. Value-of-information calculations and complex risk models can have this structure.

Nested Monte Carlo can be computationally expensive because numerical error exists at more than one layer. Spending the same number of simulations at every layer is rarely optimal.

Before scaling up, identify which layer dominates total numerical error and whether analytic simplification, surrogate models or variance reduction can remove unnecessary computation.

Surrogate models can accelerate simulation but introduce a new model layer

If one high-fidelity simulation takes hours, millions of evaluations are impossible. A surrogate or emulator can approximate the simulator after being trained on a selected set of runs.

Gaussian-process emulators, polynomial chaos expansions and machine-learning surrogates can be powerful. But the surrogate is a model of a model. Its approximation error should be measured and propagated when it matters.

A fast inaccurate surrogate can create extremely precise Monte Carlo answers to the wrong numerical function. Speed is not validation.

Sensitivity analysis and Monte Carlo should be connected, not confused

A Monte Carlo output distribution tells us how results vary under the chosen joint input distribution. Sensitivity analysis asks which inputs or assumptions drive that variation or alter a decision.

We can use Monte Carlo samples to estimate Sobol’ indices, Shapley effects or other sensitivity measures. But a wide output distribution does not tell us by itself which uncertainty matters most.

Likewise, a narrow Monte Carlo distribution under one chosen input model does not prove robustness to alternative models. Use Sensitivity Analysis and Robustness Checks to challenge the input ranges, dependence assumptions and structural choices.

Validation needs real observations, not only agreement between simulators

Two simulation codes can agree because they implement the same mistaken equation. Ten independent random seeds can agree because the model is numerically stable. Neither establishes that the model represents reality adequately.

The NIST Standard Reference Simulation Website, whose data-content record was last updated in September 2024, illustrates the importance of well-documented reference simulation results and comparison across methods. Such reference problems support verification and benchmarking.

Real-world validation additionally requires appropriate experimental or observational evidence. The model’s domain of validity should state where that evidence supports use.

Stopping a simulation is a statistical decision

“Run one million simulations” is not a universal standard. The required number depends on the precision needed for the target quantity and the variance of the estimator.

A practical strategy estimates Monte Carlo standard error during the run and stops when it is comfortably below the reporting or decision resolution, provided diagnostics remain acceptable.

For a tail probability, far more draws may be needed than for a mean. For a threshold decision, the simulation precision needs to be high enough to determine whether numerical noise could change the action.

Do not spend computational budget resolving the sixth decimal place of a quantity whose model assumptions shift the first decimal place.

Reproducibility requires more than recording a seed

A reproducible simulation needs the code version, input data, parameter configuration, random-number algorithm, seed or seed-generation method, software environment and any parallelisation behaviour that affects the stream.

Library and model editions matter too. If an upstream parameter file is revised, a result generated from the old file should not silently inherit the new evidence date.

Use Reproducibility and Replication for the wider distinction between computational reconstruction and independent scientific confirmation.

A Monte Carlo audit works backwards from the reported number

REPORTED RESULT
→ summary statistic
→ simulated output sample
→ simulation algorithm
→ random stream and configuration
→ input distributions and dependence
→ parameter estimates and evidence
→ model equations / rules
→ validation domain
→ observations from the world

If the path breaks, the simulation may still be an exploratory calculation, but it is weak accountable evidence.

A practical Monte Carlo checklist

• What exact expectation, probability, quantile or posterior summary is being estimated?
• What distribution is being sampled, and why?
• Are input dependencies represented?
• Which uncertainties are world, model and numerical uncertainties?
• Can a small reference case be solved exactly?
• What random-number generator and stream policy are used?
• How large is Monte Carlo standard error?
• Are rare events adequately sampled?
• Would variance reduction or quasi-Monte Carlo help?
• If MCMC is used, do convergence and effective-sample diagnostics pass?
• Has the simulator been verified and validated for the intended use?
• Does the reporting precision reflect uncertainty rather than printer digits?

What a learner should remember

Monte Carlo is not powerful because randomness is magical. It is powerful because repeated sampling converts many hard probability and integration problems into a common computational form.

The price of that flexibility is discipline. We must know which world is being simulated, why its probabilities are justified, whether the algorithm explored it correctly, how large the numerical error remains and whether the simulated world has been checked against reality.

A million simulations can make a model implication extremely precise. They cannot make an unsupported assumption true.

Sources and further reading

Current documentation and NIST reference pages were checked for this edition on 5 September 2026. The Uniform-distribution examples are original reproducible calculations used only to expose numerical error.

• NumPy, Random sampling documentation, current stable documentation accessed for this edition.
• SciPy, Quasi-Monte Carlo submodule, current documentation accessed for this edition.
• NIST, Standard Reference Simulation Website, data-content record last updated September 2024.
• Stan Development Team, Stan Reference Manual version 2.39 and MCMC Sampling using Hamiltonian Monte Carlo.

Continue through eduKate: Models and Simulations → Sensitivity Analysis and Robustness Checks → Bayesian Inference and Updating → Bootstrap and Resampling → Research Collections Directory.