Ten trials produce eight successes. Is the success probability 0.8?

It might be reasonable to begin there. But the number 0.8 is not the only way to express what we have learned. We may have previous evidence, uncertainty about the data-generating process, a need to predict the next trial, or a decision that depends on the probability of exceeding a threshold rather than on one point estimate.

Bayesian inference treats uncertain quantities probabilistically and updates those probability statements when evidence arrives. A prior distribution represents uncertainty before the current data. A likelihood represents how the observed data relate to possible parameter values under a model. Bayes’ rule combines them into a posterior distribution.

This is not a licence to insert personal belief into a calculation and call the result objective. Priors, likelihoods, model structure, data quality and computation all become explicit parts of the inferential argument. The resulting posterior is conditional on those choices.

This article owns the Bayesian updating job. Statistical Inference and Uncertainty owns the wider comparison of inferential ideas; Models and Simulations owns model credibility; and Value of Information owns the decision-specific benefit of obtaining more evidence.

Bayes’ rule is simple; Bayesian modelling is not

In its familiar form, Bayes’ rule says:

posterior ∝ likelihood × prior

The equation is compact because it suppresses the modelling work. We still need to decide what quantity is uncertain, what probability model connects that quantity to the observations, which prior distribution is appropriate, whether observations are independent, whether the data are measured well, and whether the computation actually approximates the intended posterior.

Bayesian inference is therefore best understood as a workflow:

1. Define the question and target quantity.
2. Build a probability model for the data-generating process.
3. Specify prior information and uncertainty.
4. Check the implications of the model before fitting.
5. Condition the model on observed data.
6. Compute or approximate the posterior.
7. Check computation and model fit.
8. Summarise the posterior for the reader’s question.
9. Make predictions or decisions only within the model’s justified scope.
10. Update again when new evidence arrives.

A worked Beta-Binomial example

Imagine a fictional process in which each trial is classified as success or failure. Let p be the underlying success probability. We observe eight successes and two failures in ten trials.

For a simple demonstration, assume trials are conditionally independent given p and use a Binomial likelihood. Before observing the ten trials, assign p a Beta(2,2) prior. This prior is symmetric around 0.5 and places less mass near 0 and 1 than a uniform Beta(1,1) prior.

Beta and Binomial distributions are conjugate, so the posterior is available analytically:

prior:      Beta(2,2)
data:       8 successes, 2 failures
posterior:  Beta(2+8, 2+2) = Beta(10,4)

The posterior mean is 10/(10+4) ≈ 0.714. A central 95% posterior credible interval is approximately 0.462 to 0.909.

These numbers are not the same as saying “the observed success rate is 0.8”. The data still contain eight successes out of ten. The posterior combines those observations with the prior model. The difference between 0.8 and 0.714 is not a correction of the data; it is the consequence of the stated inferential model.

All numbers in this example are constructed for explanation and do not describe an eduKate service, learner or experiment.

A prior is a distribution over uncertainty, not a hidden vote

Priors can encode previous measurements, historical evidence, physical constraints, domain knowledge or deliberately weak regularisation. They can also be badly chosen. The important discipline is to make their role visible.

A prior may be informative when substantial previous evidence constrains plausible values. It may be weakly informative when the goal is to rule out implausible extremes without dominating realistic data. A nominally “noninformative” prior is not automatically neutral in every parameterisation.

Calling a prior objective can therefore obscure important choices. A prior that is flat in p is not flat in log-odds. A prior that looks broad on one scale can place surprising mass on derived quantities. Prior predictive simulation is one way to inspect these consequences before the data are used.

The likelihood is the bridge from parameter to data

The likelihood answers a model-based question: if a candidate parameter value were the one governing the process, how compatible would the observed data be with that value?

In the Binomial example, the likelihood treats ten trials as having a common probability p and conditional independence. If probabilities differ across trials, if outcomes influence one another or if classification errors occur, the likelihood can be structurally wrong even when Bayes’ rule is applied flawlessly.

This is a major boundary. Bayesian inference does not immunise a study against measurement error, selection bias, confounding or a poor sampling frame. Those problems belong to the data-generating model and research design.

Use Measurement Error and Misclassification for recorded-value problems and Observational Studies for design limitations.

The posterior is conditional, not absolute truth

After the model is conditioned on the observed data, the posterior distribution represents uncertainty about the model’s unknown quantities given the data and assumptions.

Within the simple Beta-Binomial model, a statement such as “the posterior probability that p exceeds 0.6 is X” has a direct probabilistic interpretation. But the conditioning set matters. The statement does not include the probability that the Binomial model itself is an adequate description unless model uncertainty has been explicitly represented.

This is why “there is a 95% probability the parameter lies in this interval” can be legitimate for a Bayesian credible interval under the stated posterior, while the same sentence is not the ordinary interpretation of a frequentist confidence interval.

Different inferential frameworks can sometimes produce numerically similar intervals. Similar numbers do not make their interpretations identical.

Prior sensitivity is part of the result

Return to the eight-success, two-failure example. Under a uniform Beta(1,1) prior, the posterior becomes Beta(9,3), with posterior mean 0.75 and a central 95% credible interval of roughly 0.482 to 0.940.

Under a much more concentrated Beta(20,20) prior centred on 0.5, the posterior becomes Beta(28,22), with posterior mean 0.56 and a central 95% credible interval of roughly 0.422 to 0.693.

Those are very different answers because ten observations are not strong enough to overwhelm a highly informative prior. This is not automatically a flaw. If the strong prior genuinely represents a large, applicable body of previous evidence, it may deserve influence. If it was chosen merely to pull the result toward a preferred conclusion, the analysis is not credible.

A useful Bayesian report therefore shows how important conclusions move under defensible prior alternatives. “The posterior changed under a completely implausible prior” is not informative. “The decision changes across two priors both supported by reasonable external evidence” is.

Weakly informative priors can stabilise noisy models

In regression and hierarchical models, completely unconstrained parameter estimates can become extreme when data are sparse or separation occurs. Weakly informative priors can regularise those estimates toward a plausible region without pretending the parameter is already known precisely.

This is conceptually related to regularisation in prediction. The Bayesian version places a probability model on the parameters and carries the resulting uncertainty into the posterior. The amount and direction of regularisation should still be inspected.

A prior that improves numerical stability can still be scientifically inappropriate. Computational convenience is not sufficient justification for a prior.

Posterior prediction returns the model to observable quantities

Parameter estimates are often intermediate objects. Readers may care more about what will be observed next.

In the Beta(10,4) posterior, the posterior predictive probability of success on the next exchangeable trial is the posterior mean of p, 10/14 ≈ 0.714. For several future trials, posterior prediction averages over uncertainty in p rather than plugging in one fixed estimate and pretending it is known.

Posterior predictive distributions can answer questions such as: what range of counts might we see next month? How often would a replicated summary be more extreme than what we observed? Which future outcomes are plausible under the fitted model?

Prediction remains conditional on exchangeability and model scope. If the process changes, yesterday’s posterior may not describe tomorrow’s mechanism.

Posterior predictive checking asks whether the fitted model can reproduce important features

A Bayesian workflow should not end when the sampler finishes. Simulate replicated data from the fitted posterior predictive distribution and compare them with relevant features of the observed data.

If the model repeatedly produces much less dispersion than the observed data, the assumed variance structure may be inadequate. If it cannot reproduce zero inflation, tail behaviour, subgroup patterns or temporal dependence, a posterior summary of a poorly fitting model is not rescued by mathematical elegance.

Posterior predictive checks are not a universal pass/fail test. A model can fit selected summaries and fail elsewhere. The checks should be connected to the features that matter for the intended use.

Hierarchical Bayes shares information without pretending groups are identical

Suppose several classes each have their own underlying success probability, but those probabilities are believed to arise from a common population of classes. Estimating every class completely separately can be unstable for small classes. Pooling all classes into one rate erases real differences.

A hierarchical model offers partial pooling. Group-specific parameters are modelled as draws from a population distribution. Small or noisy groups are pulled more strongly toward the common pattern; groups with substantial data retain more independent information.

This is not a free accuracy upgrade. The model assumes a relationship among groups. If one group belongs to a genuinely different process, partial pooling can be misleading. Clustered and Multilevel Data remains the canonical owner for the wider structural problem.

Bayesian updating through time requires attention to process change

A tempting story is: today’s posterior becomes tomorrow’s prior. Mathematically, sequential conditioning can equal batch conditioning when the model and data assumptions are unchanged. Operationally, the world may change between batches.

A policy change, new measurement instrument, different population or altered intervention can break exchangeability. Reusing the previous posterior unchanged would then smuggle obsolete assumptions into the new analysis.

Version the model, evidence window and prior provenance. A living Bayesian analysis should know whether new data are another observation from the same process or evidence that the process itself has changed.

MCMC is a computational method, not the definition of Bayesian inference

Simple conjugate models can be solved analytically. Complex posteriors often require numerical approximation. Markov chain Monte Carlo, or MCMC, constructs a dependent sequence of samples whose long-run distribution targets the posterior.

The current Stan Reference Manual version 2.39 documents Hamiltonian Monte Carlo, the no-U-turn sampler, variational methods, Laplace approximations and posterior analysis tools. Stan is one implementation ecosystem, not the definition of Bayesian statistics.

An MCMC chain can run without adequately exploring the target posterior. Divergent transitions, poor mixing, low effective sample size or convergence problems are computational warnings. Ignoring them because a posterior plot looks smooth converts approximation error into false certainty.

Monte Carlo uncertainty sits on top of posterior uncertainty

Suppose the true posterior mean under the fitted model is fixed, but an MCMC algorithm estimates it using a finite correlated sample. Re-running the sampler can change the numerical estimate slightly. That variability is Monte Carlo error.

The width of the posterior distribution expresses uncertainty about the parameter under the model. Monte Carlo standard error expresses uncertainty in our numerical estimate of a posterior summary. Running the sampler longer can reduce the latter. It does not make the underlying posterior narrower.

This is the same general distinction developed in Bootstrap and Resampling: more computation can stabilise a calculation without adding substantive observations.

A prior can be checked before data are allowed to rescue it

Prior predictive checking simulates observable outcomes from the model before conditioning on the current data. This asks whether the prior and likelihood together generate worlds that make sense.

A regression prior that regularly implies impossible test scores, billion-fold service volumes or negative durations reveals a scale problem before any posterior result is interpreted. The data may later pull the posterior into a reasonable region, but the prior predictive failure still teaches us that the model was badly calibrated.

Prior predictive checks are therefore not merely philosophical. They are practical unit tests for probabilistic models.

Bayes factors and model probabilities require extra care

Bayesian model comparison can place prior probabilities on models and update them using marginal likelihoods. Bayes factors compare how well models predict the observed data after averaging over their parameter priors.

These quantities can be highly sensitive to prior choices on parameters that may seem unimportant in ordinary estimation. A diffuse prior can penalise a model because it spreads predictive mass over a huge space. Using a Bayes factor without checking that prior predictive scale can turn a modelling convenience into a decisive comparison.

Information criteria and predictive cross-validation answer related but not identical questions. A model comparison should say whether it is choosing a predictive model, estimating posterior model odds or testing a specific scientific contrast.

Bayesian probability does not remove causal identification

One can place priors on causal parameters, fit Bayesian propensity models or build Bayesian hierarchical causal models. None of those operations automatically identifies a causal effect from confounded data.

The posterior will be conditional on the causal structure that was assumed. If an unmeasured confounder has been omitted, a narrow posterior can express high confidence about the wrong identified quantity.

For interventions and causes, route through Causal Inference. Bayesian machinery can quantify uncertainty inside an identification strategy; it does not replace the strategy.

Bayesian analysis does not remove missing-data assumptions

Missing values can be represented inside a joint Bayesian model and sampled along with other unknown quantities. This can be elegant because uncertainty propagates through the complete model.

But if missingness depends on unobserved outcomes in a way the model ignores, the posterior can still be biased. Bayesian computation does not turn missing at random into a law of nature.

Use Missing Data Analysis for the canonical missingness mechanisms and sensitivity problem.

Decision-making needs utilities as well as posteriors

A posterior distribution tells us what the model says about uncertain states. It does not by itself tell us what action to take. Actions have costs, benefits, constraints and asymmetric consequences.

A decision-theoretic analysis combines posterior uncertainty with a utility or loss function. One action may be preferred even when it is not the most probable state because the downside of being wrong is very different across choices.

This is why “choose the most probable explanation” is not a universal decision rule. Probability and preference are separate objects. Value of Information extends this logic to deciding whether more evidence is worth obtaining before action.

A reporting standard for Bayesian work

A reader should be able to reconstruct the probabilistic argument. Report the likelihood, prior distributions and their rationale, model structure, target quantities, posterior summaries and predictive quantities that matter.

For computational models, report the software and version, algorithm, chain configuration where relevant, convergence and sampling diagnostics, and how Monte Carlo error was checked. Show meaningful prior sensitivity and posterior predictive checks.

Separate model assumptions from observed facts. “The posterior probability is 0.9” should not silently become “there is a 90% objective fact in nature” when the result depends strongly on a disputable prior or likelihood.

Where the result feeds a consequential decision, state the decision rule separately from the inferential result.

A learner’s checklist

• What uncertain quantity is being modelled?
• What does the prior say before the current data?
• Why is that prior plausible on the actual scale of the problem?
• What likelihood links parameters to observations?
• Which independence, measurement and sampling assumptions does it make?
• How much did the data move the prior?
• How sensitive is the conclusion to defensible alternative priors?
• Can the fitted model reproduce relevant features of the observed data?
• Is computation reliable enough for the reporting precision?
• Does the decision follow from a declared utility or threshold rather than probability alone?

What Bayesian updating is really doing

Bayesian inference gives us a language for learning under uncertainty in which previous information, new evidence and remaining uncertainty can inhabit the same probability model.

Its great advantage is coherence: uncertainty can flow from parameters into predictions and decisions instead of disappearing behind point estimates. Its great danger is also coherence: a beautifully coherent posterior can make an inadequate model feel more certain than it deserves.

The correct habit is therefore not “trust the posterior”. It is “inspect the probability model, inspect the evidence, inspect the computation, then interpret the posterior within the boundary those inspections justify.”

Sources and further reading

Current Stan documentation and NIST records were checked for this edition on 5 September 2026. The Beta-Binomial calculations are original worked examples. They do not describe an operational eduKate dataset.

• Stan Development Team, Stan Documentation, version 2.39 and Stan Reference Manual, accessed for this edition.
• Stan Development Team, MCMC Sampling using Hamiltonian Monte Carlo.
• Raghu Kacker and Albert Jones, NIST, On Use of Bayesian Statistics to Make Guide to the Expression of Uncertainty in Measurement Consistent, 2003.

Continue through eduKate: Statistical Inference and Uncertainty → Models and Simulations → Regression Models → Value of Information → Research Collections Directory.