Five Sigma and the Chance of a False Cosmic Alarm

AI-generated illustration for Galileo Whispers of a qualitative binary-black-hole concept. It is not an optical observation of GW150914, a waveform or a representation of detection significance.

Five sigma expresses how unusual a detection statistic would be under a specified background model, using an equivalent Gaussian tail. It does not mean there is only that tail probability that the astronomical explanation is wrong. The background, search procedure and opportunities to obtain an extreme result are part of the claim.

Advertisements

Begin with the question the statistic answers

A search first defines a way to rank candidate events. It then asks how frequently the assumed background would produce a statistic at least as extreme. A small tail probability indicates tension with that background. Turning it into a probability that a source model is true would reverse the conditioning and require additional assumptions.

The American Statistical Association’s March 2016 statement explicitly warns that a p-value is not the probability that the studied hypothesis is true. Statistical significance also does not measure an effect’s size or scientific importance. A threshold is one element of an inference, not its entire argument.

GW150914 keeps the quantities apart

The 2016 first-version GW150914 detection paper reports a matched-filter signal-to-noise ratio of 24. That is not a claim of 24-sigma discovery significance. The paper’s searches assessed significance against empirical detector backgrounds because the noise was neither perfectly Gaussian nor stationary.

The analysis used 16 days of coincident observations. Artificially shifting one detector’s timestamps by more than the intersite travel time generated accidental coincidences. Repeated shifts provided a much longer equivalent background exposure, under the assumption that relevant instrumental disturbances were uncorrelated between sites. Checks for instrumental and environmental causes remained necessary; a time shift alone cannot rule out every correlated disturbance.

In the binary-coalescence search, the reported false-alarm rate was below one event per 203,000 years, corresponding to false-alarm probability below 2 × 10⁻⁷ and significance above 5.1 sigma. The generic transient search obtained a different significance. These are results of different search definitions, not interchangeable estimates of one universal “chance the event is fake.”

Rate needs an exposure to become a probability

For an explicitly assumed constant-rate Poisson background, a false-alarm rate r over duration T gives a probability of at least one qualifying noise event:

P = 1 − exp(−rT).

When rT is very small, P is approximately rT. Using the rounded LIGO scale as an illustration, (16/365.25) years divided by 203,000 years is about 2.2 × 10⁻⁷. This checks the order of magnitude, not the paper’s precise bound; its rounded duration and rate do not recover the full analysis.

A rate has units of inverse time. Probability and sigma-equivalent significance are dimensionless. SNR is dimensionless too, but it describes a different statistic. Equal units do not make quantities interchangeable.

Count the opportunities and inspect the model

Searching many classes or candidate locations creates more opportunities for a striking fluctuation. The LIGO paper explicitly applies a trials factor for searched classes. Reporting only a favorable local tail while ignoring the wider search can exaggerate significance.

Finally, estimates of source properties use a signal model and an inference framework, including priors when Bayesian posteriors are reported. A compelling rejection of a specified noise background does not by itself assign all posterior probability to one physical explanation. Read “five sigma” together with its background, search scope, validation and source analysis.

Newsroom
Newsroomhttp://galileowhispers.com
Newsroom is the editorial byline of Galileo Whispers, an English-language astronomy publication. Our focus is explaining stars, planets, galaxies, and the science behind them, alongside practical observing guides and celestial events.

Latest articles

Related articles