“It is likely that supply will tighten next quarter.” One reader hears sixty percent, another ninety. The writer may have meant either, and may also have meant that they were not sure. Analytic standards exist to remove exactly this ambiguity, and they are worth following outside intelligence work too.
This article is for analysts, editors and reviewers who write or approve assessments: in intelligence and security, but equally in risk, finance, public health and operations, wherever a report states how probable something is.
Probability words mean different things
The problem is old. Sherman Kent, writing about the CIA’s Board of National Estimates, found that its members had read the phrase “serious possibility” in an agreed estimate as anything from about 20 to 80 percent [3]. Surveys since then have found the same spread among the general public and among professionals: phrases such as “real possibility”, “serious possibility” and even “likely” are read across a wide range of probabilities [4]. The words feel precise to the writer and are not precise to the reader.
Two remedies follow. Tie each word to a declared range of probability, and use the range consistently across products so that readers learn it.
Declared scales
The US Intelligence Community’s analytic standards, ICD 203, give one such scale (Table 1). Each range has two equivalent terms, and analysts are strongly encouraged not to mix terms from the two sets [1].
| Term | Alternative term | Range |
|---|---|---|
| almost no chance | remote | 01–05% |
| very unlikely | highly improbable | 05–20% |
| unlikely | improbable (improbably) | 20–45% |
| roughly even chance | roughly even odds | 45–55% |
| likely | probable (probably) | 55–80% |
| very likely | highly probable | 80–95% |
| almost certain(ly) | nearly certain | 95–99% |
Table 1. Expressions of likelihood in ICD 203 [1]. A product draws its terms from one of the two sets.
The UK’s Professional Head of Intelligence Assessment publishes a different scale, the Probability Yardstick, used across UK government assessment (Table 2) [2]. Its seven ranges are approximate and deliberately separated by gaps; as UK Defence Intelligence puts it, the scale is not continuous “to avoid a false impression of accuracy” [7]. Other fields have their own scales: the IPCC’s guidance for climate assessments uses nested, overlapping ranges [5], and threat-intelligence formats such as STIX record confidence as a number out of one hundred.
| Term | Approximate range |
|---|---|
| Remote chance | >0% – ≈5% |
| Highly unlikely | ≈10% – ≈20% |
| Unlikely | ≈25% – ≈35% |
| Realistic possibility | ≈40% – <50% |
| Likely or probable | ≈55% – ≈75% |
| Highly likely | ≈80% – ≈90% |
| Almost certain | ≈95% – <100% |
Table 2. The PHIA Probability Yardstick, as published in 2025 [2]. Note the gaps between ranges.
No scale is universally right, and practice differs on how the numbers reach the reader. UK assessments use the words and show the yardstick alongside, partly to avoid implying more precision than the evidence holds [2]; other products print the range beside the word, as in “likely (55–80%)”. Either way, what matters is that a product names its scale, uses it consistently, and lets readers see the numbers behind the words.
Two different questions
Likelihood is a statement about the world: how probable is the event? Confidence is a statement about the analysis: how good is the basis for the judgment, in terms of sources, reasoning and the assumptions it rests on? A judgment can be “likely” with low confidence, when thin evidence points one way, or “unlikely” with high confidence, when strong evidence points against it.
Mixing the two in one phrase, such as “we are highly confident it will probably happen”, makes both harder to read. ICD 203 states that products expressing a confidence level must not combine it with a degree of likelihood in the same sentence [1]. Putting confidence in its own sentence, with its reasons, keeps each statement checkable.
Making confidence explicit
Confidence is often written as a feeling. It is more useful as a summary of named factors that a reader can inspect. The UK’s Analytical Confidence Ratings, for example, assign high, moderate or low confidence after evaluating three categories, Information Base, Analytical Rigour, and Complexity & Volatility [2]. In practice, the factors are these:
- Sources. How reliable are they, and are they independent? Two reports that copy one original are one source, not two.
- Corroboration. Do independent lines of evidence agree?
- Currency. How recent is the evidence relative to the question?
- Rigor. Does the method fit the question, and do independent estimates agree?
- Assumptions. Would the judgment change if a key assumption were false?
- Volatility. How quickly is the situation changing?
A good confidence statement names the level, what limits it and what would raise it: “Moderate confidence: limited by a single source group; another independent source would strengthen it.” A reader then knows both how much weight to give the judgment and what to watch for. Where a factor has not been assessed, saying so is better than assuming it is fine.
Habits that make judgments checkable
- Start from a number. Decide the probability first, then let the scale supply the word, never the reverse.
- Keep facts, assumptions and judgments apart. Use consistent signals, such as “reporting shows”, “we assume” and “we assess”, so a reader can tell which is which.
- Test the key assumptions. For each, ask what the judgment would be if it were false. An assumption that would change the conclusion deserves to be named in the product.
- Consider alternatives. State the most plausible competing explanations and why they rank lower. Where two analysts disagree widely, present both views rather than an average.
- Explain change. When a judgment differs from the last one on the same question, say what new or dropped evidence moved it.
- Make judgments resolvable. Where possible, state what would count as the event happening and by when. A judgment with a due date can be scored afterward.
- Keep score. Over time, compare stated probabilities with what happened, for example with the Brier score [6]. A team that says “likely” should find that such events happen about as often as its scale implies.
Common mistakes
- Words with no scale. “Possible”, “could” and “may” are not on most scales, because almost anything is possible. They convey no probability at all.
- Mixed scales. Switching between “probable” and “likely” within one product, or between different organizations’ scales, invites readers to infer a difference that is not there.
- Confidence used as likelihood. “We are confident the merger will close” usually means “very likely”, and hides how good the evidence is.
- Precision the evidence cannot bear. A point estimate of 73% implies a resolution the analysis rarely has. A range is usually more honest.
- Averaging disagreement. If one analyst says “unlikely” and another “likely”, “roughly even chance” describes neither view.
Sources
- Office of the Director of National Intelligence. Intelligence Community Directive 203, Analytic Standards (2015, as amended). dni.gov
- UK Professional Head of Intelligence Assessment (2025). Explaining Uncertainty in UK Intelligence Assessment: the Probability Yardstick and Analytical Confidence Ratings. gov.uk
- Kent, S. Words of Estimative Probability. Studies in Intelligence, Central Intelligence Agency. cia.gov
- Mauboussin, A., & Mauboussin, M. J. (2018). If you say something is “likely,” how likely do people think it is? Harvard Business Review. hbr.org
- Mastrandrea, M. D., et al. (2010). Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties. ipcc.ch
- Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review 78(1), 1–3.
- UK Defence Intelligence (2023). Defence Intelligence: communicating probability. gov.uk