A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis
Publisher: US Government / CIA Center for the Study of Intelligence
Date: March 2009 (45 pages)
URL: https://www.cia.gov/resources/csi/static/Tradecraft-Primer-apr09.pdf
Summary
This primer presents 12 structured analytic techniques for improving intelligence analysis. It is explicitly not a comprehensive overview of intelligence analysis — its scope is limited to techniques that help analysts challenge judgments, identify mind-sets, stimulate creativity, and manage uncertainty. The document is organized around three functional categories and a concluding strategies section.
The foundational argument: analysts face three perennial problems — complexity of international developments, incomplete and ambiguous information, and inherent limitations of the human mind. Structured analytic techniques address the human limitations component.
Core Premise: The Mind-Set Problem
“Intelligence analysts should be self-conscious about their reasoning processes. They should think about how they make judgments and reach conclusions, not just about the judgments and conclusions themselves.” — Richards Heuer, The Psychology of Intelligence Analysis
All individuals process information through “mental models” (also called frames or mind-sets) — experience-based constructs of assumptions and expectations. These models:
- Cause analysts to perceive what they expect to perceive
- Are resistant to change even when faced with contradicting evidence
- Cause new information to be assimilated (sometimes erroneously) into existing models
- Lead conflicting information to be dismissed or ignored
Seasoned analysts may be more susceptible to mind-set problems because of expertise and past success with existing models.
Historical examples of unchallenged strategic assumptions:
- 1941: Japan would avoid all-out war (Pearl Harbor)
- 1950: China would not cross the Yalu River (Korean War)
- 1962: Soviets would not put nuclear weapons in Cuba (Cuban Missile Crisis)
- 1973: Arabs lacked sufficient coordination for surprise attack (Yom Kippur War)
- 1989: East Germany could not unify against Soviet wishes
- 1998: India would not conduct a nuclear test
- 2003: Iraq was continuing WMD programs
Cognitive and Perceptual Biases (Taxonomy from Primer)
Perceptual Biases:
- Expectations — we perceive what we expect; more information needed to recognize unexpected phenomena
- Resistance — perceptions resist change even in the face of new evidence
- Ambiguities — initial exposure to ambiguous stimuli interferes with accurate perception even after better information arrives
Biases in Evaluating Evidence:
- Consistency — small consistent datasets engender more confidence than larger inconsistent ones
- Missing Information — difficulty judging the impact of gaps even when the gap is known
- Discredited Evidence — even when evidence is disproved, the perception it supported may persist
Biases in Estimating Probabilities:
- Availability — probability estimates influenced by how easily one can imagine or recall similar events
- Anchoring — probability estimates only incrementally adjusted in response to new information
- Overconfidence — especially in those with considerable expertise
Biases in Perceiving Causality:
- Rationality — events seen as part of orderly causal patterns; randomness and accident rejected
- Attribution — others’ behavior attributed to fixed nature; own behavior attributed to situation
The 12 Techniques
Category 1: Diagnostic Techniques
(Make analytic arguments, assumptions, or intelligence gaps more transparent)
1. Key assumptions check
List and review key working assumptions on which fundamental judgments rest. Most useful at the start of a project. A four-step process: review current analytic line → articulate all premises → challenge each assumption → refine to only “must be true” assumptions. [key::diagnostic]
Example: DC Sniper case (2002) — explicit assumption checking could have prevented premature narrowing of the suspect profile.
2. Quality of information check
Evaluates completeness and soundness of available information sources. Detect deception, identify intelligence gaps, help policymakers understand analyst confidence levels. [key::diagnostic]
Example: Senate Intelligence Committee report on Iraq identified “over-reliance on a single, ambiguous source” as an analytic error.
3. Indicators or signposts of change
Periodically review observable events or trends to monitor targets and warn of unanticipated change. Creates objective baseline, can “depersonalize” disagreements among analysts. [key::diagnostic]
Example: Political instability tracking matrix — indicators across government capacity, legitimacy, opposition activity, economic factors, environmental issues, and trigger mechanisms.
4. Analysis of competing hypotheses (ach)
Identification of alternative explanations (hypotheses) and evaluation of evidence that will disconfirm rather than confirm them. Most effective with a small team; best for controversial issues requiring a clear audit trail. Creates a matrix of hypotheses × evidence; rates evidence as consistent (C), inconsistent (I), or neutral (N). Focus is on disproving hypotheses rather than proving one. [key::diagnostic]
Example: Tokyo sarin attack (March 1995) — matrix evaluated four hypotheses: kooky cult, terrorist group, political movement, criminal group.
Category 2: Contrarian Techniques
(Explicitly challenge current thinking)
5. Devil’s advocacy
Challenging a single, strongly held view by building the best possible case for an alternative explanation. Most effective against analytic consensus or critical assumptions. [key::contrarian]
6. team b
Two separate teams analyze the same problem from opposing assumptions or perspectives. Forces explicit articulation of competing analytical frameworks. [key::contrarian]
7. High Impact low Probability analysis
Examines events assessed as unlikely but which would have significant consequences if they occurred. Forces analysts to work through scenarios they might otherwise dismiss. [key::contrarian]
8. What if? analysis
Assumes that an unexpected event has occurred and asks analysts to explain how it could have happened. Identifies indicators/signposts and forces consideration of low-probability scenarios. [key::contrarian]
Category 3: Imaginative Thinking Techniques
(Develop new insights, different perspectives, alternative outcomes)
9. Brainstorming
Generates hypotheses and alternative explanations through free association in a group setting. Foundation technique used with many other SATs. [key::imaginative]
10. Outside In thinking
Examines the macro-level environment and how external forces (economic, political, social, technological) might shape an issue from the outside in. [key::imaginative]
11. Red team analysis
Analysts adopt the perspective of an adversary or other actor to evaluate courses of action, capabilities, and intent from that actor’s point of view. [key::imaginative]
12. Alternative futures analysis
Develops a range of plausible future scenarios by systematically varying key drivers and uncertainties. Prevents fixation on a single projected outcome. [key::imaginative]
Key References
- Richards j. heuer jr. — The Psychology of Intelligence Analysis (1999), Washington: Center for the Study of Intelligence. Primary intellectual foundation for the primer.
- Sherman kent center — Occasional Paper “Making Sense of Transnational Threats,” Vol. 3, No. 1, October 2004. Source for the cognitive bias framework.
- Senate Select Committee on Intelligence — Report on US Intelligence Community’s Prewar Intelligence Assessments on Iraq
- Commission on Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction
Tags for Cross-Reference
Cognitive bias | Mind Set | Structured analytic techniques
Cia | Center for the study of intelligence