How we measured the talk

Why these sources

EU institutions covers anything EU bodies or their members publish or say: Commission press material, Parliament speeches, questions and resolutions, and formal Commission documents. Parliamentary questions react within weeks, which makes them useful for timing.

The press is the English-language international press: the Guardian through its free API, and Reuters, the Financial Times and Euractiv exported by hand from Factiva through the Sciences Po library. Factiva’s licence bans bulk downloading, so no script touched it.

We dropped think tanks and academic work because their publication delays make dates unreliable. The Council could not be collected: its site blocks automated access, and we did not get around that.

The filter

We split documents into paragraphs and kept those that mention graphite, battery, lithium, cobalt, critical / strategic raw materials, or export controls together with China. We dropped paragraphs under 15 words and boilerplate, and removed exact and near duplicates (paragraphs sharing 80% of their five-word sequences) within each source.

Very long EU documents were capped at 30 paragraphs each so that one impact assessment could not swamp a month.

Is the filter any good?

We hand-checked a random sample of kept and dropped paragraphs. 76–82% of kept paragraphs are on topic (precision). The filter finds 76–85% of the relevant paragraphs (recall); the worst case consistent with our sample is 37%. Off-topic paragraphs that slip through are mostly other uses of “battery”, and the coder marks them as “no claim”.

Three steps

1. Is there a dependency claim? Does the paragraph say Europe depends on, relies on, or is exposed to someone? Explicit words count (depend, reliance, vulnerable, chokepoint), and so do implicit ones (supply-chain resilience, de-risking, diversifying imports). Future demand on its own does not. If no, code No claim.
2. Where does the dependency sit? Mark every stage the paragraph names as the place of the dependency. A stage that is only mentioned gets nothing.
3. No stage named? A claim with no stage is coded No stage named.

Located vs mentioned

We also coded which stages a paragraph mentions at all. The gap between the two is a result: packs and vehicles are mentioned in 31% of paragraphs but are the place of dependency in only 5% of those.

Hard cases, decided in advance

• “China controls 90% of graphite supply”, with no stage verb: no stage named. With “mines”: mining. With “produces”: refining.
• “The battery value chain” with no stage: no stage named. Batteries as a product: packs & vehicles. A battery production step: components.
• Recycling, technology and “manufacturing” on their own are not stages.
• The UK counts as Europe; another country on its own does not.

How the 150 were drawn

• 75 from EU institutions and 75 from the press, split between the Guardian and Factiva in proportion to their size.
• About 25 per stream in each period (Before Dec 2023 · Dec 2023 – Sep 2025 · From Oct 2025), so we can see whether accuracy changes over time.
• At most two or three paragraphs from any one document, because paragraphs from the same debate are not independent.
• Paragraphs likely to be about components were over-sampled, since they are rare. Each paragraph is weighted back to its share of the corpus when we estimate error rates.

A separate set of 100 paragraphs (a 40-paragraph pilot and 60 more) was used to write the rules and to pick the worked examples in the prompt. It never touched the scoring.

Do two humans agree?

A second coder coded 40 of the 150 blind. Agreement is measured with Krippendorff’s alpha (0 = chance, 1 = perfect). It is moderate: 0.56 on the full set of codes and 0.61 on the main question. We then settled disagreements together, which changed 13 of 150 codes. This is the ceiling against which the model should be read.

The three candidates

A word list. Terms for each stage, extended with frequent word pairs from the corpus (“needle coke”, “battery grade”). A stage only counts as the place of dependency if it sits in the same sentence as a dependency word. Free and fully transparent.

Sentence similarity. A free sentence-embedding model (all-mpnet-base-v2) compares each paragraph with short model sentences such as “The EU relies on Chinese refineries to process graphite.” It catches synonyms but measures what a text is about, not what it claims.

Claude. Claude Sonnet 5.5 through the Batch API, given the full codebook, the edge-case rules, worked examples from the pilot set and a strict JSON answer format, 25 paragraphs per call. It also returns a short quote as evidence and an “unsure” flag.

How we scored them

F1 combines how many of the true cases a method finds (recall) with how many of its hits are right (precision); 0 is useless, 1 is perfect. We averaged it over the six codes and got 95% ranges by resampling the check set within its sampling groups (2,000 times).

Checks on the winner

• Another model: Claude Opus 5.5 scored 0.72 and gave identical codes on 92.7% of paragraphs; most errors are shared, so they come from hard cases, not from one model.
• The unsure flag holds 74% of errors. On the paragraphs it was sure about, the score is 0.82.
• By stream: 0.74 on EU texts, 0.67 on the press. By period: 0.68, 0.79, 0.66.
• Direction of errors: mostly over-claiming (seeing a claim where there is none), which the next step corrects.

How it works

From the check set we estimate how often Claude puts a paragraph of each true kind (no claim, no stage named, names a stage) into each box. That gives a 3 × 3 table of error rates, weighted to the corpus. Inverting it turns observed shares into corrected ones. This is the matrix version of the Rogan–Gladen correction used for imperfect medical tests. The corrected stage share is then split across stages in the proportions Claude used.

Uncertainty. We redraw the check set and the paragraphs 1,000 times and repeat the correction each time. The 95% ranges on the site come from these runs. Changes between periods are computed within each run, so the shared uncertainty about error rates cancels out; that is why changes are much tighter than levels.

Documents, not only paragraphs

For document-level figures, each paragraph gets a simulated “true” code drawn from the same error rates, and a document counts as a claim if any paragraph is one (1,000 runs, 5,489 documents). If errors were fully linked within a document, levels would be 1–3 points lower and changes almost the same.

Mentions

Mentioned stages were scored against a second hand-coded set of 150 paragraphs (agreement 0.74–0.90 by stage) and corrected one stage at a time, with the same 1,000-run ranges.

What the correction assumes

• Error rates are the same over time, across streams and across materials.
• The rate of false claims rests on four randomly drawn check paragraphs, which is why level ranges are wide.
• The two charts of stage combinations use codes as recorded.

Two methods, one rule

• PELT scans the series for the set of break points that best splits it into flat segments, with a penalty for each extra break (2σ² ln n).
• Bai–Perron fits the best set of up to five breaks by least squares and picks how many with the BIC.
• A break is confirmed when both find it within two months of each other. Each break gets a 95% range for its date (Bai 1997). We also halved and doubled the PELT penalty to see which breaks hold.

We ran this on monthly shares (no claim, no stage named, names a stage, and each stage) for all texts, EU institutions and the press, and quarterly for lithium and cobalt. Because the correction is linear, break dates are the same on adjusted and as-coded series.

Eras

As a cross-check we grouped neighbouring months with similar mixes of stages (Ward clustering, where only adjacent months may merge). It produced no clear eras.

Why each shift happened

For every break we built a file of the paragraphs either side, the topics that moved and the events nearby. Claude drafted a cause for each of 31 breaks in 9 episodes; we reviewed each one against the event dates and the paragraphs before it went on the site.

Limits

• English only. National debates in other languages are missing, except plenary speeches in translation.
• No Council texts. Member states speak only through Parliament and Commission sources.
• The press is four outlets. It stands for the international English-language conversation, not Brussels insiders or national papers. The Guardian search required Europe and China, so UK-only articles are under-counted.
• After Oct 2025 the check set has almost no stage examples (0 refining, 0 components, 1 mining). Stage accuracy after that date is carried over from earlier periods.
• Error rates are assumed constant across time, streams and materials; the check set is too small to estimate them separately.
• Thin cells. Graphite after Oct 2025 (71 paragraphs) and some lithium quarters are fragile and shown with hollow markers.
• Human agreement is moderate (0.56), so the line between “no stage named” and a stage is partly a judgement call.
• Data gaps. 10% of parliamentary questions failed to download; plenary speeches stop in Apr 2026; Oct 2026 is a partial month.

How we used AI

• Coding. Claude Sonnet 5.5 coded every paragraph. Claude Opus 5.5 recoded the check set as a cross-check. Model, prompt version and dates are logged.
• Relevance check. Claude labelled a random sample of 500 paragraphs; we hand-checked 70.
• Writing code. Claude wrote the collection, analysis and chart scripts; we ran and checked them.
• Interpretation. Claude drafted topic names, cluster names and the causes of each shift. We reviewed every one.
• What humans did. Wrote and froze the codebook; coded the 150 check paragraphs and the mentions set blind; set every decision rule before seeing results; approved the choice of coder.
• This website was built with Claude from our results.