The core problem: positive speech, negative grades
Every autumn, the European Commission publishes its Progress Reports on EU candidate countries. Politicians quote them selectively, journalists summarise them briefly, and most readers take the language at face value. ALIGNED does not. These reports are diplomatic texts. That means they follow an implicit rule of moderation: problems are rarely named bluntly, and even critical assessments are wrapped in careful, forward-looking language.
A sentence like “further efforts are still needed to implement the existing framework” sounds constructive. It is not. It means the framework exists but is not working, and has not been working for long enough that the Commission feels obliged to mention it again. This gap between how something sounds and what it actually means is the central methodological challenge ALIGNED addresses.
Our approach: core-sentence analysis
ALIGNED uses a method called core-sentence analysis, a quantitative, relational content analysis approach that assigns a numerical score to each evaluative sentence in the reports. The scale runs from -1 (strongly negative) to +1 (strongly positive), with intermediate values at -0.5, 0, and +0.5.
Not every sentence in a Progress Report is evaluative. A sentence like “Presidential elections were held in April” is factual and is excluded from analysis. We also exclude two specific sentence types that appear regularly at the start of each section: the qualitative readiness assessment and the summary progress rating when these appear alone, without elaboration. When a single sentence contains two distinct claims pointing in different directions, we treat them as separate units and score each independently.
The scoring rules
| Score | What it means | Key linguistic signals |
|---|---|---|
| -1 | Strongly negative. Something that should exist does not, or a known problem continues unchanged. |
|
| -0.5 | Moderately negative. Something exists but needs to be improved or modernised in the future. The problem is acknowledged but not framed as persistent or overdue. |
|
| 0 | Neutral. A factual statement about something that would have occurred regardless of EU integration, or a goal being set for the first time with no indication of delay. |
|
| +0.5 | Moderately positive. An already-established goal is on track for future implementation. The goal exists and progress is expected, not demanded. |
|
| +1 | Strongly positive. Serbia has taken a concrete, proactive step that the Commission recognises as progress. |
|
Why “limited progress” is a negative score
This is the most important methodological decision in ALIGNED. The Commission uses a qualitative scale for progress with six possible ratings: no progress, limited progress, some progress, good progress, and in rare cases, backsliding and very good progress. Other quantitative analyses of these reports treat only “no progress” as negative, mapping “limited progress” to a neutral position. We disagree.
Diplomatic language almost never uses direct negatives. “Further efforts are still needed” means “this has not been done.” “Limited progress” means “almost nothing has changed.” Treating these as neutral erases precisely the information that makes these reports worth analysing.
Our mapping: no progress and backsliding both score -1. Limited progress scores -0.5. The absence of any rating on an opened topic scores 0, though in practice we then read the surrounding paragraph to determine a score from the text itself. Some progress scores +0.5. Good progress and very good progress both score +1.
Reliability
The dataset underlying ALIGNED v1.0.2 covers 483 evaluative sentences across 9 years of Progress Reports on Serbia (2015 to 2023), spanning 20 topic areas. Coding was conducted manually by three coders following the rules described above. Inter-coder reliability was tested on approximately 15% of the corpus, yielding a simple agreement score of 0.81, indicating high reliability. Training was provided before coding began, and a detailed codebook governed all decisions throughout.
What ALIGNED measures, and what it does not
The scores in ALIGNED reflect how the European Commission frames reform progress in its official reports. They are not an independent measure of whether reforms actually happened, whether Serbia is objectively more or less democratic than in previous years, or whether the Commission’s own framing is politically neutral. The reports themselves have an ideological function: they are used by governments to claim validation and by oppositions to claim failure, often from the same text. ALIGNED helps you see what the text actually says, sentence by sentence. What that means for Serbia’s EU trajectory is a question we leave open.