Research note

How Effective CA was measured, and where it is weak

Every scouting tool for Football Manager claims a better rating than the stars. This page shows what ours is better at, and how we know. Effective CA was fitted to what players actually produced in the match engine, then tested on careers it had never seen: five on FM26, and four on FM27, two of them twenty seasons long. The parts that failed are in here too.

Nine AI-managed careers FM26 and FM27 Twenty-season FM27 careers Every result held out

The question

Does it predict what a player produces better than current ability?

The stars are your assistant's opinion, scaled to your squad. Current ability is the hidden number underneath, and it is a budget rather than a forecast: two players on 150 can be a centre-back who wins everything in the air and a winger who does not. So the claim we set out to test was narrow: Effective CA predicts what a player goes on to produce in the match engine better than current ability does. A score that cannot beat CA is CA with extra steps.

Why it takes whole careers. FM throws its detailed statistics away at the season rollover. Each career here is a save captured every summer for ten to twenty seasons, and a season missed is gone for good.

The design

Measure in a world nobody is managing

Every career was played unemployed, on holiday, with every club managed by the AI. If we picked the team, the numbers would measure our selections as much as the engine.

01

Predict forwards

A season's output is set against the attributes from before it. FM develops players partly on how they played, so a same-season snapshot points both ways.

02

Compare inside a club

Every comparison is between players at the same club in the same season. Team quality, tactic and league drop out rather than being modelled.

03

Read the position off the pitch

A classifier recovers the position actually played from the stat line, about 95% accurate, into seven families. It recovers position, not role.

04

Hold everything out

Every figure is scored on careers or seasons the fit never saw. The pass marks were written down before the second career existed and have been run unchanged since.

The careers

Nine careers, each on its edition's shipped database
Career Seasons What it was for
FM26 one10The original fit. Nothing on FM26 has been fitted since
FM26 two to five10 eachFour out-of-sample tests, each against the same written criteria
FM27 one and two10, 11Does the FM26 model hold on a new match engine? Career two was saved after the rollover, so its seasons are partial
FM27 three20The FM27 weights were fitted on careers one to three
FM27 four20Kept locked, then scored once against the FM27 weights

Together the careers hold well over a million player-seasons: one player's season at one club with at least 900 minutes and an attribute snapshot from before it. The unit is the player-season, not the save. What extra careers add is independence: each is a full replication in a world the model had not seen.


The headline

Effective CA beats current ability on every career, on both editions

Effective CA puts every position on one scale: at what current ability does this position normally produce at this level? Tested as itself, against what players produced at the same club in the same season, all positions pooled:

Correlation with what players produced, within club and season
Career Current ability Effective CA
FM26 one.281.416
FM26 two.252.400
FM26 three.293.428
FM26 four.277.417
FM26 five.259.415
FM27 one.271.421
FM27 two.284.404
FM27 three.288.416

These are all the FM26 weights, including on FM27, which they were never fitted on. FM27 career three is read on its first ten seasons here, like for like with the others. The FM27 weights, and career four, are in the next section.

The gain over CA is .12 to .16 on every career, with no sign of shrinking out of sample. The gap between a player's Effective CA and his CA, the "vs CA" chip, correlates about .31 with what he goes on to produce on its own, on every career on both editions. That is the chip that says who is beating his number.

One check worth knowing about: attributes move together, so almost any set of weights separates good players from bad. On FM26, 150 profiles with random weights scored about as well as current ability does. The game's own hidden number is worth about what chance is worth, and our margin over it is the margin over chance.

By position

How far the position score beats current ability, within club and season (FM26 weights, original keeper profile)
Family FM26 five FM27 three
Strikers+.23+.32
Attacking midfielders+.23+.31
Wingers+.22+.26
Centre-backs+.24+.15
Central and defensive midfielders+.17+.10
Full-backs and wing-backs+.13+.10
Goalkeepers−.03−.03

Six of seven families beat CA on every career, on both editions. Goalkeepers do not, and get their own section below. The margins hold on newgens alone, players who exist only inside one career, so the score is not quietly re-learning who the famous players are. Among players the game prices as equals (same reputation, same wage), the score's surplus still tracks what they produce. What was never tested is transfer fees.


FM27

FM27 rewards some attributes differently, so four positions got their own weights

On FM26, refitting the weights on more careers gained nothing: between −.01 and +.01 at every position. On FM27, a refit gained at every position on every career. The match engine had moved. The clearest change is in central midfield, where on FM27 the defensive side of a box-to-box player earns real output and the FM26 weights barely counted it.

A new weight set only ships if it beats the old one on every career and every season held out, not on average. Fitted on FM27 careers one to three, that passed at central midfield, centre-back, full-back and winger, and those four ship as FM27's own weights. Strikers passed on prediction but read further above their CA, because the refit leans on hidden attributes CA does not charge for, so they keep the FM26 weights for now. Attacking midfield and goalkeepers did not pass.

Then the twenty-season career that had been kept locked was scored, once, with nothing changed afterwards:

FM27 career four, never seen by any fit: Effective CA within club and season
Family Current ability FM26 weights FM27 weights
Central and defensive midfielders.421.530.617
Centre-backs.247.405.461
Wingers.254.521.542
Full-backs and wing-backs.400.495.514

All four held on the locked career, by margins like those on careers one to three. The app picks the weight set from the save's edition. One thing to watch: on career four, FM27 centre-backs read about a point low on average, just outside our ±1 target.


The surprise

Pace is not in a player's own numbers. It is in his team's.

The settled wisdom is that pace and acceleration beat everything in the match engine. Fitted to what players produced on FM26, both dropped to a fraction of that at every outfield position. Pace adds a little to a striker's goals, and both drive dribbles; for assists, key passes, tackles and headers neither registers. Matched on current ability, the best goalscorers are on average slower than the rest.

And yet a squad's average acceleration does predict its goal difference, comparing sides in the same division and season. Whatever pace buys is real, but it shows up in other people's statistics: recovery runs, transitions, stretching a back line. We tried forcing a floor under pace anyway. It cost accuracy, and it came out the same day.

  • Heading matters at least as much as finishing for a central striker, at every ability level.
  • Vision is the largest single attribute effect in the study, for every creative output at every position that creates.
  • Some of what matters is not on the attribute screen. FM keeps attributes it never shows you, and one of them is among the strongest predictors of a striker's goals. Current ability appears to charge a player nothing for them, which is exactly the gap a score can live in. The app can weight them because it reads the save, not the screen.

The negative results

What did not work, kept at full size

Goalkeepers are the weak spot

The original keeper score lost to current ability on four FM26 careers of five and on every FM27 career. The reason is not the weights. Shot-stopping barely repeats: a keeper's save percentage correlates about .2 from one season to the next, and CA already holds most of that. Clean sheets and goals conceded turned out to be mostly about the team in front of him, so the keeper score that ships is fitted to save percentage alone. That helped, but Effective CA at goalkeeper still reads below current ability on most careers, on both editions. The app labels keepers low confidence.

A free-form model added almost nothing on FM26

Unconstrained models on the same attributes added between nothing and .01 at most FM26 positions, so the shape of the score was not what limited it. On FM27 they found more, and that is what led to the FM27 weights above.

A shipped column was withdrawn

Of seven focused skill scores, the winger goals score fell below its own bar on later careers. It was withdrawn rather than explained away.

Two smaller nulls. Player traits added nothing measurable, so no weight is spent on them. And a player outperforming his ability does not then grow faster: "he will kick on" has no support in this engine.

For anyone else digging

Two data traps that will silently ruin an FM analysis

Detailed stats are partial in leagues not simulated in full

Distance per 90 has two peaks: around 10 to 11 km, and between one and six. The low one is the leagues the game did not simulate in full, where goals and minutes are complete but most detailed stats are a fraction of the real figure, and the row still says it has detailed data. Which leagues are affected changes from save to save, so measure it in the database in front of you every time. The same gap breaks a keeper's goals prevented: it covers only the fully simulated matches, while goals conceded covers them all.

FM26 and FM27 stats are on different scales

In the same leagues, FM27 records a third to a half fewer headers won, a quarter to a half fewer tackles, and more blocks per 90 than FM26. Goals, xG and pass completion barely moved. Never put the two editions' per-90 figures on one scale or one threshold.


What this cannot license

An approximation of what the engine rewards, not a copy of the formula

One shipped database per edition

Each edition was tested on its own shipped database. A major match-engine update is a reason to test again, and FM27 is the proof: the FM26 weights held, but FM27 rewarded some attributes differently.

It measures AI management

Under your own tactics the same attributes may pay differently. Role is never observed, so every number is an average over how AI managers use a position.

Minutes are selected on

AI managers pick partly on ability and reputation. Comparing inside one club mitigates that. It does not abolish it.

It drifts in a long save

The weights favour mental attributes that do not fall with age, while CA does. Late in a long save, veteran wingers and strikers read several points above their CA. Compare across positions with care.

Judged on statistics

Against FM's own average match rating instead, on the first three FM26 careers, the score beats CA at five families of seven, and by less. Which yardstick is better is not settled, so both are stated.

The simulated match

These careers were played on holiday, so every match was simulated, not watched. That is most of the football in any save, but it is worth stating.


One deliberate omission

The method is here. The weights are not.

This page shows how the number was built, how it was tested, what it beat and where it failed. It does not publish the fitted weights or the attribute list behind each position. That line is a commercial one, and we would rather say so: the analysis is the argument, the coefficients are the product.