Mirror, mirror on the wall, who’s got the highest metrics of all?

Questioning the power relations behind audience metrics in journalism

By Sophie Chauvet, ESR at SAMSA.fr

What do fields such as academia, health, police and journalism all have in common? Metrics. Ubiquitously imbricated in the fabric of our professional lives, these statistics are thought to possess a wide range of defining abilities: at times unlocking rewards through coveted funding and promotions to those abiding to them, on others condemning the recalcitrants to professional invisibility and managerial surveillance. Benefitting some and harming others, metrics are about power.

What are Metrics? A promise of efficiency and an ambiguous reflection

KPIs and scores, indicators and measurement tables, metrics are fed by data and can take different shapes and sizes. But they always come with a few promises: to increase efficiency (and profit), to replace gut feelings, to measure performance and credibility, et cetera.

Mostly, whatever the field, metrics are seen as an opportunity to dive deeply into a previously unknown pool of insights. As Desrosières put it, statistics serve as a useful mirror, for a prince, a nation, or any decision-maker, including newsroom editors, to observe a supposedly objective reflection. But some also see metrics as a tyranny that lose any meaning once they are left to rule. And, you know how the saying goes: “lies, damned lies… and statistics.”

Metrics: The questions to ask the mirror

When researching metrics, by virtue of being a component of the same socio-technical complex as big data and algorithms, one runs into similar debates and framing. It is all too tempting to fall into the trap of placing them on an unambiguous spectrum, whose cursor is either on extreme dystopia, or unquestioned idealism, and where these phenomena take on an elusive and neutral shape, in some sort of magic, or blackbox hardly within grasp.

But the question is not so black and white, and metrics can sometimes rather be a conjuror: in the black box lie a set of decisions made by the people who built it, thus invisibilized and accepted. Until the day it stops working. To take Desrosières’ analogy of the mirror further, rather than asking if metrics are good or bad, ask who built the mirror, who is holding it to the prince, and with what intentions.  And that may be particularly possible when the mirror breaks. You may wonder, how does that apply to journalism at all?

What it means for journalism:  The media crisis as a perfect storm for metrics

While metrics have a long history linked to the sociology of quantification, that ranges beyond the field of journalism, they have, in the last 20 years, found their perfect storm in the crisis of the media. As the rise of the internet has pushed media outlets in a digital world where content is cheap, outlets are many and attention is limited, online platforms offered just what the media seemingly needed: an interface where outlets could compete and monetise their credibility and already established audience.

One could argue that, in an attempt to scrape together the ever decreasing online advertising revenue, outlets created more and more content; to be read by an audience faced with an immensely broad offer, whose behavior, mediated by opaque algorithms, seemed to reflect a questionable appetite for sensational content. For the media, up until recently, this was the recipe for disaster: a race for clicks and shrinking revenues led to a decrease of quality, that accelerated a crisis of trust: trust from the audience, and trust in journalistic flair. With its promise of efficiency, incentives to use metrics were strong: the journalists whose content performed best online would be promoted, audience behavioral data could replace the failing gut feeling, and strategies could be tailored to limited resources.

The day the mirror broke: Facebook’s pivot to video

But the game was rigged: at play was the opaqueness of the informational infrastructure, providing a distorted reflection of the public, and the consolidating monopoly of platforms over these statistics. While cracks in the mirror have been rumored in the corridors of marketers who long knew not to trust all metrics, one day the mirror definitely broke: in 2016, after having dictated that videos would be favored on its feed, Facebook admitted that the video metrics provided were inflated by a significant amount. Those newsrooms, who made the gamble to pivot to video after Facebook’s first announcement, were harmed.

The lesson: Metrics are manufactured

While the possibility to gain insights from metrics, and to have an accurate reflection of the audience, is still an exciting prospect, the case of Facebook’s pivot to video offers some valuable lessons against a blind trust in these figures. Processed via cookies, filtered with algorithms, refined by softwares, and aggregated into anonymised segments, audience data looks nothing like a loyal subscriber. It rather captures a few clicks which, far from being asked to provide an accurate reflection, instead provides a basis for a common language and practice around it, to fulfill one of metrics’ fundamental promise: profit. Indeed it is only with this exchange of numbers and this benchmarking of attention that advertising and data brokering transactions can happen, a process called, in political economic terms, audience commodification.

My research in a nutshell:

While the choice of audience metrics and editorial strategies accompanying them may evolve, the following questions still remain: who holds the main mirror? How is that holding negotiated, between the prince and other mirror holders? This is still about power, relations and transactions, rather than accuracy and normativity. In this specific case of audience metrics in journalism, the arena is wide and negotiated between several, international actors: media outlets, advertisers, platforms, analytics companies, and data brokers.

To attempt to find answers, this research will use qualitative methods, such as ethnography, interviews, and document analysis, to unveil the components that form the ecosystem of audience metrics, and hopefully discover the asymmetries that threaten its equilibrium.

Project Members


Funding Image

This project is funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska Curie grant agreement No 765140