Facebook’s Meta Algorithms Harming Children by Pushing Toxic Content

A new report reveals how easy it is to prove “Meta” business logic intentionally harms children

“One of the things I was struck by was how profoundly easy it was to identify this pro–eating-disorder bubble,” said Rys Farthing, data policy director at the advocacy group Reset Australia and the leader of the research.

Farthing said that exposure to the content was primarily driven by Instagram’s suggestions about which users to follow. Test accounts that expressed an interest in weight loss or disordered eating were quickly flooded with recommendations from the platform to follow other users with these interests, including those that openly encourage disordered eating.

The most telling part is two-fold. First, Facebook tries to undermine trust in journalists, as I’ve written about before here. Their official response cited in the article is to say journalists don’t understand what it’s really like to want to profit on harming children.

Second, the researcher in fact says the exact opposite — he’s in the business of putting himself out of business.

Researchers, journalists, and advocates have been raising alarms about disordered eating content on Instagram for years, culminating in fall 2021 when internal Facebook documents provided by whistleblower Frances Haugen showed that Instagram led teen girls to feel worse about their bodies. This new report shows that Meta’s struggles to curb this kind of harm are still ongoing.

But Farthing and others hope change may be around the corner: US Sens. Richard Blumenthal and Marsha Blackburn recently introduced the Kids Online Safety Act, which would create a duty for platforms to “act in the best interests of a minor” using their services. The California legislature is considering a similar provision, modeled after the UK’s Age Appropriate Design Code, that would require companies to consider children’s “best interests” when building or modifying their algorithms.

“If we can muster the courage to actually hold tech companies to account, we could get some of this legislation through,” Farthing said. “And maybe when we have this conversation next year, I might actually have put myself out of business.”

Think hard about that contrast in integrity of work.