AI's safety gatekeepers step into the spotlight — and wonder who pays
The weekend read is about the people paid to say no: independent AI evaluators are suddenly the industry's most important small organizations, while a culture essay asks whether "made by humans" is becoming a premium product.
Independent evaluators went from a sleepy corner of the AI industry to its center of gravity — and nobody has settled who funds them. CNBC's weekend feature lays out how nonprofits like METR, Apollo Research and Transluce are being asked to monitor the models of labs that collectively raise tens of billions, in the absence of any federal regulator: Anthropic pledged last month to embed independent evaluators inside the company, Sam Altman endorsed the idea within hours, and Trump's late-September "morally binding" accord gently encourages signatories to partner with an outside auditor. The money question is unresolved — METR raised about $71 million in six months against $13.6 million in total contributions for 2024 and runs on fewer than 50 full-time staff, while benchmark startup Vals AI grew from 8 employees to roughly 30 and announced a $40 million round in August, per CNBC. The friction is already public: OpenAI fired three employees last week over handling sensitive information, two of whom say the real issue was how they communicated with third-party evaluators, and an open letter from fired safety researchers warns of a chilling effect; OpenAI disputes that reading and says it is finalizing contracts with outside assessors to announce in the coming weeks. Our take: the labs are happy to fund the referees as long as the referees need the labs — which is exactly why California's auditor registry and the bipartisan FRONTIER Act's licensed-evaluator provision matter more than any voluntary one-pager. We saw the pledge stage of this story earlier this month — Altman matches Amodei's evaluator pledge, Musk says 'Dario is right'.
A "human premium" may be emerging for art made by people — even when nobody can tell the difference. BBC InDepth's essay runs on three stats from Deezer's research (published in April): 44% of tracks uploaded to the platform are AI-generated, 97% of listeners in a blind test couldn't distinguish AI from human-made, and more than half of those surveyed still said they care whether a human made it. That gap between inability to detect and unwillingness to accept is the whole story — as the New Yorker's Kyle Chayka puts it, once AI writing reads as "generic, obsequious and banal," readers and designers lean into work that is messier and weirder, and academics are now racing to build a universally recognized "human made" symbol in the spirit of Fairtrade. Our take: provenance is becoming the product, which is the same force behind the labeling fights we covered in Nikon disqualifies contest winner over generative AI use.
What to watch: whether OpenAI's promised evaluator contracts land with published findings — and real redaction limits — or a softer announcement.
Should labs be required to fund independent evaluators through pooled contributions rather than their own contracts? Tell us in the comments.
