Hello, dear readers!

This is our weekly brief on remarkable AI topics, so you can keep up with the machine without spending your weekend arguing with it online.

Today's focus — the people building frontier AI suddenly want everyone to slow down. An Anthropic researcher quit with a warning that the race toward superintelligence could end very badly, and some of the biggest names in AI responded by calling for more oversight. The awkward question is: why now?

Also in this week's edition:

The AI Race Discovers Brakes

Jacob Coxon spent three years working on AI pretraining at OpenAI and Anthropic. Last week, he slammed the door on Anthropic with a very audible thud on social media. According to Coxon, the companies building AI are racing toward self-improving superintelligence even though people inside them genuinely believe it could end humanity by the end of the decade. “They are racing straight to self-improving superintelligence and gambling with our lives,” he wrote.

Then his former boss, Anthropic alignment researcher Evan Hubinger, basically confirmed the scary part. “Jacob is correct here,” he wrote, adding that he personally puts the chance of AI killing everyone at more than 10% within the next decade. Anthropic CEO Dario Amodei has been sounding similar alarms for a while, and this weekend went considerably further, calling for frontier AI development to slow down so safety measures can catch up.

This is where some people understandably lost patience. You’re telling us this now? To borrow the tone of some of the less diplomatic backlash: The Guy Who Spent Years Building a Child-Murder-o-Matic Quits, Saying the Tech Has Gone Too Far. The same industry that spent years making AI more powerful, more autonomous and more connected to the real world was suddenly warning everyone that powerful autonomous AI connected to the real world might be dangerous.

But the backlash did not stop the industry pivot. Amodei proposed outside evaluators with deep access to frontier systems, shared safety standards and more international coordination. OpenAI’s Sam Altman backed the general idea. Elon Musk did too, and Google DeepMind’s Demis Hassabis also came out in support of slowing things down. Suddenly, the people running the AI race had discovered that brakes are also useful technology.

Maybe they really are scared — and given what some of them say they believe, that would be understandable. But the sudden enthusiasm for regulation leaves another awkward question: who gets to write the rules? Tough safety standards could reduce very real risks. They could also make life much harder for smaller competitors while the companies already sitting on billions of dollars, giant compute clusters and armies of lawyers sail through the compliance process. So yes, regulating possible superintelligence sounds sensible. Letting the people building it play a major role in deciding what “safe” means deserves a little more scrutiny.

Robot See, Robot Do

Teaching robots new tricks usually takes a lot of training, examples and engineering. Skild AI’s new S1 model is built around a much simpler idea: show the robot what to do once and let it figure out the rest.

In demonstrations, S1 watched a single video and then attempted unfamiliar tasks such as potting a plant, assembling a kit, making pancakes and brewing pour-over coffee. Some of those jobs lasted several minutes and involved dozens of separate actions, rather than the usual impressive-looking demo where a robot very slowly moves one cube from A to B.

In one test, Skild says it went from recording the human demonstration to having the robot perform the task by itself in 11 minutes. So the good news is that your future robot may learn to make breakfast just by watching you. The bad news is that “I can make pancakes better than any machine” may not remain a personality trait for much longer.

AI Music Gets a License

For the past few years, the relationship between AI music companies and record labels has mostly followed a familiar rhythm: launch model, refuse to explain the training data in much detail, get sued.

Now everyone appears ready to try something more traditional. Suno launched its new generation of music models with partners including Warner Music Group, BMG and Believe, while Universal Music Group and ElevenLabs announced their own licensing deal. Their plans include tools that would let fans make AI remixes and mashups using music from artists who choose to participate.

So AI music may finally be entering its respectable phase: train first, get sued, sign licensing agreement, announce exciting new partnership. Nature is healing.

Thanks for reading AIport. Until next Monday — by then, AI will almost certainly have hired the lawyers who regulate it.