Yoshua Bengio Backs Legal Limits on AI Systems Building Successor Models

Yoshua Bengio has sharpened his warning about recursive self-improvement, backing hard limits on letting untrusted AI systems design their successors until verification and safety standards are much stronger. In remarks published May 7, the Turing Award-winning University of Montreal professor and LawZero scientific director urged companies not to use an untrusted AI system to design the next generation of AI systems; when the exchange turned to whether the trajectory leading labs expect to pursue should probably be illegal unless the competitive game changes, Bengio agreed that this captured the race dynamic.