Note: This post is also shared on Medium and LinkedIn.

Pierre de Fermat managed to puzzle generations of mathematicians with his tantalizing margin note that his “last theorem” had a proof—apparently, the margin was too narrow to contain it. While everyone was busy chasing that elusive proof, one of Fermat’s other pursuits—adequality—slipped into obscurity. Who would have guessed that this seemingly innocuous method for finding the minima of functions could one day bring us to the brink of Armageddon? And yet, here we are. Bear with me.

Over the years, Fermat’s quiet little problem of finding the minima of a function has made remarkable progress. New methods, often named after the mathematicians who concocted and refined them, have emerged slowly over the years. Like the Gauss-Newton algorithm, the Levenberg-Marquardt refinement, and more recently, the backpropagation technique. All these methods since Newton can be regarded as one form or another of “gradient descent”—a set of handy techniques that help find the unknown parameters of a parametric model to best fit a given dataset.

While the “scaling of agile” remains a mystery shrouded in corporate buzzwords, the scaling of gradient descent-like techniques has been going rather well. So much so, in fact, that a few years ago, a new breed of enormous models appeared seemingly out of the blue. These models, dubbed LLMs, or abusively “GenAI,” or yet, even more abusively, simply “AI,” are disturbingly fluent in human language. So fluent, in fact, that they’ve sparked existential debates, sometimes accompanied by loud proclamations of impending doom.

To this day, no one seems to know what will be the consequences. What we know though, is that it has happened, among other things, because of gradient descent. That, we know!

Who would have suspected that the problem of finding the minima of a function would make the prospect of opening Pandora’s box seem like a relatively tame scenario? The doomsday caused by a gradient descent? I mean, the dinosaurs were allegedly wiped out by a massive meteorite going full speed that collided against the Earth—that’s quite something! But a gradient descent?

I guess we must find it reassuring that some heavyweights in the AI realm find the doomsday scenario implausible, such as Yann LeCun, the Breton who has left an indelible mark on the field by, among a few other things, penciling the ten digits—zero to nine—on a piece of paper many times and scanning the result—or perhaps borrowing someone else’s longhand digits—to form what is sometimes dubbed the “hello world” of computer vision (go figure!).

In a far-fetched scenario, should the Breton gentleman be proven wrong, this Damocles sword is also pendulating over Fermat’s head. Think about it! Should future generations send a “Terminator” back to the past to fix the mess, who would be the target?