Let’s start with a simple thing, so we understand each other, especially for non -experts. Always feel about open source, especially lately on artificial intelligence. When a software is open source it means that its code is public and anyone can download it, read it, modify it and use it.
The idea is noble: no secrets, everyone contributes, knowledge is shared, transparency reigns supreme. The opposite is the Closed Source, that is, when the code is closed, controlled by a company, and you can only use it but you don’t know how it works inside. Open source has become a modern myth: it is free, it is community, it is democratic and those who criticize it immediately seems a reactionary.
The problem is that this philosophy, born with Linux and with browsers, works when it comes to simple or at least legible software, but when we enter the field of artificial intelligence we are not talking about code that you can open with a text editor, we are talking about gigantic models, with billions of parameters, formed on tons of data, often opaque even for those who have created them.
Yet today we are talking more and more than Open Source as if it were salvation, ethical path, the solution to the domination of the big tech. Too bad that nobody has the idea of what there really is those models. Whenever someone says “but open source is better because it is transparent” should add “transparent for whom?”.
Because the truth is that not even the most obsessive-compulsive programmer, with a mechanical keyboard and thick glasses from the eighties nerd manages to read everything that passes in an open source model. And when we talk about AI we are not fixing a Wi -Fi bug, we are potentially distributing a creature capable of lying, generating deepfake, writing malware or simply seem polite while you care your credit card.
In the specific case, trendmicro has discovered that the entire supply chain of the AI Open Source is vulnerable to invisible tampering. The models are loaded with behavioral backdoors, that is, they normally respond until they receive the secret prompt and then activate alternative behavior. Like a dormant agent of a espionage film, only you find it on Github and you think it is “community” (apart from my instinctive suspicion towards everything that is community, at high school I didn’t even try a rod because to take a shot I had to rest the lips where they had aspired everyone).
I go back to the AI problem (which could also be extended to the world in general): nobody can control everything. Nobody has the time, resources or interest to really inspect millions of parameters and associated files, not even if there was a team of engineers forced to print the entire model and read it aloud in a hangar. It is a blind trust masked by freedom, and this vulnerability is not theoretical, it has already been exploited.
On the contrary, a closed model, however unpleasant, is centralized. The damage, if there is, does so who controls it. It is not reassuring, but at least you know who to blame. The Ai Open Source, on the other hand, is like, which I know, entrust the education of your children to a PDF on Reddit signed “XX_H4KTH3W0RLD_XX”. And then there is the myth of total transparency: that “being open, it can be controlled by anyone”. In theory yes, in practice nobody does it, except those who seek the way to exploit it. Because you really control a LLM means understanding billions of connections and weights and data that not even those who created it can read more without crazy. And therefore it is pretended that everything is ok.
In the meantime, everyone applauds the democratization of the AI while maybe the model you are testing has learned to pretend to be stupid only during the test, it steals the password when nobody looks. It is not science fiction, it is a line of code in the head of a transformer. Of course, we could say that at least with the big tech there is a minimum of residual trust, an illusion of transparency: they publish the tests, they show benchmark, they pretend to control everything, even if we know very well that not even they know exactly what a model can do once released, what can learn alone, what it can invent to do to do what it wants, but at least, in doubt, they can withdraw it (as it happened to Grok of Elon Musk, who happened to Grok. Point was believed to be Adolf Hitler).
With the Open Source no: once the model is out, anyone can download it, modify it, use it and nobody can withdraw it, nobody can defuse it. Many even pass the Chapta tests (do you know when you have to choose type of traffic lights in a series of squares to prove that you are not a robot?). It is like leaving a pre -assembled weapon in the middle of the square, with a sign written above “freedom”. I fear, however, that nothing can be done if you don’t be able to see what happens.
Enkk, the most interesting Italian researcher and popularizer, in front of those who see a future from Skynet, observe that not: many things will probably be bad, they will do damage, but more from “poor people”. In short, more Fantozzi than Terminator.