People who work to close or even inside the global AI ecosystem is kind of scared this week. It all started with a post on X.com that went viral with a doomsday tone content from a former OpenAI and Anthropic employee. Here’s a segment of it:

Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.

— Jacob Coxon

And contrary to most big threats that humanity got used to dealing with along its existence on planet Earth (big tigers, volcanoes, crop plagues etc.), this time the threat hides itself below the water line.

It’s a like a piraña infested river, which looks perfectly safe and calm when looked from the riverbed. You can even risk a swim if you a re brave enough.

You just need to show no weakness. School of piraña attack in a kind of chain reaction to wounded prey in the water. When a single fish detects blood or suspect movement it will do what any average fish in the planet would do: taste the game. If it tastes like food, a chain reaction starts in the school and all fishes from it will attack at the same time. “Food is low, so bite as much as you can”; is how their survival instinct work.

Some of use have resources on the Internet that found their way to survive in the global Internet river without ideal levels of maintenance and security hardening.

Threats would evolve at a human limited pace which could be dealt by security teams with patches and security updates on a reactive but effective fashion.

Bots were a threat, of course, but not because of their capabilities, but for their persistence. They will try and try until they find a vulnerable resource, then would exploit it.

Exploitation would happen within the boundaries of the predefined malware algorithm. Anything were different from the expected, the malware would just drop it and move to another victim.

This scenario is now in our past.

AI based threats (by design or not) are adaptive enough to persist on any resource they suspect could be somehow vulnerable. Malware will all tools and artifacts known to men to overcome protective barriers and fulfill the attack.

Well, the piraña river now sounds like a wrong metaphor, but I’ll keep it anyway.

Pirañas are stone dumb but have a strong survival instinct. They won’t punch holes in a fishing boat until it sinks so they could eat its crew. On the same token, old school malware won’t create new features to itself on the fly when it runs into a Windows Server instead of a Linux one.

AI enhanced malware will, and fast and with extreme accuracy. And, of course, its persistence is now unconstrained. It will try, learn, adapt and try again indefinitely until its goals are met.

Maybe the reference to pirañas still makes sense. The threat is real, but not immediately visible to the naked eye.

Maybe our typical observability tools can no longer see the threats as they used to.

Maybe our digital resource are now deep into the river water, fully exposed to schools of digital pirañas just waiting for the tiniest weakness to be sensed, and then attack on certain victory.

Many people now are scared of predictions like Coxon’s. It’s understandable. Some are reacting in favor of halting AI development or, at least, “pacing” it so everybody else can catch-up and build defenses before it’s too late.

I believe this is a great opportunity for moving even faster ahead. Let’s intensify risk management, monitoring and other supporting tools so that we could build a combative stance.

We need this. We need to evolve our defensive infra with the same power and intention as threats are now being developed and release (again, by design or not).

Sooner than we think, we might find ourselves swimming in rivers infested with other threats, not pirañas. Equally dangerous, but way more intelligent.

original post: https://x.com/hilbertspaess/status/2097476196791709843