Post by @teapot_ben@glammr.us
I got a call from someone I don't know.
"Hi. We have a FreeBSD server. It's been working perfectly for years, but it hasn't been updated since then. We asked AI to update it, but now it won't boot anymore. It didn't even make a backup before starting. The AI keeps trying things, everything is down, and we've already spent over €500 in tokens. Can you help us?"
"Sure. €500, plus the time it'll take me to try to fix it. Assuming it can still be fixed and nothing important has been deleted."
"That much?"
Some silence.
"Well, I hope AI takes your job away!" And he hung up.
Believing that autopilot automatically makes you an invulnerable professional pilot will ruin far more people than job losses will.
I don't know if AI will take my job away, but it's already cost them €500 and, probably, their server.
@stefano 🤯 it's amazing (by which I mean worrying) how many people seem to think they can absolve themselves of any responsibility by using AI.
I don't understand why anyone would just let AI loose on a production system like that, and even blame AI for not taking backups. Although why was it not being backed up anyway?
I wonder how much more money they will waste in tokens rather than pay a professional to fix the mess.
@teapot_ben we saw this coming when people would follow instructions from any Web site or forum without knowing what they mean, now it's been skinned with a golden sheen.
@teapot_ben @stefano Capabilities based security models (CBSMs) won't fix this. Nothing can, except a return to healthier attitudes -- valuing humans, not the flavour of the month. I think of CBSMs as more like a set of software bulkheads for CWEs to not turn into CVEs in cybersecurity terms. That they could be applied to constrain the state space for "rogue" "AI" "agents to wreck the place is but a side benefit, but I guess I need to phrase it that way when "selling" the work. CBSMs can be applied to oppress as much as liberate, though, without heed to means of understanding such as Zooko's Triangle.
