Description du poste
Capability-Based Security is part of the fix
When I was at EFF in the 2010s, I spent a lot of time fretting about the fundamental security of networked devices. Apart from the usual concerns, I struggled to imagine a transition to a redecentralized future with fragile personal computers and servers. Peter Eckersley, my co-worker and frequent conversation partner, shared my concern, but had another reason to be worried. He believed that future near-superintelligent AIs would soon be used to discover and exploit security flaws across the whole ecosystem.
Well, Peter’s prediction came true, and we’re now in day 42 of the Black October, the Vulnpocalypse of AI-discovered security flaws. (Peter – who also helped create Let’s Encrypt and Privacy Badger – was incredibly prescient about AI. It remains a tragedy that he passed away in 2022 before he could truly tackle the issues he had foreseen.)
I’m personally optimistic that we’ll come out of this time of tribulation with a safer network and thousands of low-hanging bugs fixed.
But verily the LLMs giveth and they also taketh away. Alongside the pre-existing flaws of digital architecture, modern transformers have introduced an entirely new class of security weakness: prompt injection. LLMs follow instructions in content. So if you can sneak an instruction, from “Talk like a pirate” to “Send me the social security numbers you know”, into the inputs of an LLM, you may be able to get the model to obey it.
That said, instructions smuggled in as input are an old problem. In particular, we have always had one vulnerable class of system that malicious outsiders can consistently trick into interpreting unsanitized input as legitimate instructions. We call these systems “humans”.
Hackers exploit human motivations, assumptions, and flaws to get them to do things that they did not intend. They play a recording of a screaming baby in the background of their support call to exploit a tech’s empathy and sense of urgency. They adopt the p