I find myself using AI quite frequently while hacking, yet my usage is still mostly asking ChatGPT or Copilot to explain something or, at best, generate code. While I don't believe that hacking bots will entirely replace humans soon, I do think that individuals...
AI
TL;DR: Every AI Talk from BSidesLV, Black Hat, and DEF CON 2024
Clint Gibler wrote a summary of each AI talk from BSidesLV, Black Hat, and DEF CON 2024 so if you want to quickly get up to speed and see what’s going on without spending hours on each talk, check out his TL;DR....
They Hacked Google A.I. for $50,000
Joseph "rez0" Thacker, Justin "Rhynorater" Gardner and I, Roni "Lupin" Carta collaborated on hacking Google Bard which resulted in $50,000 of bounties. They have interesting bugs as well like an IDOR that allowed you to describe someone’s...
How NOT to Train Your Hack Bot: Dos and Don’ts of Building Offensive GPTs
Will AI hacking agents replace us, hackers? This is the question I’ve been asked a few times already. It’s hard not to think about it. Certainly, I did. So this week, I watched a Black Hat talk about Offensive GPTs to see what was presented at Black Hat about the...
NIST’s document about AI security
NIST published a document that defines terminology in the new and growing industry of security around AI. I’m no authority in this industry but @rez0 is and he described the document as “the best AI Security Publication that he’s ever seen” and that’s a very strong...
Hacking Google Bard – From Prompt Injection to Data Exfiltration
For me, the moment that Google Bard got access to Gmail and Google Docs was the moment I stopped to see new bug classes like prompt injection or jailbreak escapes like some attacks of the future and I started to see them as having the real, severe impact here and now....
Who should worry about prompt injections?
Prompt injection is a very new subject in security. If you, like me, don’t have all the time to dedicate to it, it’s quite hard to grasp all the possible insecure scenarios. But luckily, rez0, probably the most active AI hacker, published Prompt Injection Primer for...
LLM OWASP TOP 10
I think OWASP TOP 10 lists are great resources for developers. They have a single resource that can give them sufficient amount of information to be at least somewhat aware of what risks are present. It’s also good when you are just getting familiar with a new area of...
AI Canaries
When I was creating the transcript of my latest video, I asked chatGPT to add some interpunction and change the capitalization of the text, without modifying the content. But in the middle of the text, chat stopped rewriting the transcript and started to explain to me...
The AI Attack Surface Map
AI is a new and emerging area and so is its cybersecurity. One of the very first comprehensive resources about potential attack vectors is this AI Attack surface map by Daniel Miessler. Use it as a starting point whenever you are auditing AI-based solutions....