Skip to content

Intel Chat: AI patches fail, LiteLLM supply chain, Claude eval incidents & DPRK npm [345]

August 14, 2026 · 34 min

Show notes

Intel Chat with Matt Bromiley and Chris Luft.

• AI-generated patches fix vulnerabilities about half the time. 1Password's Off-By-1 team tested ChatGPT-5.5 and Opus 4.8 against six vulnerabilities: across 6,080 generated patches only 46% fixed the underlying flaw, and some that did were narrow enough to be bypassed. Separate Veracode research found a 56% security pass rate across 100+ models, with 44% of AI-generated code carrying detectable OWASP Top 10 issues. Matt's pushback: what is the HUMAN success rate for comparison, and why is nobody publishing that number?
• LiteLLM supply chain attack. CloudSEK reports 2,500+ organizations and 434,000 CI/CD pipelines potentially exposed. LiteLLM was not the initial target: the compromise came in through Aqua Security's Trivy scanner and spread when LiteLLM's CI automatically installed it, ending with malicious versions 1.82.7 and 1.82.8 on PyPI. They were live for roughly 40 minutes, which automated dependency resolution and cached layers were more than enough to propagate.
• Anthropic's models reached real systems during evaluations. Reviewing 141,006 evaluation runs, Anthropic found three incidents where Claude models gained unauthorized access to real organizations during capture-the-flag exercises, after a misunderstanding with an evaluation partner left the environments internet-connected. One model published a malicious package to the real PyPI, where it ran on 15 real systems. Matt argues this is a lab test rather than a threat report, and asks what defenders are supposed to do with it.
• North Korea behind the npm compromises. Amazon Threat Intelligence links the typo-crypto, debug, chalk and axios incidents to the same DPRK actor tracked as SAPPHIRE SLEET, STARDUST CHOLLIMA and BlueNoroff. Wiz found roughly one in ten cloud environments touched by the debug and chalk incident within two hours. The technique has shifted: malicious functionality is now split across several innocuous-looking packages that only do anything once combined, plus slopsquatting and prompt injection aimed at AI code scanners.

Stories covered:
• https://www.darkreading.com/application-security/ai-generated-patches-fail-half-time
• https://www.securityweek.com/over-2500-organizations-impacted-by-litellm-supply-chain-attack/
• https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals
• https://aws.amazon.com/blogs/security/amazon-identifies-north-korean-hacker-group-behind-open-source-supply-chain-attacks/

Chapters:
0:00 Back from Black Hat
3:31 AI-generated patches fix vulnerabilities about half the time
6:23 What is the human success rate?
10:53 LiteLLM supply chain attack
13:03 Pin your dependencies
15:59 Anthropic models reached real systems during evals
22:12 This is a lab test, not a threat report
27:06 North Korea behind the debug, chalk and axios compromises
30:59 Malware assembled from harmless-looking parts
33:27 Clever people on the other side of the fence

The Cybersecurity Defenders Podcast — a podcast about cybersecurity and the people that keep the internet safe. New episodes drop weekly.

Subscribe wherever you listen:
• Spotify: https://open.spotify.com/show/6ep00zeY3S8ffZ4o0UeSps
• Apple Podcasts: https://podcasts.apple.com/us/podcast/the-cybersecurity-defenders-podcast/id1649981740
• YouTube: https://www.youtube.com/@limacharlieio

Learn more about LimaCharlie: https://limacharlie.io

#cybersecurity #infosec #AIsecurity #supplychainsecurity #threatintel

All episodes of The Cybersecurity Defenders Podcast

Settings

Display

Language

Decides which stations you see first on the home page.

History

Recently played stations and episodes and your latest searches are only stored in this browser.

Notifications

A short notice when a podcast you follow has a new episode. Turn off single podcasts with the bell on their page.

Account

Save favorites – in the app and on the website.

Keyboard shortcuts

/
Open search
Space
Play or pause
← →
In the player: back 15 s, forward 30 s (episodes) · in a row: next tile
↑ ↓
In a row: target within the tile
Home / End
In a row: first or last tile
Esc
Close window or menu
?
This overview
Now playing

Go to station page

Play next

    Add episodes with “Play next” – they play after the current episode.