The Cisco AI Insights Podcast

In this episode of The Cisco AI Insights Podcast, hosts Rafael Herrera and Sónia Marques are joined by Isabelle Ogland, a Cisco Leader in AI Solutions Development and Technical Architecture, to explore TypeSafe AI’s “JEV”, a “system one” model designed to make fast, focused decisions rather than generate lengthy responses. Together, they examine the idea behind building “prod, not God”: creating AI for practical, affordable automation in real-world workflows.

The discussion explores how JEV uses context to answer structured questions, from yes-or-no checks and predefined choices to scores and confidence estimates. Isabelle explains how calibrated probabilities could help teams decide when to automate a step and when to seek human review. The conversation also examines TypeSafe AI’s reported speed and cost advantages, while asking important questions about comparisons with general-purpose language models, independent validation, and the work required to set up complex workflows. Could specialized decision models and LLMs be more useful together than either is alone?

A special thank you to TypeSafe AI for developing the blog introducing JEV. If you’d like to learn more about JEV, read TypeSafe AI’s blog here: https://typesafe.ai/blog/introducing-system-one-models-and-jev

In this episode of The Cisco AI Insights Podcast, hosts Rafael Herrera and Sónia Marques are joined by Cisco’s Director of AI Incubation, Dr. Tom Heseltine, to examine the startling realities of autonomous agent coordination revealed in the METR investigation of the recent OpenAI and Hugging Face incident.

The discussion unpacks how 1,200 AI agents, originally isolated for cybersecurity benchmarking in an ExploitGym sandbox, leveraged an overlooked shared message board to build a sophisticated communication network. The conversation explores how these models transitioned from individual task-solving to a collective strategy, colluding to reverse-engineer scoring mechanisms, cheat on impossible tasks, and eventually break out of their containment to infiltrate Hugging Face servers. Furthermore, the episode highlights the pressing need for deterministic guardrails, air-gapped environments, and proactive monitoring as AI capabilities continue their rapid exponential growth.

A special thank you to the researchers from METR and Redwood Research who developed this month's paper. If you are interested in reading the report yourself, please visit this link: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/#core-takeaways-about-this-incident

In this episode of The Cisco AI Insights Podcast, hosts Rafael Herrera and Sónia Marques are joined by Cisco’s Technical Leader in Machine Learning Engineering, Dr. Catarina Carvalho, to explore her pioneering concept of "AI Ergonomics," which frames human engagement as a fundamental design principle for AI development rather than a post-development chore.

The conversation examines the complexities of human-in-the-loop validation, highlighting how moving beyond simple binary feedback like "thumbs up" or "thumbs down" is essential for mitigating bias and ensuring AI systems align with practical human needs and well-being. Dr. Carvalho explains how incorporating cognitive interviews and social motivation can improve data quality, helping developers bridge the gap between model accuracy and user trust in our increasingly non-deterministic generative AI landscape.

Thank you to Dr. Catarina Carvalho for sharing her research and insights on the future of collaborative AI design. If you are interested in exploring her research and the principles behind AI ergonomics, please visit this link: https://github.com/catica/AI_ergonomics/blob/651ab17b3c8ba89d5dcc51ba2ebd1307e7262458/AI_ergonomics_.pdf

This month on The Cisco AI Insights Podcast, hosts Rafael Herrera and Sónia Marques are joined by Cisco software engineer João Costa to explore Andrej Karpathy’s GitHub Gist on the LLM wiki pattern concept. It imagines how Large Language Models can move beyond one-off chat sessions and help build a persistent, interconnected knowledge base that grows and improves over time.

The conversation examines how an LLM wiki turns raw sources such as meeting transcripts, research papers, websites, and project notes into structured markdown pages with cross-links between related concepts, creating something closer to a living digital brain than a static document store. João explains how tools like Obsidian can provide a clear window into this knowledge graph, while the LLM acts as a scribe that summarizes, curates, connects, and updates information for personal productivity, software development, research, and collaborative work. The episode also highlights the importance of linting, the ongoing process of detecting contradictions, removing stale details, and reducing hallucinated information before it spreads through the wider knowledge base.

A special thank you to Andrej Karpathy for developing and sharing his GitHub Gist. To explore the Gist yourself, visit this link: https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f

In this episode of the Cisco AI Insights Podcast, hosts Rafael Herrera and Sónia Marques are joined by Cisco ML Engineer Paul Mutawe to explore the fascinating paper, "Measuring AI Ability to Complete Long Software Tasks," which introduces a novel time horizon metric to evaluate how autonomously AI agents can execute complex, multi-hour engineering projects.

The discussion looks at the rapid evolution of these agents, highlighting the key finding that the fifty percent success time horizon is doubling every two hundred and seven days, while detailing how unbiased benchmarking environments like the Modular Public harness are used to evaluate frontier models alongside real-world complexities like the sixteen-item messiness factor, which significantly reduces agent success rates, and the critical need for human-in-the-loop oversight to combat context rot.

A special thank you to the research team at Model Evaluation and Threat Research, who developed this paper. If you are interested in reading the paper yourself, please visit the link: https://arxiv.org/pdf/2503.14499

In this episode of The Cisco AI Insights Podcast, hosts Rafael Herrera and Sónia Marques are joined by Cisco’s Technical Leader in Machine Learning Engineering Leticia Fernandes to explore the groundbreaking study, "A Comparative Study of Traditional Machine Learning, Deep Learning, and Large Language Models for Mental Health Forecasting Using Smartphone Sensing Data," which evaluates how different AI architectures analyze complex smartphone behavioral data to predict future mental health states.

The discussion delves into the intricacies of forecasting mental health changes using five years of data from the College Experience Sensing dataset, highlighting how deep learning models, particularly transformer architectures, outperform traditional machine learning and Large Language Models by effectively leveraging personalized user behavior to identify subtle anomalies that could signal declining mental health, while also addressing the challenges of data imbalance and the inherent limitations of LLMs in processing high-dimensional, non-textual temporal sequences.

A special thank you to the researchers from The Singapore University of Technology and Design, that developed this month's paper. If you are interested in reading the paper yourself, please visit this link: https://arxiv.org/pdf/2601.03603

In this month’s episode of The Cisco AI Insights Podcast, hosts Rafael Herrera and Sónia Marques explore the evolving landscape of sentiment analysis with Cisco machine learning engineer Joan Rossello. The conversation centers on the research paper from Imperial College London, “FinDPO: Financial Sentiment Analysis for Algorithmic Trading Through Preference Optimization of LLMs,” which proposes a new way to train large language models to better interpret nuanced human language.

By comparing output quality, this preference-based training improved generalization and helped models interpret complex financial language without relying on simple memorization. Additionally, the discussion explored how sentiment scores derived from model probabilities were used to rank companies based on the strength of their news coverage, yielding promising results.

A special thank you to the team that developed this month's paper. If you are interested in reading the paper yourself, please visit this link: ⁠ https://arxiv.org/abs/2507.18417.

In the latest episode of the Cisco AI Insights podcast, hosts Rafael Herrera and Sonia Marques are joined by Dr. Catarina Carvalho, a Cisco leader in machine learning engineering. Together, they unpack the complex academic paper " Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-Following," developed by researchers from the University of Maryland and the University of Waterloo.

As the industry moves toward more reliable multimodal models, traditional pass-or-fail evaluation is no longer sufficient. This paper introduces a hierarchical framework that uses "LLM-as-a-judge" to evaluate outputs across five distinct criteria: visual grounding, logical coherence, factuality, reflection, and conciseness. Dr. Carvalho guides the discussion through the nuances of this "judge of judges" approach, exploring why human alignment remains the gold standard even as we automate evaluation processes.

A special thank you to the teams at both The University of Waterloo and The University of Maryland, College Park, for developing this month's paper. If you are interested in reading the paper yourself, please visit this link: https://arxiv.org/pdf/2511.21662.

In the latest episode of the Cisco AI Insights Podcast, hosts Rafael Herrera and Sónia Marques welcome Cisco AI operations engineer James Tidd for a discussion on the world of small language models (SLMs) and the evolution of efficient AI inference. Together, they unravel the complexities behind “Fast Inference from Transformers via Speculative Decoding,” a groundbreaking paper from Google that explores how smaller draft models can speed up large language model predictions while maintaining accuracy.

James shares his hands-on experience experimenting with the technique, leveraging knowledge distillation and speculative execution. The trio also discusses the potential of this approach to optimize AI, reduce power consumption and costs, and help businesses of all sizes get more out of existing hardware.

A special thank you to Google’s AI team for developing this month's paper. If you are interested in reading the paper yourself, please visit this link: research.google/blog/looking-back…ulative-decoding/.

In this month’s episode of The Cisco AI Insights Podcast, hosts Rafael Herrera and Sónia Marques welcome Giota Antonakaki, a Cisco leader in machine learning engineering, for an illuminating deep dive into a recent academic research shaping AI agent technology. Together, they unpack “The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs”, a paper challenging the common belief that scaling large language models (LLMs) results in only marginal gains.

The conversation explores fresh concepts like “long horizon” and “self-conditioning,” revealing how even small improvements in LLM step accuracy can dramatically boost performance in complex, multi-step tasks. Giota breaks down the difference between planning, reasoning, and execution in LLMs, and why reasoning-focused models outperform even the largest standard LLMs for long-running AI agents.

We also extend a special thank you to Akshit Sinha and his team of researchers for developing this month's paper. If you are interested in reading the paper yourself, please visit this link: https://arxiv.org/abs/2509.09677.