The Stepwise rEasoning Error Disruption (SEED) attack is a method designed to mislead large language models (LLMs) by disrupting their step-by-step reasoning process. Instead of changing the original instructions, SEED injects subtle errors into specific reasoning steps, causing LLMs to produce incorrect but plausible answers that are difficult to detect. (Join the AI Security group at https://www.linkedin.com/groups/14545517 for more similar content) 🛠 What is SEED and How Does it Work? SEED targets reasoning chains (like Chain-of-Thought prompting) by introducing errors at key stages: 1️⃣ Subtle Errors are injected into the reasoning flow. 2️⃣ These errors propagate step by step, misleading the model into an incorrect outcome. 3️⃣ The process appears coherent and natural, making it hard to detect. 🔄 How Attackers Target SEED ⚙️ SEED-S (Step Modification) Modifies a specific reasoning step (e.g., a calculation or logic jump). Disruption is small but impactful, causing logical drift. 📋 SEED-P (Problem Modification) Slightly alters the original problem to create misleading reasoning. Generates a logical chain that ends with an incorrect but coherent solution. 🎯 Where Do Attackers Inject Errors? Attackers target specific stages of the LLM reasoning process: 1️⃣ Early Steps: Errors at the foundation ripple through subsequent steps. 2️⃣ Intermediate Steps: Logical misdirection happens during reasoning transitions. 3️⃣ Final Steps: Subtle changes mislead the final outcome while retaining logical flow. 📊 What Are the Results of SEED Attacks? Researchers tested SEED across 4 datasets (MATH, GSM8K, MATHQA, CSQA) and 4 major LLMs (GPT-4o, Llama3, Qwen, Mistral). ✅ Attack Success Rates (ASR): Up to 80%. 📉 Significant accuracy drops across all models. 👀 Stealth: SEED outperforms existing attacks in effectiveness and difficulty of detection. ⚠️ Why Does This Matter? 🕵️ Hard-to-Detect Errors: Disruptions appear natural and coherent. 📂 Data Poisoning Risk: Continuous learning systems may absorb faulty reasoning data, worsening performance. 🔑 High-Stakes Applications: Domains requiring precise reasoning, such as healthcare, finance, or law, are particularly at risk. 📎 Study Reference: Stepwise Reasoning Error Disruption Attack of LLMs https://arxiv.org/html/2412.11934v1 Thank you, Pascal Biese for sharing this interesting study 🙏 #AISecurity #Cybersecurity #AITrust #AIRegulation #AIRisk #AISafety #LLMSecurity #ResponsibleAI #DataProtection #AIGovernance #AIGP #SecureAI #AIAttacks #AICompliance #AIAttackSurface #AICybersecurity #EthicalAI #CISO #AdversarialAI #AIThreats #AIHacking #MaliciousAI #OffensiveAI #AIGuardrails #AIResearch #ISO42001 SEED Attack: Subtly Disrupting LLM Reasoning Step by Step was originally published in InfoSec Write-ups on Medium, where people are continuing the conversation by highlighting and responding to this story.

Japanese and U.S. authorities attributed the theft of $308 million cryptocurrency from DMM Bitcoin to North Korean cyber actors.

Contractors working on Google Gemini are comparing its responses to Claude's, according to internal correspondence seen by TechCrunch. © 2024 TechCrunch. All rights reserved. For personal use only.

Junior Barros De Oliveira, Brazil, has been indicted in the United States for orchestrating an extortion scheme involving data stolen.

According to recent reports, Bluesky's rapid growth has caught the attention of Meta CEO Mark Zuckerberg, prompting swift updates to Threads in an effort to...

AI investment is massive, but AI profits are not. Here’s why investors are still optimistic.

IDA Pro plugin with a rich set of features: decryption, deobfuscation, patching, lib code recognition and various pseudocode transformations - KasperskyLab/hrtng

A new Linux backdoor called 'WolfsBane' has been discovered, believed to be a port of Windows malware used by the Chinese 'Gelsemium' hacking group.

Life is good when you can lab from anywhere. Cisco Modeling Labs Free makes it fun to design, test, troubleshoot, and learn with the Cisco premier platform for network simulations. It's the perfect to...

From the chipmaker’s Q3 2025 earnings.

Uni sysadmin who ran the lab he erased was a big part of the problem Who, Me?  Another Monday and what a fine one it is here in the lair of Who, Me? – the reader contributed column in which your fellow Reg-admirers admit to the moments they messed up…

At least six L.A. County sheriff's deputies have been relieved of duty amid an investigation into their work for a 24-year-old cryptocurrency entrepreneur accused of extortion and hiding millions of d...