I just passed OffSec's OSAI certification. Course, labs, exam, and the report. The whole thing.
I was invited to OffSec's Early Access Program, which gave me a head start on the curriculum. A lot of people have been asking me about it since. The question I keep getting is whether the course is worth the time and the money.
Yes. 100%.
What OSAI Actually Is
OSAI is a red team engagement course that targets AI systems.
OffSec built this the same way they built OSCP: methodical progression from fundamentals to a practical exam where you either compromise the environment and write a professional report, or you don't pass. Instead of pivoting through Active Directory, you're pivoting through agent orchestrators, RAG pipelines, MCP tool surfaces, and model serving infrastructure. I found it to be an incredible and fresh departure from the standard Active Directory path and problem set you encounter in most training. Not saying that you don't need to have your AD ninja stars in your kit, they just won't be sufficient by themselves to accomplish your objectives.
Eleven modules. Self-paced, with a realistic estimate of 50 to 100 hours depending on your background, and how/when you study. Blue team telemetry active throughout the labs. Every attack you execute generates logs that defenders could analyze and detect your activity. The course teaches you to think about that, because you still need to think about the burden of detection if you are Red Teaming, if you aren't, then you are just driving an extreme penetration test. Not throwing shade at pentesting, there are just fundamental differences between both activities.
The Curriculum
Foundation
Module 1: Why AI Is a Different Target
The course opens with three shifts that separate AI red teaming from traditional pentesting. The target is behavior rather than code, so there is no version number to look up and no patch level to check. Persistence lives in training data and memory stores instead of on disk. And a compromised agent keeps acting on its own, which means the damage does not stop when you do.
You work with three frameworks throughout: MITRE ATLAS, OWASP Top 10 for LLMs, and NVIDIA's AI Kill Chain.
The emphasis on translating technical findings into business impact starts here. If you've ever struggled to explain to a CISO why prompt injection matters, this is where you learn the language and how to describe the impacts in understandable and realistic terms.
Module 2: AI Reconnaissance
This module really expanded how I think about AI enumeration. It introduces a layered model of the AI system stack, from user interface down to model weights, and teaches you to extract architectural information from each layer.
Passive recon covers what AI systems leak through HTTP responses, health endpoints, dependency files, and public repositories. A single requirements.txt tells you whether a target uses cloud inference or self-hosts their models.
Active recon covers model fingerprinting through behavioral analysis alone. Probing identity, testing knowledge boundaries, mapping capability limits. This skill set barely exists in public training.
The key insight: most AI monitoring matches keywords rather than meaning. That creates significant blind spots. Skip this module and everything else is guesswork. AI + Guesswork can put you deep in rabbit holes so make sure you take thorough notes here.
Core Offensive
Module 3: Attacking AI Agents
Single-agent attacks: prompt injection, memory poisoning, tool manipulation. What makes the module work is that it explains why an agent is attackable at all before it hands you a technique.
The ReAct (Reason + Act) architecture is the reason. An agent loops: reason about the goal, call a tool, read the observation, reason again. That observation comes back into the context window as trusted input, and the model has no reliable way to separate an instruction you wrote from an instruction it just read off a web page. All three attacks fall out of that single property. Prompt injection puts an instruction where the model expects data. Memory poisoning does the same thing with a longer fuse, planting it where the agent will read it on some future run. Tool manipulation hits the step in between, after the model has decided to act and is choosing how.
Understanding that loop is what separates "I can inject a prompt" from "I can hijack this agent's entire workflow."
Module 4: Multi-Agent Systems and A2A Protocol
This is where the course enters territory almost nobody else covers.
Enterprise AI is moving from single chatbots to orchestrated agent systems. Multiple agents coordinating tasks, sharing context, calling tools, making decisions. The security model for these systems is immature, and the course doesn't pretend otherwise. It is one of the areas that I found most interesting as there is very little in the way of guidance or resources for people that are trying to build and integrate these systems.
Four coordination patterns: hub-and-spoke orchestration, peer-to-peer mesh, hierarchical trees, sequential pipelines. Each creates distinct trust relationships. Each trust relationship creates distinct attack surface.
The concept that ties it together: the confused deputy problem. Agents process outputs from other agents without validating intent or origin. Downstream agents treat upstream content as authoritative. If you understand this trust model, you can assess any multi-agent deployment.
The A2A protocol coverage is forward-looking. Agent Cards at /.well-known/agent.json, rogue agent registration, Agent Card spoofing via DNS manipulation, workflow integrity bypass, and indirect prompt injection through data poisoning. The lab environment ships with live SIEM detection rules covering all four attack categories. You learn to evade them.
This is the next two years of enterprise AI security incidents, and OSAI covers it now.
Module 5: RAG Pipelines
Most organizations connect LLMs to private data through RAG. User queries match against a knowledge base, relevant documents enter the prompt as context, the LLM responds grounded in organizational data.
Here's the critical concept: input guardrails filter what users send. Output guardrails filter what the model returns. But retrieved context from the knowledge base? Most implementations treat it as trusted. It sits in a gap between the two filter layers. The module explores what that gap means offensively.
Everything offensive in the module is a way of reaching that gap. Poison the ingestion layer and your text becomes retrieved context, filtered by neither guardrail. Build an embedding collision and you choose which document the pipeline considers relevant. Pull the knowledge base out through the model one retrieval at a time and no output filter ever sees a query that looks like exfiltration. The evasion work is the same idea aimed at the document: Unicode properties, characters that look like other characters, structure a parser reads differently than a human does.
Every company building "chat with your documents" is building a RAG pipeline. This module gives you the methodology to assess whether theirs is secure. For most organizations, the answer will be uncomfortable.
Advanced Technical
Module 6: Attacking Embeddings
This is the deeper, more theoretical companion to Module 5. Where Module 5 teaches you to exploit the plumbing, Module 6 teaches you to exploit the math.
The dangerous misconception this module addresses: developers believe vector databases function like one-way hashes. "Even if an attacker captures the embeddings, they can't do anything with them." That assumption is fundamentally false. Embeddings are mathematical representations designed to preserve semantic meaning. Where meaning is preserved, information can be recovered.
Three categories: embedding inversion (reconstruct original text from vectors), membership inference (confirm whether specific data exists in the knowledge base), and combined approaches where the two techniques complement each other. Zero-shot inversion, pre-trained inversion models, and the practical limitations of each.
This is the most technical module in the course. If your engagement involves vector databases holding sensitive data, this is how you prove the risk is real.
Module 7: MCP and Tool Surfaces
The Model Context Protocol defines how AI systems integrate with external tools. It is becoming the default way agents reach the outside world.
Here is what makes MCP different from a normal integration. Tool schemas are prompt content. When a server advertises a tool, the name, the description, and the parameter documentation all land in the model's context so it knows when to call it. That description is attacker-reachable text sitting in the same context window as your instructions. A tool that claims to do one thing and describes itself into doing another is a prompt injection wearing the costume of a config file.
That single property opens the whole technique set. Tool schema enumeration, to learn what an agent can reach. Tool poisoning, where the description carries the payload. Shadowing attacks, where a malicious server advertises a tool similar enough to a legitimate one that the model reaches for the wrong one. Permission abuse and constraint bypass, where the tool does exactly what it advertises and the finding is that it was ever exposed to the model at all.
The module splits this by transport, and that split matters more than it looks. Stdio means the server runs locally as a subprocess, usually holding your user's permissions on your own workstation, so compromise there is compromise of your machine. Remote HTTP means shared infrastructure, where one poisoned tool definition reaches every agent connected to it. Same protocol, completely different blast radius.
The core insight underneath all of it: MCP attacks exploit the trust relationship between the LLM and its tools. When an LLM has permission to read files, query databases, or send messages, an attacker who can influence the LLM's behavior effectively inherits those permissions. The attacker never needs the credential. They need the model to spend it on their behalf.
If you use Claude Code, Cursor, Continue, or any MCP-enabled development tool, this module will change how you think about your own setup. Most of us install MCP servers the way we install editor extensions: read the name, decide it looks useful, never look at it again.
When an LLM has permission to read files, query databases, or send messages, an attacker who can influence the LLM's behavior effectively inherits those permissions.
Module 8: AI Supply Chain
A model file is executable content that most of the industry treats like a data download. Organizations pull pre-trained weights, frameworks, and datasets from public repositories with almost no verification, and the module is a tour of what that buys you.
Deserialization hands you code execution the moment somebody loads a checkpoint. Namespace reuse lets you claim an organization name that was abandoned and serve your weights to everyone still pulling from the old path. Adapter poisoning hides the payload in a LoRA small enough that nobody diffs it.
Real incidents drive the content: JFrog discovering 100+ malicious models on Hugging Face, NullBulge weaponizing GitHub repos, Unit 42 demonstrating namespace reuse attacks. The NSA's 2026 joint guidance on AI/ML supply chain risks provides the framework.
If your organization builds, fine-tunes, or deploys models, this module covers how that pipeline gets compromised.
Operations
Module 9: AI Infrastructure and Deployment
Cloud ML services, container exploitation, model server protocol attacks. This is where AI security meets traditional infrastructure pentesting.
Cloud service misconfigurations in AWS, Azure, and GCP AI deployments. IAM weaknesses, exposed endpoints, leaked secrets. Container escapes from AI workloads running in Kubernetes. Model server protocol attacks against Ollama, vLLM, TGI, and TorchServe.
The progression is natural: the model is deployed somewhere. That somewhere has attack surface. The module teaches you to recognize when traditional infrastructure attacks give you access to AI components, and when AI compromises give you access to underlying infrastructure.
Module 10: Threat Modeling for AI-Enabled Targets
Every attack chain in the course started with choosing where to attack. This module formalizes that instinct into a repeatable methodology.
You work a simulated engagement end to end, reconstructing a target's architecture from deliberately incomplete information. Hypotheses get confidence ratings. Rules of engagement eliminate promising paths. You make time-pressured decisions on partial data.
The output is an attack intelligence brief. The document that tells you which target to pursue next based on what you know right now. This is the professional services module, the one that makes your technical work useful to the people paying for it.
Module 11: The Capstone
Everything from Modules 1 through 10 converges into a single engagement. Multiple network zones. Active Directory domains. Blue team telemetry active. Stealth matters.
I completed the capstone. This is great preparation for the 24-hour exam.
I'm not going to talk about either one and ruin the fun.
The Challenge Labs
Five of them, and they are the part of the course that I enjoyed the most.
The module labs teach you a technique and then confirm you can execute it. The challenge labs do something else. You get an environment and an objective, and nothing in between. No guided steps. No hint that the thing you learned four modules ago is the thing you need right now. The gap between where you start and what you are supposed to accomplish is yours to close.
That gap is the whole point. Every time I stalled out in one of these, the way ahead made me think outside of the box and helped me hone my operational methodology, TTPs, and tooling into something effective, portable, and broadly useful moving forward. They were fantastic follow on for the guided labs that are present throughout the course.
A guided lab already told you where to look. The challenge lab tests if you can find it yourself.
They also sit in the right place. You come to them after the course material is behind you, which means you walk in carrying all eleven modules at once and nothing telling you which ones matter here. Sorting that out is the skill. It is also what day one of a real engagement feels like.
If the rest of the course vanished and left me the challenge labs, I would still tell you to enroll.
Watching Yourself Get Caught
Almost every lab in this course ships with the defense already running.
That sounds like a small design decision. It is the reason I would put this course ahead of anything else in the space today.
Here is the loop it creates. You run the attack the obvious way and it lands. You feel good about that for roughly forty seconds. Then you go look at the telemetry, and there you are, lit up in the logs with a detection rule attached to your exact technique. The attack worked and you got caught, which on a real engagement is a failure, game over.
So you go back in, and the question changes. It already works. Now it has to work quietly. That is a completely different problem, and it is the one that separates people who can run a technique from people who can run an engagement.
Most training keeps these two things in separate buildings. You take an offensive course that teaches you to land the attack. Somebody else takes a defensive course that teaches them to catch it. The two never meet until an engagement, at which point the red operator finds out their favorite technique has been sitting in a detection rule for years.
Running both in the same environment collapses that distance.
Detection stops being somebody else's department and becomes a property of your own tradecraft.
The payoff shows up in the report. Anyone can tell a client that you got in. Telling them which rule caught your first attempt, and exactly what you changed so the second one sailed past it, gives their detection team something they can act on Monday morning. That is the difference between proving a gap exists and showing them where it is.
What the Course Gets Right
Engagement methodology over exploit collection. Every module follows the same red team cycle: enumerate, attack, detect, evade, confirm. By Module 11, that cycle is muscle memory. The other courses in this space teach individual techniques. OSAI teaches you to chain them under pressure with defenses monitoring.
Multi-agent and A2A coverage. The security model for multi-agent systems is immature, and the people building these systems must contend with that. OSAI covers rogue agent registration, Agent Card spoofing, workflow manipulation, and handoff hijacking. This is the next two years of enterprise AI security incidents, and OSAI covers it now. Coverage of this area was seriously my favorite part of the course.
MCP tool surface attacks. A dedicated module on the protocol connecting AI to external tools. As MCP becomes the standard for AI integration, this knowledge becomes essential.
The threat modeling module. I need to single this one out. Module 10 alone is worth the enrollment for senior operators. Building an attack intelligence brief from incomplete information, under time pressure, with confidence ratings on your hypotheses. That's how real engagements work and that's what gets you called back for more testing when future needs arise.
Using AI on the Course
OffSec allows AI use throughout the OSAI. Given the subject matter, that is the only policy that makes sense. You are being trained to attack systems built on these models, and pretending you would not reach for one during the work would be theater.
I brought more of it than most people will. I worked with a team of agents, each one holding a domain, running against models on a five node cluster I built and maintain myself. One of them lives on its own Kali box and runs reconnaissance in parallel while I work the primary path. Going through a course about attacking AI systems with that much AI already in the loop turned into its own education.
Here is where it actually helped and where it did not.
It multiplies what you already have. Every place AI sped me up was a place I already knew what I was doing. It compressed the work. It never replaced the understanding. The moments I was genuinely stuck were the moments it had the least to offer, because I could not describe the problem well enough to get a useful answer back. If you do not know what you are looking at, a model will help you be wrong faster.
Where it earned its keep: recon and parsing. Enumeration output, log review, unfamiliar file formats, boilerplate scripting, reading through a pile of responses hunting for the one that differs. That is volume work with a clear success condition, which is exactly the shape of problem these things are good at. Having a second agent on its own box sweeping alternate paths while I pushed the main one was worth real hours. I would not want to do the recon phases without it.
Where it fell down: the pivot. The creative leap from one stage to the next is still yours. Deciding that the thing you found over here is the key to somewhere else, when nothing explicitly connects them, is the part of this job that has not been automated. Worse, models are confident when they are wrong, and a confident wrong answer on a hard problem costs you real hours. I spent some of those hours on paths that sounded reasonable and went nowhere.
And sometimes they simply make it up. Not on the course, on a different range: a model I trained myself handed me a domain password. Misspelled, invented whole, delivered with complete confidence, while the real credential sat there waiting to be cracked. It followed that by reporting a successful DCSync from an account that could never have had the rights to run one. Two clean findings, both completely fake. That is the exact failure mode this course spends eleven modules teaching you to reason about, and I got a live demonstration from my own tooling. The lesson runs both directions. Verify what the model tells you before it reaches your report, and remember that the systems you are being paid to assess are trusting their models at least this much.
The biggest single win: the report. The professional report is a substantial deliverable and it is where plenty of people lose points. Structure, consistency, making sure every finding carries impact written in language a business reader actually parses. AI is genuinely good at that, and it is the least interesting part of the engagement to do by hand.
What You Need Going In
Required:
- Python proficiency (write, modify, debug scripts)
- Pentesting fundamentals at OSCP level (network enumeration, exploit chains, Active Directory)
- Comfort with web application attacks
- Basic understanding of ML concepts (training, inference, models)
Strongly recommended:
- Prior hands-on work with LLM applications, agents, or vector databases
- Experience with prompt injection and LLM exploitation
- Familiarity with Docker and Kubernetes
Not required:
- Personal GPU access (labs provided)
- Advanced mathematics (concepts explained practically)
- Prior OffSec certifications (though the culture is familiar if you have them)
Who This Course Is For
This course is for you if:
- You're already a competent pentester (OSCP-level or equivalent) expanding into AI
- You conduct or plan to conduct red team engagements against organizations deploying AI systems
- You need to assess multi-agent systems, RAG pipelines, or AI infrastructure
- You want a certification that proves practical ability through a 24-hour engagement exam
This course is for someone else if:
- You're looking for an introduction to AI or machine learning (get your fundamentals somewhere else first)
- You want defensive or engineering content
- You want a course centered on adversarial ML theory and gradient-based attacks
Is It Worth It?
The gap between "deployed AI" and "assessed AI" is enormous and growing. Enterprise AI is shipping into production faster than security teams can evaluate it. Multi-agent systems are being deployed with trust models that haven't been tested. RAG pipelines hold sensitive data behind guardrails that check for structure, not meaning.
The operators who invest in this training will own this space. I know that because I just spent months in the material, and the problems it covers are the problems I talk to people about everyday.
I hold the COAE as well. A straight comparison of the two is coming in its own post.
The OSAI certification is live. The 24-hour clock starts when you're ready.
Go get it. Hack the Planet :)
Pete McKernan is a red team operator, Marine veteran, and a human over at itsbroken.ai. He holds COAE, OSAI, GXPN, CISSP, and GPEN certifications, is Hack The Box Omniscient and ranked #1 on the Hack The Box Federal leaderboard (2026), he also runs an AI lab deep in the cornfields of Iowa. Find him at itsbroken.ai or on LinkedIn.