AI Pentesting Agents Find Vulnerabilities Humans Missed—But Still Have Critical Blind Spots

AI security agents are increasingly competitive with human penetration testers, discovering complex vulnerabilities that have evaded expert teams for years—yet they remain far from autonomous independent testing. Recent benchmark research shows that ARTEMIS, a sophisticated AI agent, detected nine valid vulnerabilities with 82% accuracy and outperformed nine out of ten OSCP-certified penetration testers on a real university network with approximately 8,000 hosts. The agent operates as a multi-layered system: a supervisor agent orchestrates workflow, specialized sub-agents conduct parallel reconnaissance and exploitation, and a triage module validates findings and minimizes false positives.