The challenge
Today's SOC industry is highly manual: analysts triage thousands of alerts a day, response time is measured in hours, and coverage scales linearly with headcount. This security analytics platform's vision is to make the SOC autonomous: AI agents that detect threats from user and system behavior on the endpoint, take dynamic local action, and feed learning back to a continuously retraining cloud model, without keeping a human in every loop. The customer needed an AWS partner who could deliver across the endpoint agent (C++, edge ML), the ML detection pipeline (training + retraining), and a production-grade fault-tolerant cloud architecture.
What we built
A next-generation SOC platform on AWS across seven architectural layers, endpoint to SOC dashboard to retraining loop. A custom C++ endpoint agent (Wazuh-base, headless plugin architecture, Lua + CLIPS + ONNX) with File Integrity Monitoring shipped to production-ready status (v1.0.0, Feb 2025). A supervised XGBoost ML detection pipeline trained on 3.85M MITRE-labeled events. A 7-layer cloud pipeline: Amazon API Gateway to Amazon SQS to AWS Lambda + Amazon ECS Fargate (overflow) to Amazon Security Lake (OCSF Parquet, Athena-queryable) + Amazon Security Hub (ASFF findings, SOC dashboard) to Amazon EventBridge routing to Slack / PagerDuty / Jira to an Amazon SageMaker retraining loop that closes the cycle from SOC analyst verdict back to agent.
Why this matters
The ML detection model lands at 98.23% accuracy, 99.61% ROC-AUC, 99.58% PR-AUC, 98.45% precision, and 98.01% recall on the held-out test set, trained across 36 MITRE techniques. The two-tier transform design (Lambda for normal load, ECS Fargate for burst overflow on SQS queue depth) keeps the cloud pipeline cost-efficient at steady state and elastic under spikes. The headless plugin agent architecture lets detection logic update daily without re-shipping the agent binary. And the closed-loop SageMaker retraining cycle turns every SOC verdict into ground truth for the next model, so the agent gets smarter every release.