STIXAgent—A Multi-Agent Framework for Standardized Management of Cyber Threat Intelligence (CTI) Reports

Abstract

As the volume of Cyber Threat Intelligence (CTI) reports from multiple sources continues to rise, automated tools are essential for consistent and accurate management of heterogeneous data. Existing methods, such as STIXnet and aCTIon, require substantial human intervention, limiting their scalability. This paper introduces STIXAgent, a multi-agent framework that automates the conversion of unstructured CTI reports into STIX-compliant JSON structures. Our approach leverages LangGraph to orchestrate modular task execution, incorporating Large Language Models (LLMs) for information extraction, structured validation, and error handling. This paper utilises 5 LLMs (GPT4o, Gemini, Llama3 (8B), DeepseekR1-distilled Qwen32B, and LLama 70B) for multiagentic LLM evaluation, introducing in the process a novel Bayesian statistical evaluation framework to assess model performance, offering probabilistic insights into content accuracy and structural consistency. We benchmark STIXAgent across 10, 40, 100, and 400 reports from Microsoft and Talos, respectively. The results demonstrate that STIXAgent not only enhances automation but also improves reliability through structured evaluation, setting a new standard for scalable CTI processing.