SAGENT: An Intelligent System for the Management of Complex Workflows
- 1 College of Computing, Georgia Institute of Technology, Atlanta, GA, USA
- 2 College of Computing, Georgia Institute of Technology, Atlanta, GA, USA
Abstract
In today’s fast-paced, data-driven environments, many industries face challenges in handling time-consuming, complex human-machine tasks, such as chargebacks, dispute resolution, and insurance claim verifications. These tasks often involve multiple human-machine interactions and require the processing of unstructured multimedia data and nonstandardized man-machine interfaces, leading to inefficiencies, security risks, and delays. The absence of an integrated, autonomous system capable of managing such complex workflows hinders operational effectiveness. This paper presents SAGENT (System for Autonomous Graph-Enhanced Multimodal LLM AgeNTs), an innovative framework designed to automate and optimize these tasks using advanced multimodal language models (LLMs) combined with graph-based techniques. By leveraging the synergy of graph-enhanced data structures and AI-driven decision-making processes, SAGENT offers a solution to reduce human intervention and errors, accelerate resolution times, and improve overall task efficiency. We demonstrate the potential of this system in streamlining operations, enhancing accuracy, and maximizing ROI for industries reliant on the processing of complex, unstructured data.
- Zhu, Y.Q., Wang, X.H., Chen, J., Wang, S.F., et al . (2023) LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities. Zhejiang University, ZJU-Ant Group Joint Research Center for Knowledge Graphs, China, National University of Singapore and NUS-NCS Joint Lab. https://arxiv.org/abs/2305.13168v4
- Wang, Y., Lipka, N., Rossi, R.A., et al . (2023) Knowledge Graph Prompting for Multi-Document Question Answering. Vanderbilt University, Adobe Research, San Jose, USA. https://arxiv.org/abs/2308.11730v3
- Cui, J.X., Ning, M.N., Li, Z.J., et al . (2023) Chatlaw: A Multi-Agent Collaborative Legal Assistant with Knowledge Graph Enhanced Mixture-of-Experts Large Language Model. Peng Cheng Laboratory, Peking University. https://arxiv.org/abs/2306.16092v2
- Anokhin, P., Semenov, N., Sorokin, A., Evseev, D., et al . (2024) AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents. AIRI and London Institute for Mathematical Sciences. https://arxiv.org/abs/2407.04363v2
- Xie, J.L., Chen, Z.H., Zhang, R.F., et al . (2024) Large Multimodal Agents: A Survey. The Chinese University of Hong Kong, Shenzhen Research Institute of Big Data, Sun Yat-sen University. https://arxiv.org/abs/2402.15116
- Durante, Z., Huang, Q.Y., Wake, N., Gong, R., et al . (2025) Agent AI: Surveying the Horizons of Multimodal Interaction. Stanford University. https://arxiv.org/pdf/2401.03568v2
- Li, Z.J., Xie, Y.Q., Shao, R., Chen, G.W., et al . (2024) Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks. Harbin Institute of Technology, Shenzhen Peng Cheng Laboratory. https://arxiv.org/abs/2408.03615v2
- Sun, Q., Luo, Y.Y., Li, S.R., et al . (2024) OpenOmni: A Collaborative Open Source Tool for Building Future-Ready Multimodal Conversational Agents. University of Western Australia, Harbin Institute of Technology, Murdoch University. https://arxiv.org/abs/2408.03047v2