AI Selection in Organizations: A Decision Framework for Large Language Model and Generative AI Deployments

Abstract

Artificial intelligence selection has emerged as one of the most impactful decision areas facing modern organizations. As AI adoption has accelerated from experimental use to broad enterprise deployment, the central challenge is no longer whether organizations should adopt AI, but how leaders and technology professionals can make informed, defensible decisions about which AI systems to select, how to govern them, and where they can create sustainable organizational value. The urgency of this challenge is heightened by rapid vendor proliferation, uneven executive technical literacy, growing regulatory expectations, and the increasing operational risks associated with poorly matched AI solutions, including wasted investment, vendor lock-in, unreliable outputs, and uncontrolled shadow AI adoption. This paper addresses that problem by providing a practical and conceptually grounded framework for AI selection in organizations. It first explains how current large language models (LLMs) work, including transformer architecture, training, alignment, fine-tuning, and key limitations, while translating these concepts into plain language for executive audiences. It then examines why technical literacy has become essential for organizational decision-makers responsible for AI strategy, procurement, and governance. Building on that foundation, the paper presents a structured selection framework that evaluates AI systems across eight dimensions: capability assessment, alignment and safety, integration fit, data governance, total cost of ownership, vendor risk and lock-in, governance and compliance, and lifecycle management, aligned with the NIST AI Risk Management Framework (AI RMF 1.0) and ISO/IEC 42001:2023. The paper further introduces an executive decision model for the build, configure, or buy choice, and concludes with strategies for preventing extreme AI proliferation through governance controls, technical safeguards, and workforce education. The contribution of this paper is both strategic and practical. For organizational leaders, it offers a clearer basis for aligning AI decisions with business goals, risk tolerance, compliance obligations, and long-term operating models. For technology professionals, it provides a structured approach to evaluating AI capabilities, deployment options, and lifecycle implications in a way that supports sound architecture and governance decisions. By integrating academic literature with current industry evidence, this paper equips decision-makers with a more informed foundation for selecting AI systems that are not only innovative, but operationally viable, governable, and strategically appropriate. Objective : This paper develops and demonstrates a structured, repeatable framework that enables the senior technology executive accountable for AI strategy (principally the CIO or CTO, supported by enterprise architects and a cross-functional selection team) to make defensible decisions about which AI systems to select and whether to build, configure, or buy them. Method : The study follows a design-science approach. The framework is synthesized from the NIST AI Risk Management Framework (AI RMF 1.0), ISO/IEC 42001:2023, and the peer-reviewed and industry literature, and is then validated on two complementary, panel-free legs: a dimension-to-standard mapping that establishes coverage of the governing standards, and a retrospective comparative analysis of six documented enterprise AI deployments. Results : Across six documented enterprise deployments the framework gates align with the recorded outcome: each failure violated a gate the framework evaluates early (accountability and verification in two public-facing chatbots, capability on real operating data in an automated drive-thru, and capability, integration, and data readiness in a $62-million healthcare build), while a disciplined success cleared the gates in sequence and a partial reversal corrected a lifecycle gate it had under-weighted. Measurable success criteria for the framework (decision auditability, avoidance of post-proof-of-concept abandonment, total-cost-of-ownership forecast accuracy, and reduction of vendor lock-in exposure) are defined. Conclusion : For organizational leaders, the framework offers a defensible basis for aligning AI decisions with business goals, risk tolerance, and compliance obligations; for technology professionals, it provides a sequenced method for evaluating capability, governance, sourcing, and lifecycle fit. Expanding the case set and prospective field application are identified as the primary directions for future work.