Engagement Type: Enterprise Generative AI Demonstration/Proof of Concept
Focus: Secure Retrieval-Augmented Generation (RAG) Architecture, Safety Controls, and Data Readiness
Industry: B2B Travel Media and Information Services
Outcome: Validated approach for responsible AI adoption
Client Overview
The client is a leading business-to-business organization operating in a content- and data-intensive industry, providing information services, research, events, and insights to a global professional audience.
As a data-rich organization, the client manages large volumes of structured and semi-structured information. This made the organization a strong candidate for exploring generative AI, while also requiring a thoughtful, governed approach to ensure accuracy, trust, and operational control.
Keyhole Software partnered with the client to design and implement an enterprise-grade generative AI proof of concept that allowed internal teams to safely evaluate large language models in a real-world, governed environment.
The Challenge: Secure AI Access to Complex Enterprise Data
Our client wanted to explore how large language models (LLMs) could be used to improve access to internal knowledge and complex datasets through natural-language interaction. However, adopting off-the-shelf AI tools introduced meaningful risks.
At the core of this initiative was a growing challenge around knowledge accessibility. Critical information lived across complex, distributed datasets, including structured sources like event metadata and editorial databases, as well as semi-structured content. Locating and interpreting this information often required significant manual effort and deep technical expertise, slowing research and insight generation.
The client’s reliance on complex XML-based data structures further compounded the problem. While powerful, these structures made data retrieval time-consuming and created friction for non-technical users, increasing dependency on specialized resources to answer everyday business questions.
At the same time, the client had a high risk tolerance threshold:
- Data complexity: Standard LLMs struggle to interpret structured and semi-structured data accurately, increasing the likelihood of incomplete or misleading outputs.
- Risk profile: Off-the-shelf AI tools introduced concerns around hallucinations, unreliable responses, and lack of traceability. These are unacceptable risks for an enterprise that depends on trustworthy information.
- Governance & compliance: As a leader in global travel media and conferences, any AI initiative needed to align with internal security practices and provide clear auditability.
Rather than building a superficial prototype, the client needed to validate generative AI in a real enterprise context, one that balanced innovation with governance, security, and confidence in the outputs. This foundational, enterprise-grade generative AI architecture would not only address immediate knowledge-access challenges but also serve as a scalable platform for future AI-driven use cases.
Before and After: Enterprise Knowledge Access with RAG
The snapshot below highlights how a governed RAG architecture changed the way teams accessed and trusted enterprise knowledge.
Before
- Knowledge distributed across complex, XML-heavy datasets
- Manual effort required to locate and interpret information
- High dependence on technical expertise for data access
- Limited confidence in off-the-shelf AI due to hallucinations and lack of traceability
After
- Natural-language access to enterprise data using a governed RAG architecture
- Faster retrieval grounded in authoritative internal sources
- Reduced friction for non-technical users querying complex datasets
- Secure, auditable AI responses aligned with enterprise governance standards
The Approach: Enterprise-Grade RAG Architecture
Keyhole Software partnered closely with the client’s technical and business stakeholders to design and implement an enterprise-grade generative AI proof of concept using a Retrieval-Augmented Generation (RAG) architecture. The solution was designed to provide natural-language access to enterprise data while preserving strict governance, security, and auditability.
The engagement was intentionally framed as a production-quality proof of concept. Rather than validating AI capabilities in isolation, the goal was to establish a safe, scalable architectural pattern that could withstand real enterprise constraints and serve as a foundation for the client’s future AI initiatives.
Architectural Strategy & Design Decisions
Several architectural approaches were evaluated, including fine-tuning large language models. Ultimately, RAG was selected due to the nature of the client’s data and the need for accuracy and traceability.
Much of the client’s internal knowledge lived in complex, XML-heavy datasets. While difficult to query directly, this data was semantically rich. By ingesting and decomposing XML content into meaningful chunks aligned with real-world travel concepts (such as property descriptions, amenities, and location context), the system could retrieve information in a way that closely matched how users naturally ask questions. This made RAG particularly effective, enabling accurate responses grounded in authoritative internal data without requiring model retraining.
The architecture also reflected internal technical standards and constraints:
- Grounded responses: Guardrails ensured that outputs were strictly based on the client’s data, naturally guiding users toward trusted internal products and content.
- Cost awareness: Usage logging and monitoring were built in to track interaction volume and estimate cost per query, giving stakeholders visibility into how adoption translated into operational expense.
- Security and governance: Retrieval boundaries, metadata filtering, and audit logging ensured compliance with internal security practices.
Core Implementation Components
Key elements of the solution included:
- Enterprise RAG architecture: Natural-language interaction with large, complex datasets by grounding LLM responses in structured and XML-derived internal sources.
- End-to-end AI data pipelines: Ingestion workflows, intelligent chunking strategies, embedding generation, and vector-enabled databases to support high-accuracy semantic retrieval at scale.
- Secure LLM integration: API layers that enforced retrieval constraints and auditability while safely integrating LLMs with internal data.
- Safety, reliability, and observability controls: Token limits, retrieval caps, deterministic prompting, and traceability to reduce hallucinations and produce explainable outputs.
- Collaborative evaluation and tradeoff analysis: Hands-on work with internal teams to assess data readiness, validate retrieval quality, and balance innovation with operational risk.
Implementation, Cost, and Infrastructure Tradeoffs
Data-layer performance emerged as one of the most important architectural tradeoffs. The initial proof of concept ran on a lower-cost Supabase Postgres environment, which was sufficient for early validation. However, as large volumes of XML-derived content were ingested, database growth led to memory pressure and slower query performance.
Because RAG systems rely heavily on fast, reliable retrieval, these constraints began to affect response time and answer quality. To address this, the team migrated the data layer to Google Cloud’s managed Postgres (Cloud SQL), allowing for increased RAM allocation and improved query performance. While this introduced a higher baseline infrastructure cost, it significantly improved system stability and retrieval speed.
To balance costs, the API layer, frontend, and model usage were intentionally kept efficient. This approach allowed the system to deliver higher-quality responses without relying on larger, more expensive model calls to compensate for slow retrieval.
Operational Readiness & Client Ownership
Beyond technical functionality, Keyhole ensured the platform was structured for long-term client ownership and operational stability. Environments, workflows, and repositories were designed for clarity, maintainability, and secure handoff, including:
- Verified access and ownership for cloud resources (including Microsoft Azure, GCP, and Supabase)
- Containerized build, deploy, and rollback workflows using Docker and CI/CD pipelines
- Secure secrets management with environment values and API keys stored outside repositories
- Database readiness, including Pgvector RAG tables and sample ingestion
- Operational workflows for health checks, logging, backup, and restore
- Clear documentation for build, run, and test procedures
Together, these measures ensured the demonstration met enterprise standards for security, governance, and operational maturity, giving the client confidence to explore generative AI safely, responsibly, and at scale.
The Outcome
Through this engagement, the client validated a secure and scalable foundation for generative AI adoption. The demonstration showed that internal teams could interact with enterprise knowledge using natural language while maintaining confidence in the accuracy, traceability, and governance of AI-generated responses.
Rather than producing a one-off demo, the work resulted in:
- Validated architectural pattern for responsible AI use: The engagement established and validated a Retrieval-Augmented Generation (RAG) architecture with clear guardrails around data access, model behavior, and output traceability.
- Improved access to complex enterprise data: Semantic search and RAG capabilities enabled more efficient natural-language querying of structured and semi-structured data compared to traditional keyword-based approaches.
- Risk-aware foundation for future AI initiatives: Built-in safety controls and governance aligned with enterprise requirements, giving the client confidence to extend generative AI capabilities responsibly beyond the initial demonstration.
By focusing on validation instead of rapid deployment, our client now has a defensible approach for expanding AI capabilities in a way that aligns with enterprise standards and long-term goals.
Why Keyhole Software
This engagement highlights Keyhole Software’s ability to guide organizations through responsible, practical AI exploration. Keyhole focused on delivering a practical enterprise generative AI proof of concept that balanced innovation with real-world constraints around security, governance, and reliability.
By combining deep expertise in generative AI and RAG with hands-on collaboration and disciplined architectural thinking, Keyhole helped this client move beyond curiosity and toward informed, confident decision-making around AI adoption. For more information, see Keyhole’s Retrieval-Augmented Generation services.
