Answer accuracy across about 1,200 test questions
OpenSearch domain in place of roughly 9,000 planned collections
Suppliers supported, up from 3,000, with no architecture changes
Halo AI Inc.
Novato, CA
Technology / AI-Powered Supplier Intelligence
Amazon Bedrock, Amazon OpenSearch Service, AWS Fargate, AWS Lambda, Amazon API Gateway, Amazon CloudWatch, AWS CloudFormation, Amazon VPC
Halo AI, a technology company serving enterprise customers with AI-powered supplier intelligence, needed to scale its “Universal Ask Halo” knowledge retrieval system from a 10-supplier testing scope to support thousands of suppliers across multiple subscription tiers. The original architecture, which used separate Amazon OpenSearch Serverless collections for each supplier and tier combination, would have required approximately 9,000 collections at full scale, exceeding AWS service limits. Avahi re-architected the solution using a single provisioned Amazon OpenSearch Service domain with metadata-based tenant isolation, hybrid search capabilities combining semantic and keyword retrieval, and a Fargate-based authorization gateway. The new design scales with data volume rather than collection count, eliminates the knowledge base ceiling, and improves retrieval quality for exact-term queries.
Halo AI Inc., headquartered in Novato, California, provides AI-powered supplier intelligence solutions to enterprise customers. Their platform enables organizations to monitor and analyze information across extensive supplier and vendor portfolios, with customers tracking up to 3,000 suppliers currently and projecting growth to approximately 10,000 suppliers. Halo AI offers tiered subscription levels that provide varying depths of supplier intelligence, requiring strict data isolation between customers and access tiers.
Halo AI’s initial “Universal Ask Halo” system was built on Amazon Bedrock with a knowledge base architecture that provisioned one Amazon OpenSearch Serverless vector collection per supplier, per subscription tier. This design provided strong tenant isolation but was validated only for a 5 to 10 supplier testing scope.
As Halo AI prepared to support their customers’ full monitored supplier portfolios, a critical scaling limitation emerged. At roughly 3,000 suppliers across 3 subscription tiers, the architecture would require approximately 9,000 individual knowledge base collections. This far exceeded Amazon Bedrock’s limit of 100 knowledge bases per AWS account. While a multi-account workaround using AWS Control Tower was technically possible, it would add substantial engineering complexity and operational overhead.
Additionally, OpenSearch Serverless billing is charged against account-level capacity minimums rather than actual data volume, making the cost structure unfavorable as collection counts grew. Without re-architecture, Halo AI would be unable to scale their product to meet customer demand, and the per-collection cost model would become increasingly inefficient.
During user acceptance testing, Halo AI’s QA team also identified a retrieval quality regression that needed investigation and resolution before the system could move to production.
AWS provided the foundation for Halo AI’s AI-powered retrieval system through Amazon Bedrock’s managed generative AI capabilities and Amazon OpenSearch Service’s vector search functionality. The combination of Bedrock for inference and OpenSearch for hybrid retrieval (semantic plus keyword search) delivered the search quality Halo AI required. AWS’s managed services reduced operational burden while providing the scalability needed to support thousands of suppliers, and native integrations between services simplified the architecture.
Avahi had already designed and built Halo AI’s initial Universal Ask Halo system, giving the team deep familiarity with the existing architecture, codebase, and business requirements. When the scaling limitation was identified during an AWS cost review, Avahi proactively worked with Halo AI to diagnose the root cause and propose a re-architecture approach. Avahi’s expertise in Amazon Bedrock and OpenSearch, combined with their understanding of Halo AI’s multi-tenant isolation requirements, positioned them to deliver a solution that preserved security guarantees while removing the scaling ceiling.
Avahi re-architected Halo AI’s knowledge retrieval system to scale with data volume rather than collection count, while maintaining strict tenant and tier isolation.
Consolidated Vector Storage
The team replaced the per-supplier, per-tier OpenSearch Serverless collections with a single provisioned Amazon OpenSearch Service domain. This domain holds vectors for all suppliers and tiers, with supplier ID, tier, and tenant ID carried as metadata on each indexed chunk. The provisioned domain was sized against the measured corpus using memory-optimized data nodes with EBS storage, configured with VPC placement, fine-grained access control, and encryption at rest and in transit. A CloudWatch storage utilization alarm was pre-configured to surface scaling triggers ahead of capacity limits.
Hybrid Search Implementation
The solution implemented hybrid retrieval combining k-NN vector search with BM25 keyword search, fused using Reciprocal Rank Fusion (RRF). The index mapping included a knn_vector field configured with FAISS engine, HNSW method, and inner product space type, alongside a chunk_text field for BM25 indexing. This hybrid approach improved retrieval quality, particularly for exact-term queries that pure semantic search can miss.
Authorization Gateway
Avahi extended the existing always-on AWS Fargate service into a retrieval and authorization gateway. Every retrieval request is validated against the requesting tenant’s entitlements using the tenant identity established by the API Gateway request authorizer, combined with chunk metadata. The gateway checks that the requesting tenant is entitled to the specific supplier and tier combination before retrieval proceeds, ensuring that customers cannot access data outside their subscription level.
Pipeline and Integration Updates
The existing Lambda retrieval functions were updated to route through the Fargate authorization layer and issue hybrid queries with RRF fusion. The daily ingestion pipeline was modified to write chunks to the new provisioned domain with correct metadata tagging. The end-to-end flow (Lambda to Fargate authorization check to OpenSearch hybrid retrieval to Bedrock inference) preserved the existing back-end API contract throughout.
Data Migration and Validation
All existing vector and text data was migrated from the OpenSearch Serverless multi-collection structure into the new provisioned index with correct metadata tagging. Chunk counts per supplier and tier were recorded before migration and validated after, with spot-check vector integrity verification before decommissioning the Serverless collections.
Quality Assurance
The team investigated and resolved the retrieval quality regression identified during UAT, re-running the agreed LLM evaluation test set, including exact-term query subsets, against the hybrid retrieval pipeline to validate results before sign-off.
Single provisioned Amazon OpenSearch Service domain with hybrid index mapping supporting k-NN and BM25 search with RRF fusion
Fargate-based retrieval and authorization gateway enforcing per-request tenant and tier entitlement checks
Migrated vector and text data with validated chunk counts and integrity
Updated Lambda retrieval functions routing through the authorization gateway
Modified daily ingestion pipeline writing to the new domain with metadata tagging
Resolved retrieval quality regression with validated evaluation results
Updated CloudFormation templates for the provisioned domain, Fargate service, and CloudWatch alarm configuration
Updated architecture diagram, data flow diagram, and operational runbook
Documented hybrid query handler configuration and RRF parameters
The re-architected solution removed the scaling ceiling that would have blocked Halo AI’s growth. The system now scales with data volume rather than knowledge base count, supporting Halo AI’s path from approximately 3,000 suppliers to a projected 10,000 suppliers without architectural changes.
The hybrid search implementation improved retrieval quality for exact-term queries while maintaining semantic search capabilities. The metadata-based isolation approach preserved the strict tenant and tier separation required by Halo AI’s multi-tenant business model, with every request validated against entitlements before data is returned.
The consolidated architecture also improved cost efficiency by moving from OpenSearch Serverless OCU billing (charged against account-level capacity minimums) to provisioned instance billing that scales with actual data volume rather than collection count.
The RAG model developed delivered an accuracy of 98.5% which is much above than what an enterprise RAG can deliver. We tested it on approximately 1200 questions.
Halo AI Inc.
Novato, CA
Technology / AI-Powered Supplier Intelligence
Amazon Bedrock, Amazon OpenSearch Service, AWS Fargate, AWS Lambda, Amazon API Gateway, Amazon CloudWatch, AWS CloudFormation, Amazon VPC
Halo AI, a technology company serving enterprise customers with AI-powered supplier intelligence, needed to scale its “Universal Ask Halo” knowledge retrieval system from a 10-supplier testing scope to support thousands of suppliers across multiple subscription tiers. The original architecture, which used separate Amazon OpenSearch Serverless collections for each supplier and tier combination, would have required approximately 9,000 collections at full scale, exceeding AWS service limits. Avahi re-architected the solution using a single provisioned Amazon OpenSearch Service domain with metadata-based tenant isolation, hybrid search capabilities combining semantic and keyword retrieval, and a Fargate-based authorization gateway. The new design scales with data volume rather than collection count, eliminates the knowledge base ceiling, and improves retrieval quality for exact-term queries.
Halo AI Inc., headquartered in Novato, California, provides AI-powered supplier intelligence solutions to enterprise customers. Their platform enables organizations to monitor and analyze information across extensive supplier and vendor portfolios, with customers tracking up to 3,000 suppliers currently and projecting growth to approximately 10,000 suppliers. Halo AI offers tiered subscription levels that provide varying depths of supplier intelligence, requiring strict data isolation between customers and access tiers.
Halo AI’s initial “Universal Ask Halo” system was built on Amazon Bedrock with a knowledge base architecture that provisioned one Amazon OpenSearch Serverless vector collection per supplier, per subscription tier. This design provided strong tenant isolation but was validated only for a 5 to 10 supplier testing scope.
As Halo AI prepared to support their customers’ full monitored supplier portfolios, a critical scaling limitation emerged. At roughly 3,000 suppliers across 3 subscription tiers, the architecture would require approximately 9,000 individual knowledge base collections. This far exceeded Amazon Bedrock’s limit of 100 knowledge bases per AWS account. While a multi-account workaround using AWS Control Tower was technically possible, it would add substantial engineering complexity and operational overhead.
Additionally, OpenSearch Serverless billing is charged against account-level capacity minimums rather than actual data volume, making the cost structure unfavorable as collection counts grew. Without re-architecture, Halo AI would be unable to scale their product to meet customer demand, and the per-collection cost model would become increasingly inefficient.
During user acceptance testing, Halo AI’s QA team also identified a retrieval quality regression that needed investigation and resolution before the system could move to production.
AWS provided the foundation for Halo AI’s AI-powered retrieval system through Amazon Bedrock’s managed generative AI capabilities and Amazon OpenSearch Service’s vector search functionality. The combination of Bedrock for inference and OpenSearch for hybrid retrieval (semantic plus keyword search) delivered the search quality Halo AI required. AWS’s managed services reduced operational burden while providing the scalability needed to support thousands of suppliers, and native integrations between services simplified the architecture.
Avahi had already designed and built Halo AI’s initial Universal Ask Halo system, giving the team deep familiarity with the existing architecture, codebase, and business requirements. When the scaling limitation was identified during an AWS cost review, Avahi proactively worked with Halo AI to diagnose the root cause and propose a re-architecture approach. Avahi’s expertise in Amazon Bedrock and OpenSearch, combined with their understanding of Halo AI’s multi-tenant isolation requirements, positioned them to deliver a solution that preserved security guarantees while removing the scaling ceiling.
Avahi re-architected Halo AI’s knowledge retrieval system to scale with data volume rather than collection count, while maintaining strict tenant and tier isolation.
Consolidated Vector Storage
The team replaced the per-supplier, per-tier OpenSearch Serverless collections with a single provisioned Amazon OpenSearch Service domain. This domain holds vectors for all suppliers and tiers, with supplier ID, tier, and tenant ID carried as metadata on each indexed chunk. The provisioned domain was sized against the measured corpus using memory-optimized data nodes with EBS storage, configured with VPC placement, fine-grained access control, and encryption at rest and in transit. A CloudWatch storage utilization alarm was pre-configured to surface scaling triggers ahead of capacity limits.
Hybrid Search Implementation
The solution implemented hybrid retrieval combining k-NN vector search with BM25 keyword search, fused using Reciprocal Rank Fusion (RRF). The index mapping included a knn_vector field configured with FAISS engine, HNSW method, and inner product space type, alongside a chunk_text field for BM25 indexing. This hybrid approach improved retrieval quality, particularly for exact-term queries that pure semantic search can miss.
Authorization Gateway
Avahi extended the existing always-on AWS Fargate service into a retrieval and authorization gateway. Every retrieval request is validated against the requesting tenant’s entitlements using the tenant identity established by the API Gateway request authorizer, combined with chunk metadata. The gateway checks that the requesting tenant is entitled to the specific supplier and tier combination before retrieval proceeds, ensuring that customers cannot access data outside their subscription level.
Pipeline and Integration Updates
The existing Lambda retrieval functions were updated to route through the Fargate authorization layer and issue hybrid queries with RRF fusion. The daily ingestion pipeline was modified to write chunks to the new provisioned domain with correct metadata tagging. The end-to-end flow (Lambda to Fargate authorization check to OpenSearch hybrid retrieval to Bedrock inference) preserved the existing back-end API contract throughout.
Data Migration and Validation
All existing vector and text data was migrated from the OpenSearch Serverless multi-collection structure into the new provisioned index with correct metadata tagging. Chunk counts per supplier and tier were recorded before migration and validated after, with spot-check vector integrity verification before decommissioning the Serverless collections.
Quality Assurance
The team investigated and resolved the retrieval quality regression identified during UAT, re-running the agreed LLM evaluation test set, including exact-term query subsets, against the hybrid retrieval pipeline to validate results before sign-off.
Single provisioned Amazon OpenSearch Service domain with hybrid index mapping supporting k-NN and BM25 search with RRF fusion
Fargate-based retrieval and authorization gateway enforcing per-request tenant and tier entitlement checks
Migrated vector and text data with validated chunk counts and integrity
Updated Lambda retrieval functions routing through the authorization gateway
Modified daily ingestion pipeline writing to the new domain with metadata tagging
Resolved retrieval quality regression with validated evaluation results
Updated CloudFormation templates for the provisioned domain, Fargate service, and CloudWatch alarm configuration
Updated architecture diagram, data flow diagram, and operational runbook
Documented hybrid query handler configuration and RRF parameters
The re-architected solution removed the scaling ceiling that would have blocked Halo AI’s growth. The system now scales with data volume rather than knowledge base count, supporting Halo AI’s path from approximately 3,000 suppliers to a projected 10,000 suppliers without architectural changes.
The hybrid search implementation improved retrieval quality for exact-term queries while maintaining semantic search capabilities. The metadata-based isolation approach preserved the strict tenant and tier separation required by Halo AI’s multi-tenant business model, with every request validated against entitlements before data is returned.
The consolidated architecture also improved cost efficiency by moving from OpenSearch Serverless OCU billing (charged against account-level capacity minimums) to provisioned instance billing that scales with actual data volume rather than collection count.
The RAG model developed delivered an accuracy of 98.5% which is much above than what an enterprise RAG can deliver. We tested it on approximately 1200 questions.
Let’s explore your high-impact AI opportunities together in a complimentary session
