Intent classification accuracy on the client’s hidden test set
Manual monitoring needed to catch casework arriving through social media
Facebook, Instagram and X comments and DMs in one pipeline
Civic Group Inc.
New York, NY
GovTech
AWS Lambda, Amazon Bedrock (Anthropic Claude Haiku 4.5), Amazon DynamoDB, Amazon SQS, Amazon EventBridge, AWS Secrets Manager, Amazon CloudWatch, AWS CDK
Civic builds Revere, a constituent relationship management platform for congressional and legislative offices. Revere already captures email, web forms, phone, and mail, but constituent comments, replies, and direct messages on social media sat outside the system entirely, leaving staff to watch notifications by hand. Avahi built and deployed a serverless ingestion and triage pipeline on AWS that connects to a member’s official social accounts, classifies every inbound message by intent using a foundation model on Amazon Bedrock, routes genuine casework to the right team with full provenance, and exposes APIs for staff replies and for Revere to pull the classified records. Against a contractual bar of 85 percent, the classifier scored 94 percent on an independent test set the client withheld until acceptance testing.
Civic Group Inc. builds Revere, a constituent relationship management platform purpose-built for congressional and legislative offices, replacing legacy systems that are decades old. Revere is designed to be the single system of record for constituent communication, covering the correspondence, casework, and contact history that a congressional office handles every day. Its users are the legislative correspondents and caseworkers who answer constituents on behalf of a member, which puts Civic in a govtech market where security expectations are set by the institutions its customers work inside.
Constituents increasingly reach their representatives the same way they reach everyone else, by commenting on a post or sending a direct message. A congressional office may field a stalled veterans’ benefits claim, a passport delay, an opinion on a pending bill, a reporter’s question, and an abusive message, all in the same comment thread under the same post.
None of that traffic reached Revere. Staff had to monitor platform notifications themselves, judge what mattered, and retype anything actionable into the CRM. The volume made that unreliable in practice. A casework request buried under dozens of policy opinions was easy to miss, and a missed message never entered the system that tracks whether the office responded at all.
The cost of leaving this unaddressed fell in two places. Constituents with real problems went unhelped because they picked the wrong channel to ask through, which is exactly the failure a casework system exists to prevent. And Civic’s core promise, one complete record of every constituent interaction, had a growing hole in it. Safety was a factor too: threatening messages need to reach a human quickly, and manual monitoring gave no guarantee of that.
Civic already ran its platform on AWS, so building the pipeline there kept it inside an account boundary and security posture its engineering team had already reviewed. That matters for a company selling into congressional offices, where the architecture is expected to line up with House of Representatives IT security expectations.
The workload also suits managed serverless components. Social traffic to a congressional office is bursty rather than steady, spiking when a member posts about a contentious bill and going quiet overnight, so paying for idle capacity made no sense. Amazon Bedrock supplied the language model as a managed service, which meant no model hosting to operate and no training pipeline to maintain, and the surrounding queueing, storage, and scheduling services were available in the same account with the same identity and logging model.
Civic needed a working intake pipeline across three social platforms, an accurate classifier, and a clean integration contract, all inside a four-week window. As a premier-tier AWS partner delivering generative AI workloads, Avahi could move straight into build without a discovery ramp, and had already completed an earlier engagement for Civic, so the working relationship and the platform context were established.
The delivery model also matched how Civic wanted to work. Civic’s own engineers own Revere and its staff-facing interface, so they wanted a backend service with a documented API contract and complete ownership of the code, rather than a long-running dependency on an outside team. Avahi scoped the engagement to exactly that boundary: Avahi built the ingestion, classification, routing, and API layer, and Civic’s team kept the parts of the product its engineers already owned.
Avahi delivered a serverless, event-driven pipeline on AWS, built around a ports and adapters design so that every external dependency sits behind an interface with both a live and an offline implementation. The practical benefit is that the same code runs against local emulation during development and against real AWS in the cloud, which kept the team building while platform credentials and approvals were still being arranged.
Ingestion runs on two paths matched to what each platform supports. Facebook and Instagram comments and direct messages arrive as webhook events at a public HTTPS entry point, where each request is verified by HMAC signature before anything is accepted. X does not offer equivalent webhook access at the tiers in scope, so a scheduled function polls for mentions every five minutes. Both paths normalize the message into a common shape and place it on an Amazon SQS queue, which separates receiving a message from processing it and gives the pipeline a retry path and a dead-letter queue when a downstream call fails.
A worker function consumes that queue and classifies each message with Anthropic’s Claude Haiku 4.5 on Amazon Bedrock, sorting it into one of five categories: Casework, Policy, Press Inquiry, Feedback, and Threat/Abuse. The classification is zero-shot, with no model training or fine-tuning, so the taxonomy and its boundary rules live in the prompt and can be revised without retraining. The hardest distinction turned out to be hostile criticism of an office’s performance versus genuine abuse directed at a person. Avahi resolved it with a principle rather than a keyword list, separating the two by what the hostility targets, which lifted recall on the Threat/Abuse category from 70 percent to 90 percent and, because it generalizes rather than pattern-matches, held up on wording the classifier had never seen.
Classified messages then flow through matching, routing, and storage. The sender is matched to an existing CRM contact by normalized handle or a new contact is created, so a constituent who writes repeatedly is not duplicated. Casework items are assigned to the right team through a configuration-driven rules layer and stored with full provenance, meaning the platform, the originating post, the comment, and a permalink, so a staffer can always trace a case back to the message that started it. Everything is persisted to a single Amazon DynamoDB table whose key design makes deduplication a direct key lookup rather than a scan.
Two APIs complete the loop. A response-posting endpoint accepts a staff-authored reply and posts it back to the originating thread, choosing a comment reply or a direct message based on how the message arrived. A read-only hand-off API, protected by an API key and closed by default when no key is configured, lets Revere pull classified casework and matched contacts on its own schedule. Replies are always written by a person; the system posts them but does not compose them.
The as-built system is deliberately leaner than the design originally proposed. A streaming service was replaced by a queue, a search cluster for deduplication was replaced by a direct key lookup, and a full API gateway tier was replaced by a single verified entry point, each because the simpler component fully covered the requirement at this volume. Social credentials are held in AWS Secrets Manager and loaded into the runtime at cold start, every function runs under a least-privilege role, structured logs go to Amazon CloudWatch, and tokens are never written to logs. The whole stack is defined as code with AWS CDK, so Civic can stand it up in its own account.
The pipeline was deployed and exercised on AWS, with an end-to-end run carrying a signed webhook event through the queue and the model to a stored casework record, and comment and direct message reply round trips confirmed against real accounts. Connecting the member’s live official accounts and the production cutover sit with Civic, which owns that step for compliance reasons. Avahi also delivered an evaluation and demonstration web application that shows accuracy results, a live inbox across the three platforms with a reply box, and a classify-any-message box used to exercise the deployed model during review sessions.
Serverless ingestion pipeline deployed on AWS Lambda covering Facebook, Instagram, and X comments and direct messages, using verified webhooks where available and scheduled polling where they are not
Zero-shot intent classifier built on Anthropic’s Claude Haiku 4.5 via Amazon Bedrock, sorting every message into Casework, Policy, Press Inquiry, Feedback, or Threat/Abuse with a confidence score and a one-line reason
Casework routing with configurable rules that assign items by platform and intent, attaching full provenance back to the originating post
CRM contact matching and creation with deduplication by normalized handle, backed by a single-table Amazon DynamoDB design
Response-posting API that publishes a staff-authored reply back to the originating comment or direct message thread
Read-only, API-key protected hand-off API that lets Revere pull classified casework records and matched contacts
Message enrichment covering sentiment, stance, emotion, and urgency, together with an AI daily briefing and a plain-English query capability, delivered as a working preview of the next phase
Evaluation and demonstration web application showing accuracy results, a live cross-platform inbox with replies, and live classification
Complete infrastructure as code in AWS CDK, 102 automated tests, and a documentation set covering architecture, API contracts, platform connector setup, and operations
Technical documentation, knowledge transfer guide, and source code handed to Civic’s engineering team
The engagement met its contractual acceptance measure with a wide margin. Civic withheld an independent set of 100 labelled messages until acceptance testing, and the classifier scored 94 percent on it, against a required 85 percent. That result tracked the 94.5 percent measured on the development set, which indicates the classifier generalizes to messages it was never tuned against rather than having learned the development data. On the two categories where a mistake costs the most, the results were strongest: every casework message and every press inquiry in the independent set was identified correctly, and nothing was wrongly labelled as a threat.
For a congressional office, the operational change is that social media becomes a monitored intake channel rather than something staff watch by hand. A constituent’s request arrives, is sorted within minutes, and reaches the caseworker responsible for it with a link back to the original message, while policy opinions, press questions, and safety concerns are separated out and sent where each belongs. Civic received the full source code, the infrastructure definitions, and the documentation to run and extend the pipeline in its own account, and the delivered enrichment and briefing capabilities give it a working preview of the sentiment and trend monitoring its offices asked for next.
94.0 percent classification accuracy on the client’s independent 100-message test set, against a contractual bar of 85 percent (macro F1 of 0.94)
94.5 percent accuracy on the 200-message development set, up from 92.0 percent before prompt tuning
100 percent recall and 100 percent precision on Casework and on Press Inquiry in the independent test set
Threat/Abuse recall improved from 70 percent to 90 percent, with 100 percent precision on the independent set
3 platforms and 5 intent categories covered by a single deployed pipeline, with new messages classified within minutes of arrival
8 of 12 platform read and reply paths verified live, with the remainder gated on platform approvals held by the client
102 automated tests delivered alongside the pipeline, with the full stack defined as code
Estimated AWS running cost of roughly 12 to 15 US dollars per month at a volume of about 10,000 messages
Civic Group Inc.
New York, NY
GovTech
AWS Lambda, Amazon Bedrock (Anthropic Claude Haiku 4.5), Amazon DynamoDB, Amazon SQS, Amazon EventBridge, AWS Secrets Manager, Amazon CloudWatch, AWS CDK
Civic builds Revere, a constituent relationship management platform for congressional and legislative offices. Revere already captures email, web forms, phone, and mail, but constituent comments, replies, and direct messages on social media sat outside the system entirely, leaving staff to watch notifications by hand. Avahi built and deployed a serverless ingestion and triage pipeline on AWS that connects to a member’s official social accounts, classifies every inbound message by intent using a foundation model on Amazon Bedrock, routes genuine casework to the right team with full provenance, and exposes APIs for staff replies and for Revere to pull the classified records. Against a contractual bar of 85 percent, the classifier scored 94 percent on an independent test set the client withheld until acceptance testing.
Civic Group Inc. builds Revere, a constituent relationship management platform purpose-built for congressional and legislative offices, replacing legacy systems that are decades old. Revere is designed to be the single system of record for constituent communication, covering the correspondence, casework, and contact history that a congressional office handles every day. Its users are the legislative correspondents and caseworkers who answer constituents on behalf of a member, which puts Civic in a govtech market where security expectations are set by the institutions its customers work inside.
Constituents increasingly reach their representatives the same way they reach everyone else, by commenting on a post or sending a direct message. A congressional office may field a stalled veterans’ benefits claim, a passport delay, an opinion on a pending bill, a reporter’s question, and an abusive message, all in the same comment thread under the same post.
None of that traffic reached Revere. Staff had to monitor platform notifications themselves, judge what mattered, and retype anything actionable into the CRM. The volume made that unreliable in practice. A casework request buried under dozens of policy opinions was easy to miss, and a missed message never entered the system that tracks whether the office responded at all.
The cost of leaving this unaddressed fell in two places. Constituents with real problems went unhelped because they picked the wrong channel to ask through, which is exactly the failure a casework system exists to prevent. And Civic’s core promise, one complete record of every constituent interaction, had a growing hole in it. Safety was a factor too: threatening messages need to reach a human quickly, and manual monitoring gave no guarantee of that.
Civic already ran its platform on AWS, so building the pipeline there kept it inside an account boundary and security posture its engineering team had already reviewed. That matters for a company selling into congressional offices, where the architecture is expected to line up with House of Representatives IT security expectations.
The workload also suits managed serverless components. Social traffic to a congressional office is bursty rather than steady, spiking when a member posts about a contentious bill and going quiet overnight, so paying for idle capacity made no sense. Amazon Bedrock supplied the language model as a managed service, which meant no model hosting to operate and no training pipeline to maintain, and the surrounding queueing, storage, and scheduling services were available in the same account with the same identity and logging model.
Civic needed a working intake pipeline across three social platforms, an accurate classifier, and a clean integration contract, all inside a four-week window. As a premier-tier AWS partner delivering generative AI workloads, Avahi could move straight into build without a discovery ramp, and had already completed an earlier engagement for Civic, so the working relationship and the platform context were established.
The delivery model also matched how Civic wanted to work. Civic’s own engineers own Revere and its staff-facing interface, so they wanted a backend service with a documented API contract and complete ownership of the code, rather than a long-running dependency on an outside team. Avahi scoped the engagement to exactly that boundary: Avahi built the ingestion, classification, routing, and API layer, and Civic’s team kept the parts of the product its engineers already owned.
Avahi delivered a serverless, event-driven pipeline on AWS, built around a ports and adapters design so that every external dependency sits behind an interface with both a live and an offline implementation. The practical benefit is that the same code runs against local emulation during development and against real AWS in the cloud, which kept the team building while platform credentials and approvals were still being arranged.
Ingestion runs on two paths matched to what each platform supports. Facebook and Instagram comments and direct messages arrive as webhook events at a public HTTPS entry point, where each request is verified by HMAC signature before anything is accepted. X does not offer equivalent webhook access at the tiers in scope, so a scheduled function polls for mentions every five minutes. Both paths normalize the message into a common shape and place it on an Amazon SQS queue, which separates receiving a message from processing it and gives the pipeline a retry path and a dead-letter queue when a downstream call fails.
A worker function consumes that queue and classifies each message with Anthropic’s Claude Haiku 4.5 on Amazon Bedrock, sorting it into one of five categories: Casework, Policy, Press Inquiry, Feedback, and Threat/Abuse. The classification is zero-shot, with no model training or fine-tuning, so the taxonomy and its boundary rules live in the prompt and can be revised without retraining. The hardest distinction turned out to be hostile criticism of an office’s performance versus genuine abuse directed at a person. Avahi resolved it with a principle rather than a keyword list, separating the two by what the hostility targets, which lifted recall on the Threat/Abuse category from 70 percent to 90 percent and, because it generalizes rather than pattern-matches, held up on wording the classifier had never seen.
Classified messages then flow through matching, routing, and storage. The sender is matched to an existing CRM contact by normalized handle or a new contact is created, so a constituent who writes repeatedly is not duplicated. Casework items are assigned to the right team through a configuration-driven rules layer and stored with full provenance, meaning the platform, the originating post, the comment, and a permalink, so a staffer can always trace a case back to the message that started it. Everything is persisted to a single Amazon DynamoDB table whose key design makes deduplication a direct key lookup rather than a scan.
Two APIs complete the loop. A response-posting endpoint accepts a staff-authored reply and posts it back to the originating thread, choosing a comment reply or a direct message based on how the message arrived. A read-only hand-off API, protected by an API key and closed by default when no key is configured, lets Revere pull classified casework and matched contacts on its own schedule. Replies are always written by a person; the system posts them but does not compose them.
The as-built system is deliberately leaner than the design originally proposed. A streaming service was replaced by a queue, a search cluster for deduplication was replaced by a direct key lookup, and a full API gateway tier was replaced by a single verified entry point, each because the simpler component fully covered the requirement at this volume. Social credentials are held in AWS Secrets Manager and loaded into the runtime at cold start, every function runs under a least-privilege role, structured logs go to Amazon CloudWatch, and tokens are never written to logs. The whole stack is defined as code with AWS CDK, so Civic can stand it up in its own account.
The pipeline was deployed and exercised on AWS, with an end-to-end run carrying a signed webhook event through the queue and the model to a stored casework record, and comment and direct message reply round trips confirmed against real accounts. Connecting the member’s live official accounts and the production cutover sit with Civic, which owns that step for compliance reasons. Avahi also delivered an evaluation and demonstration web application that shows accuracy results, a live inbox across the three platforms with a reply box, and a classify-any-message box used to exercise the deployed model during review sessions.
Serverless ingestion pipeline deployed on AWS Lambda covering Facebook, Instagram, and X comments and direct messages, using verified webhooks where available and scheduled polling where they are not
Zero-shot intent classifier built on Anthropic’s Claude Haiku 4.5 via Amazon Bedrock, sorting every message into Casework, Policy, Press Inquiry, Feedback, or Threat/Abuse with a confidence score and a one-line reason
Casework routing with configurable rules that assign items by platform and intent, attaching full provenance back to the originating post
CRM contact matching and creation with deduplication by normalized handle, backed by a single-table Amazon DynamoDB design
Response-posting API that publishes a staff-authored reply back to the originating comment or direct message thread
Read-only, API-key protected hand-off API that lets Revere pull classified casework records and matched contacts
Message enrichment covering sentiment, stance, emotion, and urgency, together with an AI daily briefing and a plain-English query capability, delivered as a working preview of the next phase
Evaluation and demonstration web application showing accuracy results, a live cross-platform inbox with replies, and live classification
Complete infrastructure as code in AWS CDK, 102 automated tests, and a documentation set covering architecture, API contracts, platform connector setup, and operations
Technical documentation, knowledge transfer guide, and source code handed to Civic’s engineering team
The engagement met its contractual acceptance measure with a wide margin. Civic withheld an independent set of 100 labelled messages until acceptance testing, and the classifier scored 94 percent on it, against a required 85 percent. That result tracked the 94.5 percent measured on the development set, which indicates the classifier generalizes to messages it was never tuned against rather than having learned the development data. On the two categories where a mistake costs the most, the results were strongest: every casework message and every press inquiry in the independent set was identified correctly, and nothing was wrongly labelled as a threat.
For a congressional office, the operational change is that social media becomes a monitored intake channel rather than something staff watch by hand. A constituent’s request arrives, is sorted within minutes, and reaches the caseworker responsible for it with a link back to the original message, while policy opinions, press questions, and safety concerns are separated out and sent where each belongs. Civic received the full source code, the infrastructure definitions, and the documentation to run and extend the pipeline in its own account, and the delivered enrichment and briefing capabilities give it a working preview of the sentiment and trend monitoring its offices asked for next.
94.0 percent classification accuracy on the client’s independent 100-message test set, against a contractual bar of 85 percent (macro F1 of 0.94)
94.5 percent accuracy on the 200-message development set, up from 92.0 percent before prompt tuning
100 percent recall and 100 percent precision on Casework and on Press Inquiry in the independent test set
Threat/Abuse recall improved from 70 percent to 90 percent, with 100 percent precision on the independent set
3 platforms and 5 intent categories covered by a single deployed pipeline, with new messages classified within minutes of arrival
8 of 12 platform read and reply paths verified live, with the remainder gated on platform approvals held by the client
102 automated tests delivered alongside the pipeline, with the full stack defined as code
Estimated AWS running cost of roughly 12 to 15 US dollars per month at a volume of about 10,000 messages
Let’s explore your high-impact AI opportunities together in a complimentary session
