Scaling Containerized Applications on AWS with Amazon EKS

Client

CustomAmaz

Location

USA

Industry

SaaS

Services & Tech

Amazon EKS, CodeCommit, CodeBuild

Project Overview

As Serviceo continues to grow, the company needs to be able to deploy, test, validate and promote components rapidly in their micro-service and component-based architecture. The software development team was experiencing significant struggles to work as a unified team. Code was being committed and deployed with broken library dependencies, resulting in manual fixes every release to ensure code was running correctly. The applications were disjointed and inconsistent, with very loosely coupled sets of software, services, and scripts. There was no robust deployment of the applications, and once deployed, there was often a need to intervene manually. The team wanted to have a Zero Downtime Deployment (ZDD) strategy to deploy a new version of services without interrupting the operation of the service.

About the
Customer

Serviceo offers an innovative software platform that leverages over a decade of industry experience with flexible business models that enables organizations to streamline their global IT field service delivery. Their platform provides On-Demand field services around the globe, meeting enterprise SLAs utilizing 600 forward stocking locations around the world and a network of thousands of vetted and highly qualified local technicians.

Architecture

Below is the architecture diagram of solution designed by Avahi’s cloud solution architects.

Following components were used to build this solution.

  • Amazon EKS cluster in us-east-2 region. Amazon EKS control plane is fully managed and is deployed in its own VPC.
  • One VPC with one public subnet and two private subnets in each Availability Zone (AZ) for nodes.
  • On-Demand instances in an Amazon EC2 Auto Scaling groups (deployed over multiple AZs) that are used as underlying computing infrastructure for the Amazon EKS cluster. EKS offers the possibility of employing multiple ASGs with different instance flavors, where each ASG has specific tags that can be used to schedule pods via label selectors. For our design, we chose C5.9xlarge instances that provide enough compute power for our CPU-hungry applications while also optimizing costs.
  • Amazon Elastic Container Registry (Amazon ECR) stores the Docker images for application microservices and Helm add-ons for application deployments.
  • An Amazon Route 53 Domain Name System (DNS) zone for service discovery and a Network Load Balancer configured for HTTPS encrypted traffic.
  • An Application Load Balancer ingress controller to load balance the application traffic.
  • Kubernetes Metrics Server to collect metrics from running pods, such as CPU and memory utilization.
  • Kubernetes Cluster Autoscaler to scale in and out of nodes.
  • Amazon ElastiCache Redis database that stores the user sessions and state.
  • Amazon RDS service for SQL databases.
  • Amazon CloudWatch with Fluent Bit for logging application logs and cluster logs.
  • Amazon OpenSearch Service (successor to Amazon Elasticsearch Service) and Amazon Simple Storage Service (Amazon S3) for centralized logging.

There are several options available for selecting an ingress controller for EKS, and most of them are open source solutions. Below are the use cases that can influence the decision when selecting ingress controller:

  • Scaling requirements, available resources, and resource use patterns.
  • Traffic types served (HTTP, gRPC, WebSocket, etc.)
  • The number of requests, networking policies, monitoring, and logging requriements.

We decided to use AWS Elastic Load Balancing (ELB) for ingress with a plan to gradually migrate to a more complex solution in the near future. While ELBs may have a limited feature set, it minimizes operational complexity and provides stability and easy scaling.

Solution

Working together, Avahi and Serviceo development team decided that the best approach is:

  • Containerize the applications.
  • Deploy the most efficient Continuous Integration/Continuous Deployment (CI/CD) pipeline.
  • Restructure the Git software repositories into specific areas of concern (apps/services/dependencies).
  • Adopt Amazon EKS for automating deployment, scaling, and management of containerized applications.

Avahi cloud engineers planned to build a more modular, extensible system to support future feature enhancements and a more frequent release cycle. Kubernetes-based solution for runtime can do that while providing the scale to meet present-day and future demands on the system. The solution also had to be secure enough to meet regulatory requirements, such as the California Consumer Privacy Act (CCPA) and General Data Protection Regulation (GDPR). Avahi proposed to use Amazon EKS for Kubernetes instead of building the cluster from the ground up.

Amazon Elastic Kubernetes Service (Amazon EKS) is a managed container orchestration service that provides templates, code samples, and best practices to help developers ramp up quickly. AWS manages the infrastructure for Kubernetes, freeing the team to focus on adding value to their application – not on operations. The benefits of using Amazon EKS for Serviceo include:

Efficient resource utilization

The fully managed EKS offers easy deployment and management of containerized applications. It enables efficient resource utilization that elastically provisions additional resources without the headache of managing the Kubernetes infrastructure.

Faster application development

Developers spent most of the time on bug-fixing. EKS reduces the debugging time while handling patching, auto-upgrades, and self-healing and simplifies the container orchestration. It saves a lot of time, and developers will focus on developing their apps while remaining more productive.

Security and compliance

Cybersecurity is one of the most important aspects of modern applications and businesses. EKS integrates with AWS IAM and offers users on-demand access to reduce threats and risks. EKS is also completely compliant with the standards and regulatory requirements such as System and Organization Controls (SOC), HIPAA, ISO, and PCI DSS.

Quicker development and integration

EKS supports auto-upgrades, monitoring, and scaling and helps minimize the infrastructure maintenance that leads to comparatively faster development and integration. It also supports provisioning additional compute resources in Serverless Kubernetes within seconds without worrying about managing the Kubernetes infrastructure.

We highly recommend Avahi as a reliable and innovative technology partner. Their expertise in cutting-edge technologies was instrumental in building our Proof of Concept (PoC) and developing our Minimum Viable Product (MVP). Avahi consistently delivered high-quality solutions on time while maintaining a collaborative, responsive approach. They went beyond expectations by identifying opportunities for enhancement, ensuring scalability and compliance for our law enforcement-focused products. Avahi is the clear choice if you need a tech partner with industry knowledge, professionalism, and a commitment to innovation.

Brandon Puhlman

Founder, Bravo Foxtrot

Ready to Transform Your Business with AI?

Book Your Free Ignition AI Workshop

Let’s explore your high-impact AI opportunities together in a complimentary half-day session

View Our Case Studies

See how we’ve delivered measurable results for businesses like yours