Introduction
In today’s cloud-driven landscape, organizations are continuously seeking ways to optimize costs while enhancing security. A key challenge in multi-account AWS environments is managing access to AWS services efficiently and securely. Traditionally, the deployment of VPC endpoints across various accounts and VPCs results in duplicate resources, increased costs, and a complex security posture.
AllCloud, a leading AWS professional services provider, collaborated with a customer to address these challenges by implementing a centralized VPC endpoints model. This approach centralizes endpoint management, reduces operational overhead, and significantly cuts costs, all while bolstering the security framework of the organization.
About the Customer
The customer is a prominent provider of enterprise software services for the communications and media sector, offering solutions to the world’s top communication service providers and media leaders.
Customer Environment
As part of the multi-account strategy, the customer is running different applications on AWS, which are separated by AWS accounts, resulting in multiple VPCs, each serving distinct business units and applications. We base the network design on a hub-and-spoke topology, creating VPCs in the spoke account and connecting them to a centralized AWS Transit Gateway. This approach allows account owners to decide on using VPC endpoints in each VPC.
The Challenge
The customer used a decentralized approach to access AWS services by setting up VPC endpoints in each VPC to connect to essential services like EC2, EKS, Secrets Manager, Systems Manager, KMS, STS, CloudWatch Logs, and ECR. This was done because of the way the network was set up and the use of multiple accounts.
However, as his infrastructure expanded, the customer noticed that the charges for VPC endpoints increased every month, and the root cause remained unclear.
In addition, he mentioned that the environments became “unmanageable” due to the amount of provisioned VPC endpoints and the inability to control which VPC endpoint was created where and determine what the configured security policy for each was.
After deep diving, AllCloud experts have found that the customer has a total of 48 VPCs with 15 VPC endpoints in each application VPC. The customer also provided us with a cost summary table.
Distributed VS Centralized VPC Endpoint Strategy
The Solution
After careful evaluation and consultation with AllCloud experts, the customer decided to pivot towards a centralized VPC endpoint approach.
The customers recognized the potential benefits of consolidating access to AWS services within a central hub, enabling them to streamline connectivity, enhance security, and simplify management of their cloud infrastructure.
The adoption of centralized VPC endpoints brought significant benefits to the customer in a very short period of time:
-
- Simplified Management: By centralizing access to AWS services, the customer simplified their network architecture and reduced operational overhead.
They no longer had to manage multiple VPC endpoints scattered across their infrastructure, streamlining resource allocation and reducing administrative burden. - Enhanced Security: Centralized endpoints enabled the customer to enforce consistent security policies and access controls across all services.
By defining consistent endpoint policies, they mitigated potential security risks and enhanced data privacy, ensuring compliance with industry regulations. - Scalability and Agility: The centralized approach provided the flexibility to scale the infrastructure seamlessly as their business grew.
Now they can easily add new VPCs or services to the central hub, adapting to evolving business requirements without disruption. - Cost reduction: Leveraging the centralized approach provided the customer a smaller footprint of VPC endpoints, a unique appearance of VPC endpoints for each AWS Service (unlike the previous network layout where each application VPC had its own VPC endpoint stack), which led to significant cost savings in overall VPC endpoint costs every month.
- Simplified Management: By centralizing access to AWS services, the customer simplified their network architecture and reduced operational overhead.
Below is a reference architecture of the implemented solution.
It illustrates the centralized approach where all VPC endpoints are shared and located in a central VPC.
This solution requires an Amazon Transit Gateway to interconnect the application (“Spoke”) VPCs with the “Hub” VPC where all the VPC endpoints are located.
Additionally, to be able to resolve AWS service endpoints from the spoke VPCs, we have to create a Route 53 private hosted zone (for example, sqs.il-central-1.amazonaws.com) and associate it with the peered VPCs.
Interface endpoints are also accessible from on-premises over AWS Site-to- Site VPN or AWS Direct Connect Connection.
To improve resiliency, it’s recommended to provision each VPC endpoint on two availability zones or more.
Cost Analysis
When we started the project, we first analyzed the overall costs of the customer’s existing network layout, which adopted the decentralized approach for VPC endpoints.
Our customer ran his workloads in the North Virginia AWS region.
We have noticed that the price for a single VPC endpoint is based on 2 main characters in his bill:
- Pricing per VPC endpoint per AZ—$0.01/hour
- Data Processed per month in an AWS Region (first 1PB) — $0.01 / GB processed
To better understand the overall monthly pricing that he has been charged for, we investigated deeply his environments, and we found the following details:
- 48 x VPCs, each deployed across 2 x Availability Zones
- 15 x VPC endpoints were in 2 x Availability zones. this results in
30 x ENIs in each VPC - In addition, 15 TB of data is processed by the VPC endpoints in total.
The customer paid $10,655 each month for VPC endpoints in the decentralized VPC endpoints layout.
Before we implemented the centralized approach, we observed that our customer had already implemented Amazon Transit Gateway in his network to interconnect his Spoke VPCs and data centers.
The layout of the centralized VPC endpoints in the North Virginia region includes the following pricing characters:
- Pricing per VPC endpoint per AZ—$0.01155/hour
- Data Processed per month in an AWS Region (first 1PB) — $0.01 / GB processed
- Transit Gateway attachment hourly cost: $0.05/hour
- Transit Gateway data processing cost: -$0.02/GB processed
Before implementing the desired solution, we analyzed and predicted the expected monthly overall cost for VPC endpoints in the centralized layout based on the information we collected:
- 1 x Centralized Shared Endpoints VPC, which is deployed across 2 x Availability Zones
- 15 x VPC endpoints are required—across 2 x Availability zones, this results in 30 x ENIs in the Centralized Shared endpoint VPC.
- 15 TB of data is processed by the VPC endpoints.
- 1 x Transit Gateway attachment for the new Centralized Shared endpoint VPC
The Transit Gateway processes 15 TB of data to connect the existing Spoke VPCs to the Centralized Shared endpoint VPC.
As can be seen, the centralized VPC endpoint layout will cost our customer $716.30 per month.
The customer is going to save $9,949 (!!) per month when he uses the centralized approach rather than the decentralized approach as he has today.
Implementation
After receiving a detailed pricing comparison report from AllCloud, the customer recognized the advantages of the centralized approach over the decentralized approach and approved AllCloud to proceed with the implementation.
The implementation took several weeks, including POC in a demo environment, tests, and conversion of the customer’s non-prod and prod environments from a decentralized model to a centralized one with a minimal distribution.
Takeaways
In the aforementioned example, where VPC endpoints in the two different architectures are processing 15 TB of data, the use of Transit Gateway and centralized VPC endpoints paid off, saving us a total of $9,949 per month, or $119,388 per year!
As demonstrated in the above example, implementing a centralized VPC endpoint architecture can result in monthly savings of hundreds or thousands of USD when dealing with multiple VPCs and low data transfer volumes.
As you can see, the cost calculation was quite simple, allowing you to simulate the overall costs and compare the two approaches based on your actual AWS environment.
Let AllCloud Assist You
AllCloud experts can help you in the whole process: from design to implementation, starting with assessing your existing AWS environment, performing an analysis, and giving you recommendations to select the right solution that best matches your needs.
In addition, AllCloud SOFA (Solution Factory Team) has developed a solution named the “AllCloud Network Module” to support customers in their daily network operations.
The solution provides an automated pipeline for provisioning and managing network components in the network account and the ability to share them with other AWS accounts in the organization.
Hundreds of AllCloud’s customers have already implemented and are operating the solution.
Contact us today to find out how we can support your cloud journey
