Optimize AWS Glue IP addresses & Expand Network with Private NAT

angelo Leave a Comment

Event Timeslots (1)

Track 2 – 2024
Presenter: Sushanth Kothapally and Senthil Kamala Rathinam

Abstract: As enterprises grow, their demand for IP addresses often exceeds what their corporate network can provide. While networks are designed to tackle future needs, evolving enterprises eventually outgrow them. This often possess constraint of running multiple AWS Glue jobs in parallel that require number of database connections. To enable more connections, organizations can obtain additional IP addresses from their corporate pool, either unique or overlapping with the existing network. Overlapping IP addresses require extra network management like private NAT gateways, AWS PrivateLink, or self-managed NAT appliances to establish connectivity. This presentation discusses two strategies to scale AWS Glue jobs within IP-constrained networks:
Optimize IP address usage by right-sizing Data Processing Units, using AWS Glue Auto Scaling, and fine-tuning jobs
Expand network capacity by adding a non-routable CIDR range with a private NAT gateway.

Implementing these strategies allows companies to maximize their use of IP addresses to support more data processing workloads using AWS Glue

AWS Services: AWS Glue, AWS Transit Gateway, AWS Transit Gateway, Amazon VPC

Audience: Advanced

angelo angelo

Leave a Reply

Your email address will not be published. Required fields are marked *