Catalog

Running generative AI apps on Amazon EKS? Learn how to reduce GPU costs, increase application resilience, and future-proof architecture. This session discusses hardware options such as NVIDIA’s GPU and AWS Inferentia accelerators, how to benchmark them, and how to gradually migrate. It showcases an image diffusion application deployed on Amazon EKS, powered by Karpenter and scaled by KEDA. The inference application dynamically runs on diverse GPUs and accelerators, and loading the system with 10,000 requests per second on hundreds of accelerators illustrates the power of Kubernetes as an AI/ML inference engine and the cost availability benefits of accelerator diversity.