https://milinddethe15.tech/tags/7-days-of-ai-platform-engineering/Recent content in 7-Days-of-Ai-Platform-Engineering on @milinddethe15HugoenWed, 10 Jun 2026 00:00:00 +0000https://milinddethe15.tech/posts/day-7-ai-platform-engineering/Wed, 10 Jun 2026 00:00:00 +0000https://milinddethe15.tech/posts/day-7-ai-platform-engineering/Over the last 6 days, I’ve learned about: GPUs Kubernetes GPU Scheduling Ray vLLM Each solved a specific problem. But I still had a question: How do all these pieces fit together in a real AI platform? The answer became much clearer when I stopped looking at individual tools and started looking at the entire system. Layer 1: AI Applications Link to heading At the top are the products users interact with.https://milinddethe15.tech/posts/day-6-ai-platform-engineering/Tue, 09 Jun 2026 00:00:00 +0000https://milinddethe15.tech/posts/day-6-ai-platform-engineering/Over the last few days, I’ve learned how AI platforms manage GPUs: Kubernetes discovers GPUs using Device Plugins. GPU utilization is hard. Scheduling AI workloads is different. Ray distributes computation across a cluster. That led me to another question: Why does everyone seem to use vLLM for serving LLMs? Why not just load a model and expose an API? The Naive Approach Link to heading Suppose we deploy an LLM.https://milinddethe15.tech/posts/day-5-ai-platform-engineering/Mon, 08 Jun 2026 00:00:00 +0000https://milinddethe15.tech/posts/day-5-ai-platform-engineering/Over the last few days, I’ve learned that AI infrastructure introduces challenges that traditional applications rarely face: GPUs are expensive. GPU utilization matters. AI workloads often require multiple GPUs at once. Scheduling becomes much harder. That led me to another question: If Kubernetes already schedules containers, why does Ray exist? What Kubernetes Does Well Link to heading Kubernetes is excellent at managing infrastructure. It answers questions like: Which node should this Pod run on?https://milinddethe15.tech/posts/day-4-ai-platform-engineering/Sun, 07 Jun 2026 00:00:00 +0000https://milinddethe15.tech/posts/day-4-ai-platform-engineering/In Day 3, I learned that GPU utilization is one of the biggest challenges in AI infrastructure. That led me to another question: Why can’t Kubernetes scheduling alone solve this problem? After all, Kubernetes has been scheduling workloads for years. The answer is that AI workloads have very different requirements from traditional applications. Traditional Scheduling Link to heading A typical application might look like this: Frontend ↓ Backend ↓ Database Each component can run independently.https://milinddethe15.tech/posts/day-3-ai-platform-engineering/Sat, 06 Jun 2026 00:00:00 +0000https://milinddethe15.tech/posts/day-3-ai-platform-engineering/In Day 2, I learned how Kubernetes discovers GPUs using Device Plugins. Once a GPU is exposed as a resource, a Pod can request it just like CPU or memory. That led me to a new question: If Kubernetes can schedule GPUs, why is GPU utilization still such a big problem? The answer is simple: Most workloads don’t use an entire GPU. The GPU Utilization Problem Link to heading Imagine you have an NVIDIA A100 GPU with 80 GB of memory.https://milinddethe15.tech/posts/day-2-ai-platform-engineering/Fri, 05 Jun 2026 00:00:00 +0000https://milinddethe15.tech/posts/day-2-ai-platform-engineering/Yesterday I learned that AI infrastructure is fundamentally about managing expensive compute resources. That led me to a practical question: How does Kubernetes even know a GPU exists? For CPUs and memory, Kubernetes can discover resources directly from the node. GPUs are different. A GPU isn’t automatically visible to Kubernetes. There is an extra layer that makes everything work. The Journey from GPU to Pod Link to heading The path looks like this:https://milinddethe15.tech/posts/day-1-ai-platform-engineering/Thu, 04 Jun 2026 00:00:00 +0000https://milinddethe15.tech/posts/day-1-ai-platform-engineering/As someone who has spent most of my time with Kubernetes, observability, and platform engineering, I started my AI Platform Engineering journey with a simple question: Why can’t AI workloads run like normal microservices? At first glance, they seem similar. Both are packaged into containers. Both run on Kubernetes. Both expose APIs. Both scale based on demand. So why is there an entirely new ecosystem around AI infrastructure? After digging deeper, I realized the answer lies in where the bottleneck exists.