Self-Healing AI on Kubernetes: Build an Operator for Your LLM App
AI workloads are flooding onto Kubernetes, and most teams still hand-roll fragile YAML to run them. In this hands-on hour you'll build a small Kubernetes operator that manages an AI agent declaratively — turning a simple AIAgent custom resource into a running, self-healing app — and learn the watch-reconcile pattern behind real projects like KServe and KubeAI.
Running an AI app on Kubernetes usually means a pile of Deployments, Services, ConfigMaps and Secrets that someone maintains by hand. An operator replaces all of that with one declarative resource: you describe the agent you want, and the cluster keeps it running, configured, and healed. In this workshop you'll build that operator yourself.
Starting from a provided skeleton, you'll define an AIAgent custom resource (its model, replicas, and config) and implement the controller that reconciles it: an informer that watches your AIAgent objects, a work queue, and a reconcile function that creates and maintains the underlying Deployment and Service running a lightweight local model. Then we make it earn the name "self-healing" — delete the agent's Deployment and watch the operator recreate it, bump the replica count and watch it scale, change the model field and watch it roll out.
The sample agent runs on CPU against a local cluster (no GPU, no external API keys required), so everyone gets a working AI app on their own laptop. We'll close by mapping what you built onto how production AI operators handle the harder parts — GPUs, model storage, autoscaling.
I’m Kritik Sachdeva, currently working as a Support Professional at IBM. I’ve worked with Ceph & OpenShift for the past 5 years, and since college, I've been very interested in technologies like K8S, containers, and Ceph.
Since then, I’ve enjoyed exploring how different technologies can be integrated to solve real-world problems,
Over the last couple of years, I’ve had the opportunity to speak at various community events & these experiences have helped me grow both technically and personally.
LinkedIn