Self-Healing AI on Kubernetes: Build an Operator for Your LLM App
AI workloads are flooding onto Kubernetes, and most teams still hand-roll fragile YAML to run them. In this hands-on hour you'll build a small Kubernetes operator that manages an AI agent declaratively — turning a simple AIAgent custom resource into a running, self-healing app — and learn the watch-reconcile pattern behind real projects like KServe and KubeAI.
Running an AI app on Kubernetes usually means a pile of Deployments, Services, ConfigMaps and Secrets that someone maintains by hand. An operator replaces all of that with one declarative resource: you describe the agent you want, and the cluster keeps it running, configured, and healed. In this workshop you'll build that operator yourself.
Starting from a provided skeleton, you'll define an AIAgent custom resource (its model, replicas, and config) and implement the controller that reconciles it: an informer that watches your AIAgent objects, a work queue, and a reconcile function that creates and maintains the underlying Deployment and Service running a lightweight local model. Then we make it earn the name "self-healing" — delete the agent's Deployment and watch the operator recreate it, bump the replica count and watch it scale, change the model field and watch it roll out.
The sample agent runs on CPU against a local cluster (no GPU, no external API keys required), so everyone gets a working AI app on their own laptop. We'll close by mapping what you built onto how production AI operators handle the harder parts — GPUs, model storage, autoscaling.
Requirements:
- Laptop: Required
- Operating system: Linux (preferable), macOS, or Windows
- Software to install:
- Git, Go, kubectl, KIND, and either Podman or Docker.
- Windows users can use Docker/Podman Desktop or Minikube. Linux and macOS users can use Podman or Docker.
- Accounts: No cloud account or API keys required.
Fallback: If you encounter issues setting up the local environment, a browser-based Kubernetes environment using Killercoda will be available as an alternative but it would be limited for k8s concepts. Alternatively you can get paired with other attendees.
Additional requirements: No GPU or prior experience building Kubernetes operators is required. Basic familiarity with Kubernetes Pods, Deployments, and kubectl is recommended.
I’m Kritik Sachdeva, currently working as a Support Professional at IBM. I’ve worked with Ceph & OpenShift for the past 5 years, and since college, I've been very interested in technologies like K8S, containers, and Ceph.
Since then, I’ve enjoyed exploring how different technologies can be integrated to solve real-world problems,
Over the last couple of years, I’ve had the opportunity to speak at various community events & these experiences have helped me grow both technically and personally.
LinkedIn