Aman Nambisan

Scales vLLM pods on KV cache pressure, not CPU.

A Kubernetes autoscaler that scales vLLM inference pods on KV cache pressure and request queue depth rather than CPU, because on a GPU server CPU is not the bottleneck and rises only after users are already waiting.

Traffic spikes fill the cache, the autoscaler reads that as a leading indicator, KEDA adds a replica and the cluster autoscaler provisions an L4 — so time-to-first-token stays flat through the spike.

• Kubernetes
• vLLM
• KEDA
• Terraform
• Prometheus

An LLM proposes model changes; winners ship themselves.

Drop in a binary-classification CSV, declare its schema, and a closed loop takes over: Claude proposes a change, the system trains it on Kubernetes, and only candidates that beat the current champion get promoted. Accuracy went from roughly 70% to 80% across 20 iterations.

DVC versions the data, Kubeflow Pipelines runs training, MLflow holds the champion/challenger registry, and a FastAPI service is rolled forward via GitOps whenever a new champion is promoted. No human in the loop.

• Kubernetes
• Kubeflow
• MLflow
• DVC
• FastAPI
• GitOps

Matches on S2 cells; a Redis lock prevents double-booking.

A Go modular monolith split by domain — booking, matching, pricing — communicating over Kafka, with asynchronous booking processing and real-time ride tracking.

Redis provides a distributed lock so a rider cannot be double-booked, and Google's S2 library handles spatial indexing for proximity search in the driver-matching service.

• Go
• Kafka
• Redis
• S2