Kubernetes 1.33: Resizing Pods
Remember that feeling? You meticulously configured your Kubernetes pods, set the CPU and memory just right (or so you thought), only to have your application start gasping for air or hogging resources like it’s Black Friday for RAM. In the old days, the only cure was a full pod restart — a disruptive event that often felt like performing open-heart surgery with a butter knife while your SRE team watched through the Operating Room window.
Well, good news, fellow Kubernetes wranglers! Version 1.33 has landed, and it brings with it a feature that many of us have been dreaming about: in-place pod vertical scaling! Yes, you read that right. You can now adjust the CPU and memory of your running pods without the dreaded restart. Cue the confetti cannons and slightly-too-enthusiastic DevOps high-fives!
This is now a beta feature in Kubernetes 1.33 and enabled by default! You no longer need to enable feature gates to use it, making it much more accessible for production workloads (Kubernetes docs confirm).
This is particularly exciting if you, like me, have been pondering the nuances of vertical pod autoscaling. While VPA is fantastic in theory, its current “recreate” mode can be a bit… dramatic. In-place resize offers a more graceful way to adjust resources, potentially paving the way for a smoother VPA experience down the line.
Why This Feature is Bigger Than My Morning Coffee ☕
For us Kubernetes folks, this feature is a game-changer. Imagine a scenario where your application suddenly experiences a surge in traffic. Previously, you’d either have to over-provision (costly!) or trigger a VPA update that would bounce your pods. Now, you can simply nudge the CPU and memory upwards on the fly, keeping your application happy and your users even happier.
Think about stateful applications, databases, or those applications that benefit from continuous availability. In-place resize can minimize downtime and provide a much more seamless scaling experience. Let’s break down why:
Real-World Use Cases
Let’s explore some concrete scenarios where this feature truly shines:
A Note on Java Applications: For JVM-based applications, it’s important to understand that simply resizing a pod’s memory resources doesn’t automatically adjust the JVM’s heap size, which is typically set at startup with flags like -Xmx (source). While in-place resizing can help with non-heap memory and CPU resources, to fully utilize increased memory limits, Java applications typically require configuration adjustments and restarts. This makes them less ideal candidates for memory resizing without restarts.
Under the Hood: How Kubernetes Performs Resource Jiu-Jitsu 🤼
Let’s geek out on the technical magic, complete with diagrams even your non-tech PM will adore:
What’s Really Happening
Container Runtime Compatibility
This feature works across container runtimes with varying levels of support:
It’s worth noting that cgroup v2 offers better memory management capabilities during resize operations compared to cgroup v1, particularly for memory limit decreases (source).
Hands-On: Break Things (Safely!) 🔧
Let’s create a simple demo that shows in-place resizing in action, viewable both from the Kubernetes API and from inside the Pod itself. This whole demo runs on GKE with Kubernetes 1.33 (Rapid channel).
1. Create a Resource-Monitoring Pod
Start by creating a Pod that continuously monitors its own resource allocations:
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: resize-demo
spec:
containers:
- name: resource-watcher
image: ubuntu:22.04
command:
- "/bin/bash"
- "-c"
- |
apt-get update && apt-get install -y procps bc
echo "=== Pod Started: $(date) ==="
# Functions to read container resource limits
get_cpu_limit() {
if [ -f /sys/fs/cgroup/cpu.max ]; then
# cgroup v2
local cpu_data=$(cat /sys/fs/cgroup/cpu.max)
local quota=$(echo $cpu_data | awk '{print $1}')
local period=$(echo $cpu_data | awk '{print $2}')
if [ "$quota" = "max" ]; then
echo "unlimited"
else
echo "$(echo "scale=3; $quota / $period" | bc) cores"
fi
else
# cgroup v1
local quota=$(cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us)
local period=$(cat /sys/fs/cgroup/cpu/cpu.cfs_period_us)
if [ "$quota" = "-1" ]; then
echo "unlimited"
else
echo "$(echo "scale=3; $quota / $period" | bc) cores"
fi
fi
}
get_memory_limit() {
if [ -f /sys/fs/cgroup/memory.max ]; then
# cgroup v2
local mem=$(cat /sys/fs/cgroup/memory.max)
if [ "$mem" = "max" ]; then
echo "unlimited"
else
echo "$((mem / 1048576)) MiB"
fi
else
# cgroup v1
local mem=$(cat /sys/fs/cgroup/memory/memory.limit_in_bytes)
echo "$((mem / 1048576)) MiB"
fi
}
# Print resource info every 5 seconds
while true; do
echo "---------- Resource Check: $(date) ----------"
echo "CPU limit: $(get_cpu_limit)"
echo "Memory limit: $(get_memory_limit)"
echo "Available memory: $(free -h | grep Mem | awk '{print $7}')"
sleep 5
done
resizePolicy:
- resourceName: cpu
restartPolicy: NotRequired
- resourceName: memory
restartPolicy: NotRequired
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "128Mi"
cpu: "100m"
EOF
2. Explore the Pod’s Initial State
Let’s look at the Pod’s resources from the Kubernetes API perspective:
kubectl describe pod resize-demo | grep -A8 Limits:
You’ll see output like:
Limits:
cpu: 100m
memory: 128Mi
Requests:
cpu: 100m
memory: 128Mi
Now, let’s see what the Pod itself thinks about its resources:
kubectl logs resize-demo --tail=8
You should see output including CPU and memory limits from the container’s perspective.
3. Resize CPU Seamlessly
Let’s double the CPU without any restart:
kubectl patch pod resize-demo --subresource resize --patch \
'{"spec":{"containers":[{"name":"resource-watcher", "resources":{"requests":{"cpu":"200m"}, "limits":{"cpu":"200m"}}}]}}'
Check the resize status:
kubectl get pod resize-demo -o jsonpath='{.status.conditions[?(@.type=="PodResizeInProgress")]}'
Note: On GKE with Kubernetes 1.33, you might not see the PodResizeInProgress condition reported in the Pod status, even though the resize operation works correctly. Don't worry if kubectl get pod resize-demo -o jsonpath='{.status.conditions}' doesn't show resize information - check the actual resources instead.
Once that shows the resize has completed, check the updated resources from the Kubernetes API:
kubectl describe pod resize-demo | grep -A8 Limits:
And verify the Pod now sees the new CPU limit:
kubectl logs resize-demo --tail=8
Notice how the CPU limit doubled from 100m to 200m without the Pod restarting! The Pod's logs will show the cgroup CPU limit changed from approximately 10000/100000 to 20000/100000(representing 100m to 200m CPU).
4. Resize Memory Without Drama
Now, let’s double the memory allocation:
kubectl patch pod resize-demo --subresource resize --patch \
'{"spec":{"containers":[{"name":"resource-watcher", "resources":{"requests":{"memory":"256Mi"}, "limits":{"memory":"256Mi"}}}]}}'
After a moment, verify from the API:
kubectl describe pod resize-demo | grep -A8 Limits:
And from inside the Pod:
kubectl logs resize-demo --tail=8
You’ll see the memory limit changed from 64Mi to 128Mi without any container restart!
5. Verify No Container Restarts Occurred
Confirm the container never restarted during our resize operations:
kubectl get pod resize-demo -o jsonpath='{.status.containerStatuses[0].restartCount}'
Should output 0 - proving we achieved the impossible dream of resource adjustment without service interruption.
6. Cleanup
When you’re done experimenting:
kubectl delete pod resize-demo
That’s it! You’ve successfully performed in-place resizing of both CPU and memory resources without any Pod restarts. This pattern works great for any containerized application and doesn’t require any special configuration beyond setting the appropriate resizePolicy.
Cloud Provider Support 🌩️
Before you rush to try this in production, let’s look at support across major Kubernetes providers:
The Fine Print: Limitations (Because Nothing’s Perfect) 📜
While this feature is cooler than a penguin in sunglasses, keep these in mind:
Platform & Runtime Limitations
Resource Management Constraints
Configuration & Integration Limits
Performance Considerations
Keep these caveats in mind, and you’ll avoid the worst of the drama. 🎭✨
VPA: The Autoscaler That Could (With Some Help) 🤖
As I recently lamented on LinkedIn, the Vertical Pod Autoscaler has been the awkward cousin at the scaling family reunion. But there’s hope!
Current Status (K8s 1.33): VPA does not yet support in-place resizing — it still recreates pods when adjusting resources. This limitation is explicitly noted in the Kubernetes documentation: “As of Kubernetes 1.33, VPA does not support resizing pods in-place, but this integration is being worked on.”
Active development is happening in kubernetes/autoscaler PR 7673 to integrate VPA with in-place resizing capability.
The Future Integration We All Want:
This hybrid approach (proposed in KEP-4951) would make VPA finally production-ready for stateful workloads. Until then, we’ll have to manually play resource Jenga with our pods.
Combining VPA and Manual Resizing Today
While waiting for full integration, you can still benefit from both:
# Example of a simple script to apply VPA recommendations via in-place resize
#!/bin/bash
POD_NAME="my-important-db"
CPU_REC=$(kubectl get vpa db-vpa -o jsonpath='{.status.recommendation.containerRecommendations[0].target.cpu}')
MEM_REC=$(kubectl get vpa db-vpa -o jsonpath='{.status.recommendation.containerRecommendations[0].target.memory}')
kubectl patch pod $POD_NAME --subresource resize --patch \
"{\"spec\":{\"containers\":[{\"name\":\"database\",\"resources\":{\"requests\":{\"cpu\":\"$CPU_REC\",\"memory\":\"$MEM_REC\"}}}]}}"
Comparison with Alternative Solutions
The Road Ahead: Vertical Scaling’s Next Chapter 🚀
Kubernetes 1.33’s in-place pod resize is a giant leap toward making vertical scaling as seamless and non-disruptive as horizontal autoscaling, but the story is far from over. As this feature matures, we’re already eyeing a true scaling renaissance:
Together, these enhancements promise a future where vertical scaling is truly dynamic, efficient, and interruption-free. So, go forth, Kubernetes adventurers, experiment with in-place pod resize in your non-production clusters, share your findings, and help shape the next wave of scaling magic. Just don’t forget to read the fine print and test thoroughly before unleashing in production. Happy scaling! 🚀