Kubernetes: Resource Requirements

Resource Requirements

• Every pod requires a set of resources to run (such as CPUs and memory units). When a pod is placed on a node, it consumes the resources available on that node.
• Kubernetes scheduler decides which node a pod goes to. The scheduler takes into consideration the amount of resources required by a pod and decide the best node for the pod.
• If the node does not have sufficient resources, the scheduler avoids placing the pod on those nodes and places the pod in a different node with sufficient resources.
• If there is no node with sufficient resources, the scheduler holds back scheduling the pod.

Resource Requests

• The minimum amount of CPU or memory requested by the container.

# example pod-definition.yaml
apiVersion: v1
kind: Pod
metadata:
  name: simple-app
  labels:
    name: simple-app
spec:
  containers:
  - name: simple-app
    image: simple-app
    ports:
    - containerPort: 8080
    resources:
      requests:
        memory: "4Gi"
        cpu: 2

• With resource requests, pod gets guaranteed to get the amount of resources it requests.

Resource - CPU

What does 1 count of CPU mean?

• 0.1 CPU means 100 m (millie)
• 1 count is equivalent to 1 vCPU (ex: 1 AWS vCPU, 1 GCP Core, 1 Azure Core, 1 Hyperthread)

Resource - Memory

• 256 Mi (similar to 256 M)
• 1G (Gigabyte) = 1,000,000,000 bytes
• 1M (Megabyte) = 1,000,000 bytes
• 1K (Kilobyte) = 1,000 bytes
• 1Gi (Gibibyte) = 1,073,741,824 bytes
• 1Mi (Mebibyte) = 1,048,576 bytes
• 1Ki (Kibibyte) = 1,024 bytes

Resource Limits

• A container may consume as much resources as available on a node.
• We can set a resource limit to ensure the container consumes a limitied amount of resources.

# pod-definition.yaml
apiVersion: v1
kind: Pod
metadata:
  name: simple-webapp
  labels:
    name: simple-webapp
spec:
  containers:
  - name: simple-webapp
    image: simple-webapp
    ports:
    - containerPort: 8080
    resources:
      requests:
        memory: "1Gi"
        cpu: 1
      limits:
        memory: "2Gi"
        cpu: 2

• What happens when a pod tries to exceed its resource limit?
  • For CPU limit, it throttles so it does not go beyond the limit.
  • For memory limit, container can use more memory resource than its limits but pod will be terminated with OOM error in the logs (OOM: Out Of Memory)

Default behaviour

By default, Kubernetes does not have a CPU or memory request or limits. Any pod can consume as much resources on any node and suffocate.

Scenarios for CPU (assume: there are two pods competing for CPU resources on the cluster)

1. No requests and no limits One pod can consume all the CPU resources on the node and prevent the second pod from getting required resources.
2. No requests and yes limits Kubernetes automatically sets requests to be the same as limits. If the limit is set to 3 vCPU, then each pod is guaranteed 3 vCPUs and no more than that.
3. Yes requests and yes limits Each pod gets a guaranteed number of vCPU requests and can go up to the limits. An issue is that if pod 1 needs more vCPU cycles than pod 2, and pod 2 is not consuming many vCPU cycles, the amount of vCPU for pod 1 gets limited and this is not so efficient when there are sufficient resources on the node.
4. Yes requests and no limits Because requests are set, each pod gets the guaranteed resources that they requested. However, since there are no limits specified, any pod can consume as many vCPU cycles. This is the most ideal setup.

A good reason to have limits is to ensure to prevent a user from misusing the infrastructure to perform resource-consuming activities.

A good practice is that when nodes do not have limits, it is important to have at least the requests defined. This way, pods will have resources guaranteed.

Scenarios for Memory (assume: there are two pods competing for memory resources on the cluster)

1. No requests and no limits One pod can consume all the memory resources on the node and prevent the second pod of required resources.
2. No requests and yes limits Kubernetes automatically sets requests to be the same as the limits.
3. Yes requests and yes limits Each pod gets a guaranteed amount of memory and can go up to the limits defined.
4. Yes requests and no limits Each pod is guaranteed requested memory resources, any pod can consume as much memory as available when resources are available. Once memory is assigned to a pod, the only way of retrieving back is to kill the pod and free up all the memory used by it.

LimitRange

How do we ensure that every pod created has some default set of resources?

• Limit ranges help define default values to be set for containers in pods that are created without a request or limit specified in the pod definition files.
• This is applicable at the namespace level.

# limit-range-cpu.yaml
apiVersion: v1
kind: LimitRange
metadata:
  name: cpu-resource-constraint
spec:
  limits:
  - default:  # limit
      cpu: 500m
    defaultRequest:  # request
      cpu: 500m
    max:  # limit
      cpu: "1"
    min:  # request
      cpu: 100m
    type: Container

# limit-range-memory.yaml
apiVersion: v1
kind: LimitRange
metadata:
  name: memory-resource-constraint
spec:
  limits:
  - default:
      memory: 1Gi
    defaultRequest:
      memory: 1Gi
    max:
      memory: 1Gi
    min:
      memory: 500Mi
    type: Container

• These limits are enforced when the pods are created. If we create or change a limit range, it does not affect existing pods.

How to restrict the total amount of resources that can be consumed by applications in a Kubernetes cluster?

• ex: All the pods shouldn’t comsume this amount of CPU or memory

Resource Quotas

• Resource quota is a namespace level object that can be created to set the hard limits for requests and limits. It limits the total requested CPU in the current namespace.

# resource-quota.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: my-resource-quota
spec:
  hard:
    requests.cpu: 4
    requests.memory: 4Gi
    limits.cpu: 10
    limits.memory: 10Gi  

References

• [KodeKloud: Kubernetes: Resource Requirements]