Simple pod to test system partitioning

[nelsonspbr]

The environment I currently have consists of:

• OpenShift deployment via openshift-installer branch release-4.8 (built from source) with libvirtd, one master and one worker;

• PAO deployment following https://docs.openshift.com/container-platform/4.7; and

• PerformanceProfile:

  apiVersion: performance.openshift.io/v2
  kind: PerformanceProfile
  metadata:
    name: perfprofwork
    namespace: openshift-performance-addon-operator
  spec:
    cpu:
      isolated: "2-7"
      reserved: "0-1"
    realTimeKernel:
      enabled: false
    numa:
      topologyPolicy: "none"
    nodeSelector:
      node-role.kubernetes.io/worker: ""

Once I did this the worker node rebooted and several tasks were moved to the reserved (housekeeping) VCPUs 0-1, as expected. Based on that I assume(d) that the overall setup is functional.

As a final test I wanted to deploy a simple pod on the reserved VCPUs. To do this I:

• Deployed a test namespace with the appropriate annotation:

  apiVersion: v1
  kind: Namespace
  metadata:
    name: ns-1
    annotations:
      workload.openshift.io/allowed: "management"

• Deployed a pod with the appropriate annotation:

  apiVersion: v1
  kind: Pod
  metadata:
    name: pod-1x
    namespace: ns-1
    annotations:
      target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}'
  spec:
    restartPolicy: Never
    nodeSelector:
      node-role.kubernetes.io/worker: ""
    containers:
    - name: cont-1x
      image: rhel-minimal
      command: ["sleep", "infinity"]
      resources:
        requests:
          cpu: 100m
          memory: 256Mi

However, when I check the process CPU mask I get all VCPUs, not just the reserved ones as I expected:

[nelson@nelson-210-ctr test]$ oc get -n ns-1 pod pod-1x -o jsonpath='{.metadata.uid}'
0704aa94-bb3c-43e3-a586-b2ac43a4ade1
[nelson@nelson-210-ctr test]$ oc describe pod -n ns-1 | grep "Container ID"
    Container ID:  cri-o://a7d7a7c6fe0361c11b3ddd0804f07872f2f614d87c12d4037bf3e0ffdb6d1f9c

[core@test1-s57sm-worker-0-gmh7j ~]$ cd /sys/fs/cgroup/cpuset/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0704aa94_bb3c_43e3_a586_b2ac43a4ade1.slice/crio-a7d7a7c6fe0361c11b3ddd0804f07872f2f614d87c12d4037bf3e0ffdb6d1f9c.scope/
[core@test1-s57sm-worker-0-gmh7j ...]$ cat tasks
597427
[core@test1-s57sm-worker-0-gmh7j ...]$ cat /proc/597427/comm
sleep
[core@test1-s57sm-worker-0-gmh7j ...]$ cat /proc/597427/status | grep Cpus
Cpus_allowed:   ff
Cpus_allowed_list:      0-7

The process name matches the one I am running, but the mask still shows 0-7. Am I missing anything? I am happy to provide more information if needed.

[cynepco3hahue]

@nelsonspbr Did you installed the OpenShift with the workload partition support? Take into account that currently, we are supporting workload partitioning only for single-node openshift.

[nelsonspbr]

@cynepco3hahue I do not recall configuring anything related to that in openshift-installer, is there any option available? I did a regular libvirt based installation, then deployed PAO, then the performance profile that specifies the housekeeping cores.

Take into account that currently, we are supporting workload partitioning only for single-node openshift.

Interesting, I did not know that. In any case I find it curious that it seems to be working for existing pods. For example, router-default from openshift-ingress:

PID        CPUS       COMM
1972       0-1        openshift-route
2151       0-1        openshift-route
2152       0-1        openshift-route
2153       0-1        openshift-route
2154       0-1        openshift-route
2183       0-1        openshift-route
2184       0-1        openshift-route
2185       0-1        openshift-route
2190       0-1        openshift-route
2194       0-1        openshift-route
2212       0-1        openshift-route
2214       0-1        openshift-route
2215       0-1        openshift-route
11092      0-1        haproxy
11093      0-1        haproxy
11094      0-1        haproxy
11095      0-1        haproxy
99650      0-1        openshift-route

And many others, which before applying the performance profile were bound to CPUs 0-7 (all available). I checked some of them and they do specify target.workload.openshift.io/management in their pods/deployments, and their namespaces specify workload.openshift.io/allowed: "management". Since I added these to my test pod/namespace, I was wondering if I missed anything else to make it work.

[cynepco3hahue]

@nelsonspbr I am afraid you can see some aligned pods because of the bug - https://bugzilla.redhat.com/show_bug.cgi?id=1979352, but in general, it should not work until you will configure the workload-partitioning and the PAO will support the workload partition configuration only under 4.9.

[nelsonspbr]

Got it @cynepco3hahue. By the way, is there any particular set of instructions to set workload partitioning for single-node OpenShift that I could follow? I tried CRC before using openshift-installer with libvirt and I think I recall reading that CRC did not support something that PAO uses — machine-operator maybe? So I am wondering now what should I use to deploy this single-node OpenShift with workload partitioning and PAO.