OKD Rook (ocs) Setup

README.md

mgr plugin fails with

1

debug 2022-12-06T03:08:59.856+0000 7f183006f700  0 [rook ERROR rook.rook_cluster] No storage class exists matching configured Rook orchestrator storage class which currently is <local>. This storage class can be set in ceph config (mgr/rook/storage_class)
debug 2022-12-06T03:08:59.857+0000 7f183006f700  0 [rook ERROR orchestrator._interface] No storage class exists matching name provided in ceph config at mgr/rook/storage_class
Traceback (most recent call last):
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 125, in wrapper
    return OrchResult(f(*args, **kwargs))
  File "/usr/share/ceph/mgr/rook/module.py", line 229, in get_inventory
    discovered_devs = self.rook_cluster.get_discovered_devices(host_list)
  File "/usr/share/ceph/mgr/rook/rook_cluster.py", line 713, in get_discovered_devices
    storage_class = self.get_storage_class()
  File "/usr/share/ceph/mgr/rook/rook_cluster.py", line 709, in get_storage_class
    raise Exception('No storage class exists matching name provided in ceph config at mgr/rook/storage_class')
Exception: No storage class exists matching name provided in ceph config at mgr/rook/storage_class
debug 2022-12-06T03:08:59.857+0000 7f183006f700  0 [dashboard ERROR exception] Internal Server Error
Traceback (most recent call last):
  File "/usr/share/ceph/mgr/dashboard/services/exception.py", line 47, in dashboard_exception_handler
    return handler(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/usr/share/ceph/mgr/dashboard/controllers/_base_controller.py", line 258, in inner
    ret = func(*args, **kwargs)
  File "/usr/share/ceph/mgr/dashboard/controllers/orchestrator.py", line 33, in _inner
    return method(self, *args, **kwargs)
  File "/usr/lib64/python3.6/contextlib.py", line 52, in inner
    return func(*args, **kwds)
  File "/usr/share/ceph/mgr/dashboard/controllers/host.py", line 506, in inventory
    return get_inventories(None, refresh)
  File "/usr/share/ceph/mgr/dashboard/controllers/host.py", line 251, in get_inventories
    for host in orch.inventory.list(hosts=hosts, refresh=do_refresh)]
  File "/usr/share/ceph/mgr/dashboard/services/orchestrator.py", line 38, in inner
    raise_if_exception(completion)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 228, in raise_if_exception
    raise e

Issue: rook/rook#10703

Fix: ceph config set mgr.a mgr/rook/storage_class <disk-storage-class>

2

debug 2022-12-06T03:25:59.010+0000 7f7bb08e4700  0 [rook ERROR rook.rook_cluster] Failed to fetch device metadata
debug 2022-12-06T03:25:59.011+0000 7f7bb08e4700  0 [rook ERROR orchestrator._interface] (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Audit-Id': 'e2c75597-1f58-4be0-b080-b306782e16e7', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': '28374a6d-236b-4e03-a764-388189ef6555', 'X-Kubernetes-Pf-Prioritylevel-Uid': '72f430bf-7eb3-49a2-8725-a36c539ac7d7', 'Date': 'Tue, 06 Dec 2022 03:25:59 GMT', 'Content-Length': '449'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"localvolumediscoveryresults.local.storage.openshift.io is forbidden: User \"system:serviceaccount:openshift-storage:rook-ceph-mgr\" cannot list resource \"localvolumediscoveryresults\" in API group \"local.storage.openshift.io\" at the cluster scope","reason":"Forbidden","details":{"group":"local.storage.openshift.io","kind":"localvolumediscoveryresults"},"code":403}

Add ClusterRoleBinding:

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: manual-rook-ceph-mgr
subjects:
  - kind: ServiceAccount
    name: rook-ceph-mgr
    namespace: openshift-storage
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: localvolumediscoveryresults.local.storage.openshift.io-v1alpha1-view

3

NFS-Ganesha is not configured

Remote method threw exception: Traceback (most recent call last): File "/usr/share/ceph/mgr/nfs/module.py", line 154, in cluster_ls return available_clusters(self) File "/usr/share/ceph/mgr/nfs/utils.py", line 39, in available_clusters orchestrator.raise_if_exception(completion) File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 228, in raise_if_exception raise e kubernetes.client.rest.ApiException: (403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Audit-Id': '1dfbdfff-9baa-4c2b-aecb-dd622ca404e8', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': '28374a6d-236b-4e03-a764-388189ef6555', 'X-Kubernetes-Pf-Prioritylevel-Uid': '72f430bf-7eb3-49a2-8725-a36c539ac7d7', 'Date': 'Tue, 06 Dec 2022 03:42:56 GMT', 'Content-Length': '363'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"cephnfses.ceph.rook.io is forbidden: User \"system:serviceaccount:openshift-storage:rook-ceph-mgr\" cannot list resource \"cephnfses\" in API group \"ceph.rook.io\" in the namespace \"rook-ceph\"","reason":"Forbidden","details":{"group":"ceph.rook.io","kind":"cephnfses"},"code":403} 

Add RoleBinding

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: manual-rook-ceph-mgr-nfs
  namespace: openshift-storage
  uid: 323d2cc9-042c-425f-917a-6ac04440ce91
  resourceVersion: '691102060'
  creationTimestamp: '2022-12-06T03:45:30Z'
  managedFields:
    - manager: Mozilla
      operation: Update
      apiVersion: rbac.authorization.k8s.io/v1
      time: '2022-12-06T03:45:30Z'
      fieldsType: FieldsV1
      fieldsV1:
        'f:roleRef': {}
        'f:subjects': {}
subjects:
  - kind: ServiceAccount
    name: rook-ceph-mgr
    namespace: openshift-storage
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cephnfses.ceph.rook.io-v1-view