Multi-tenancy in trustee-operator

plan.md

Multi-Tenancy Implementation Plan for trustee-operator

Overview

Enable multi-tenancy in the trustee-operator so each tenant can create a TrusteeConfig in their namespace and get a separate, isolated trustee instance.

Current Architecture

Controllers

• KbsConfigReconciler: Watches KbsConfig CRs, creates Deployments/Services for trustee
• TrusteeConfigReconciler: Watches TrusteeConfig CRs, generates ConfigMaps/Secrets, creates KbsConfig

Current State

• Both CRDs (KbsConfig, TrusteeConfig) are namespaced resources
• Operator has cluster-wide RBAC permissions (ClusterRole)
• Resources are created in the same namespace as the CR
• Owner references properly set for garbage collection

Critical Limitation

File: internal/controller/kbsconfig_controller.go (lines 827-844)

The KbsConfigReconciler has namespace predicates that restrict watching:

func (r *KbsConfigReconciler) SetupWithManager(mgr ctrl.Manager) error {
    // Line 806-809: Sets r.namespace from POD_NAMESPACE or defaults to "trustee-operator-system"
    r.namespace = os.Getenv("POD_NAMESPACE")
    if r.namespace == "" {
        r.namespace = KbsOperatorNamespace
    }

    // Lines 834, 839: Namespace predicates BLOCK multi-tenancy
    return ctrl.NewControllerManagedBy(mgr).
        For(&confidentialcontainersorgv1alpha1.KbsConfig{}).
        Watches(
            &corev1.ConfigMap{},
            handler.EnqueueRequestsFromMapFunc(configMapMapper),
            builder.WithPredicates(namespacePredicate(r.namespace)), // ⚠️ BLOCKS
        ).
        Watches(
            &corev1.Secret{},
            handler.EnqueueRequestsFromMapFunc(secretMapper),
            builder.WithPredicates(namespacePredicate(r.namespace)), // ⚠️ BLOCKS
        ).
        Complete(r)
}

This limits the controller to only watch ConfigMaps/Secrets in the operator's namespace (trustee-operator-system), preventing multi-tenancy.

Design Decisions

Based on user requirements:

1. Pod Security Labels: Tenant responsibility

   • Tenants must configure their namespace with appropriate pod-security.kubernetes.io labels
   • Simpler operator implementation, clearer ownership boundaries

2. Instances per Namespace: One TrusteeConfig per namespace

   • Uses fixed resource names (trustee-deployment, kbs-service)
   • Simpler implementation, sufficient for multi-tenant use cases

3. Tenant RBAC: Provide example manifests

   • Include sample Role/RoleBinding in config/samples/rbac/
   • Shows how to grant tenant users permission to create TrusteeConfigs

Implementation Approach

Core Strategy

Remove namespace predicates from KbsConfigReconciler and use the namespace from the reconciliation request (req.Namespace) instead of a fixed field.

Benefits

• Each tenant namespace can create TrusteeConfig/KbsConfig CRs
• All resources (Deployments, Services, ConfigMaps, Secrets) remain in tenant namespace
• Kubernetes RBAC naturally enforces tenant isolation
• Backward compatible - existing deployments in trustee-operator-system continue working

Security & Isolation

• Owner references ensure garbage collection per namespace
• RBAC already cluster-wide but resource creation is namespaced
• Each tenant's trustee instance isolated in their namespace
• No cross-namespace resource sharing

Detailed Code Changes

1. KbsConfigReconciler Controller

File: internal/controller/kbsconfig_controller.go

a) Remove namespace field (line 51)

type KbsConfigReconciler struct {
    client.Client
    Scheme    *runtime.Scheme
    kbsConfig *confidentialcontainersorgv1alpha1.KbsConfig
    log       logr.Logger
    // REMOVE: namespace string  ⬅️ Delete this line
}

b) Update Reconcile method (~line 76)

func (r *KbsConfigReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    r.log.Info("Reconciling KbsConfig", "namespace", req.Namespace, "name", req.Name)

    // Use namespace from request instead of r.namespace field
    namespace := req.Namespace

    // Pass namespace to all helper methods
    // ...
}

c) Update SetupWithManager (lines 803-844)

Remove namespace initialization and predicates:

func (r *KbsConfigReconciler) SetupWithManager(mgr ctrl.Manager) error {
    // REMOVE lines 806-809:
    // r.namespace = os.Getenv("POD_NAMESPACE")
    // if r.namespace == "" {
    //     r.namespace = KbsOperatorNamespace
    // }

    // Create a logr instance
    r.log = ctrl.Log.WithName("kbsconfig-controller")
    // Remove namespace from log values: r.log = r.log.WithValues("kbsconfig", r.namespace)

    configMapMapper, err := configMapToKbsConfigMapper(r.Client, r.log)
    if err != nil {
        return err
    }

    secretMapper, err := secretToKbsConfigMapper(r.Client, r.log)
    if err != nil {
        return err
    }

    return ctrl.NewControllerManagedBy(mgr).
        For(&confidentialcontainersorgv1alpha1.KbsConfig{}).
        Watches(
            &corev1.ConfigMap{},
            handler.EnqueueRequestsFromMapFunc(configMapMapper),
            // REMOVE: builder.WithPredicates(namespacePredicate(r.namespace)),
        ).
        Watches(
            &corev1.Secret{},
            handler.EnqueueRequestsFromMapFunc(secretMapper),
            // REMOVE: builder.WithPredicates(namespacePredicate(r.namespace)),
        ).
        Owns(&corev1.ConfigMap{}).
        Owns(&corev1.Secret{}).
        Complete(r)
}

d) Update all methods to accept namespace parameter

Add namespace string parameter to:

Method	Current Line	Change
finalizeKbsConfig	~149	Add namespace string param
deployOrUpdateKbsService	~170	Add namespace string param
newKbsService	~218	Add namespace string param
deployOrUpdateKbsDeployment	~258	Add namespace string param
newKbsDeployment	~315	Add namespace string param
updateKbsDeployment	~780	Add namespace string param
updateKbsConfigStatus	~966	Add namespace string param

Replace all instances of r.namespace with namespace parameter in these methods.

e) Update all method calls in Reconcile

Update all calls to the above methods to pass the namespace variable.

2. Volume Management

File: internal/controller/volumes.go

Update all volume creation methods to accept namespace parameter:

Methods to update:

// Line ~39
func (r *KbsConfigReconciler) createSecretVolume(
    ctx context.Context,
    volumeName string,
    secretName string,
    namespace string,  // ⬅️ Add parameter
) (*corev1.Volume, error) {
    r.log.Info("Retrieving details for", "Secret.Name", secretName, "Secret.Namespace", namespace)
    foundSecret := &corev1.Secret{}
    err := r.Get(ctx, client.ObjectKey{
        Namespace: namespace,  // ⬅️ Use parameter instead of r.namespace
        Name:      secretName,
    }, foundSecret)
    // ... rest unchanged
}

// Line ~66
func (r *KbsConfigReconciler) createKbsSecretResourcesVolume(
    ctx context.Context,
    namespace string,  // ⬅️ Add parameter
) ([]corev1.Volume, error) {
    // Update all r.namespace references to namespace parameter
}

// Line ~93
func (r *KbsConfigReconciler) createConfigMapVolume(
    ctx context.Context,
    volumeName string,
    configMapName string,
    namespace string,  // ⬅️ Add parameter
) (*corev1.Volume, error) {
    r.log.Info("Retrieving details for", "ConfigMap.Name", configMapName, "ConfigMap.Namespace", namespace)
    err := r.Get(ctx, client.ObjectKey{
        Namespace: namespace,  // ⬅️ Use parameter instead of r.namespace
        Name:      configMapName,
    }, foundConfigMap)
    // ... rest unchanged
}

Verify all other volume-related methods and update as needed.

3. Sample RBAC for Tenants

Create example RBAC manifests showing how to grant tenant users permissions.

File: config/samples/rbac/tenant-role.yaml (NEW)

# Example Role for tenant users to create and manage TrusteeConfigs
# Apply this in each tenant namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: trustee-config-creator
  namespace: <tenant-namespace>  # Replace with actual namespace
rules:
# Permission to create and manage TrusteeConfigs
- apiGroups: ["confidentialcontainers.org"]
  resources: ["trusteeconfigs"]
  verbs: ["create", "get", "list", "watch", "update", "patch", "delete"]
# Permission to view generated resources
- apiGroups: [""]
  resources: ["secrets", "configmaps"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get", "list", "watch"]

File: config/samples/rbac/tenant-rolebinding.yaml (NEW)

# Example RoleBinding to grant a user the trustee-config-creator role
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: <user>-trustee-config-creator
  namespace: <tenant-namespace>  # Replace with actual namespace
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: trustee-config-creator
subjects:
- kind: User
  name: <tenant-user>  # Replace with actual username
  apiGroup: rbac.authorization.k8s.io
# Or use ServiceAccount:
# - kind: ServiceAccount
#   name: <service-account-name>
#   namespace: <tenant-namespace>

File: config/samples/rbac/README.md (NEW)

# Tenant RBAC Examples

This directory contains example RBAC manifests for granting tenant users permission to create TrusteeConfigs in their namespace.

## Setup

1. Create tenant namespace:
   ```bash
   kubectl create namespace tenant-a

2. Label namespace for Pod Security Admission:

   kubectl label namespace tenant-a \
     pod-security.kubernetes.io/enforce=privileged \
     pod-security.kubernetes.io/audit=privileged \
     pod-security.kubernetes.io/warn=privileged

3. Apply tenant Role:

   sed 's/<tenant-namespace>/tenant-a/g' tenant-role.yaml | kubectl apply -f -

4. Apply tenant RoleBinding:

   sed -e 's/<tenant-namespace>/tenant-a/g' \
       -e 's/<user>/alice/g' \
       -e 's/<tenant-user>/alice@example.com/g' \
       tenant-rolebinding.yaml | kubectl apply -f -

5. Tenant can now create TrusteeConfig in their namespace

Using ServiceAccount

To grant permissions to a ServiceAccount instead of a user:

kubectl create serviceaccount tenant-sa -n tenant-a
sed -e 's/<tenant-namespace>/tenant-a/g' \
    -e 's/kind: User/kind: ServiceAccount/g' \
    -e 's/name: <tenant-user>/name: tenant-sa/g' \
    -e '/apiGroup: rbac.authorization.k8s.io/d' \
    tenant-rolebinding.yaml | kubectl apply -f -

### 4. Documentation Updates

#### File: `CLAUDE.md`

Add multi-tenancy section after the "Architecture" section:

```markdown
## Multi-Tenancy Support

The trustee-operator supports multi-tenancy, allowing each tenant namespace to deploy its own TrusteeConfig and get an isolated trustee instance.

### Prerequisites

**1. Namespace Pod Security Labels**

If your cluster enforces Pod Security Admission, the tenant namespace must have appropriate labels:

```bash
kubectl label namespace <tenant-namespace> \
  pod-security.kubernetes.io/enforce=privileged \
  pod-security.kubernetes.io/audit=privileged \
  pod-security.kubernetes.io/warn=privileged

2. User Permissions

Tenant users need RBAC permissions to create TrusteeConfigs. See config/samples/rbac/ for example Role/RoleBinding manifests.

Creating a Tenant Instance

# 1. Create tenant namespace
kubectl create namespace my-tenant

# 2. Label namespace for Pod Security
kubectl label namespace my-tenant pod-security.kubernetes.io/enforce=privileged

# 3. Grant user permissions (cluster admin)
kubectl apply -f config/samples/rbac/tenant-role.yaml
kubectl apply -f config/samples/rbac/tenant-rolebinding.yaml

# 4. Create TrusteeConfig (tenant user)
kubectl apply -f config/samples/microservices/trustee-config.yaml -n my-tenant

Tenant Isolation

• Namespace Scoping: All resources (Deployments, Services, ConfigMaps, Secrets) created in the tenant namespace
• No Cross-Namespace Access: ConfigMaps/Secrets must be in the same namespace as the TrusteeConfig
• RBAC Enforcement: Kubernetes RBAC naturally enforces tenant isolation
• Resource Limits: One TrusteeConfig per namespace (uses fixed resource names: trustee-deployment, kbs-service)

Multi-Tenancy Architecture

┌─────────────────────────────────────────────────┐
│          trustee-operator-system                │
│  ┌──────────────────────────────────────────┐   │
│  │   trustee-operator (cluster-wide)        │   │
│  │   Watches: KbsConfig, TrusteeConfig      │   │
│  │   Scope: All namespaces                  │   │
│  └──────────────────────────────────────────┘   │
└─────────────────────────────────────────────────┘
         │                    │
         │ Watches            │ Watches
         ▼                    ▼
┌─────────────────┐  ┌─────────────────┐
│   tenant-a      │  │   tenant-b      │
│  ┌───────────┐  │  │  ┌───────────┐  │
│  │TrusteeConf│  │  │  │TrusteeConf│  │
│  └───────────┘  │  │  └───────────┘  │
│  ┌───────────┐  │  │  ┌───────────┐  │
│  │Deployment │  │  │  │Deployment │  │
│  │ConfigMaps │  │  │  │ConfigMaps │  │
│  │Secrets    │  │  │  │Secrets    │  │
│  │Service    │  │  │  │Service    │  │
│  └───────────┘  │  │  └───────────┘  │
└─────────────────┘  └─────────────────┘
  Isolated             Isolated

Troubleshooting Multi-Tenant Deployments

Issue: TrusteeConfig created but no deployment appears

• Check namespace has pod-security labels: kubectl get namespace <ns> -o yaml | grep pod-security
• Verify operator is running: kubectl get pods -n trustee-operator-system
• Check operator logs: kubectl logs -n trustee-operator-system deployment/trustee-operator-controller-manager

Issue: ConfigMap or Secret not found

• Ensure ConfigMaps/Secrets are in the same namespace as the TrusteeConfig
• Cross-namespace references are not supported

Issue: Permission denied when creating TrusteeConfig

• Verify RBAC: kubectl auth can-i create trusteeconfigs -n <namespace> --as=<user>
• Apply tenant Role/RoleBinding from config/samples/rbac/

#### File: `docs/multi-tenancy.md` (NEW)

Create comprehensive multi-tenancy guide:

```markdown
# Multi-Tenancy Guide for trustee-operator

## Overview

The trustee-operator supports multi-tenancy, enabling multiple isolated tenant namespaces to each deploy their own TrusteeConfig and receive a separate trustee instance.

## Architecture

### Design Principles

1. **Namespace Isolation**: Each tenant operates in their own Kubernetes namespace
2. **Resource Scoping**: All resources created within the tenant namespace
3. **RBAC Boundaries**: Tenants only have permissions in their namespace
4. **One Instance per Namespace**: Each namespace supports one TrusteeConfig (fixed resource names)

### How It Works

The trustee-operator runs in the `trustee-operator-system` namespace with cluster-wide RBAC permissions but creates resources in the namespace where the TrusteeConfig is created.

When a TrusteeConfig is created in namespace `tenant-a`:
1. TrusteeConfigReconciler detects the CR in `tenant-a`
2. Generates ConfigMaps and Secrets in `tenant-a`
3. Creates a KbsConfig in `tenant-a`
4. KbsConfigReconciler creates Deployment and Service in `tenant-a`
5. All resources are owned by the TrusteeConfig (garbage collected together)

## Tenant Onboarding

### Step 1: Cluster Administrator Setup

As a cluster administrator, prepare the tenant namespace:

```bash
# Create namespace
kubectl create namespace tenant-a

# Label for Pod Security Admission (required for privileged pods)
kubectl label namespace tenant-a \
  pod-security.kubernetes.io/enforce=privileged \
  pod-security.kubernetes.io/audit=privileged \
  pod-security.kubernetes.io/warn=privileged

# Create tenant RBAC
cat <<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: trustee-config-creator
  namespace: tenant-a
rules:
- apiGroups: ["confidentialcontainers.org"]
  resources: ["trusteeconfigs"]
  verbs: ["create", "get", "list", "watch", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["secrets", "configmaps"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: alice-trustee-config-creator
  namespace: tenant-a
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: trustee-config-creator
subjects:
- kind: User
  name: alice@example.com
  apiGroup: rbac.authorization.k8s.io
EOF

Step 2: Tenant User Deployment

As the tenant user (alice@example.com):

# Create TrusteeConfig
cat <<EOF | kubectl apply -f -
apiVersion: confidentialcontainers.org/v1alpha1
kind: TrusteeConfig
metadata:
  name: trustee-config
  namespace: tenant-a
spec:
  deploymentType: MicroservicesDeployment
  attestationPolicy:
    type: Permissive
  resourcePolicy:
    type: Permissive
EOF

# Verify deployment
kubectl get trusteeconfig -n tenant-a
kubectl get deployments -n tenant-a
kubectl get services -n tenant-a

Step 3: Verification

# Check TrusteeConfig status
kubectl get trusteeconfig -n tenant-a -o yaml

# Check generated resources
kubectl get configmaps -n tenant-a
kubectl get secrets -n tenant-a
kubectl get kbsconfig -n tenant-a

# Check trustee deployment
kubectl get pods -n tenant-a -l app=kbs
kubectl logs -n tenant-a -l app=kbs

Multiple Tenants Example

Deploy three isolated tenants:

for tenant in tenant-a tenant-b tenant-c; do
  # Create namespace
  kubectl create namespace $tenant

  # Label for PSA
  kubectl label namespace $tenant pod-security.kubernetes.io/enforce=privileged

  # Create RBAC (substitute actual users)
  sed "s/tenant-a/$tenant/g" config/samples/rbac/tenant-role.yaml | kubectl apply -f -

  # Deploy TrusteeConfig
  kubectl apply -f config/samples/microservices/trustee-config.yaml -n $tenant
done

# Verify all tenants
kubectl get trusteeconfig --all-namespaces
kubectl get deployments --all-namespaces -l app=kbs

Each tenant now has:

• Isolated trustee-deployment in their namespace
• Separate kbs-service endpoint
• Independent ConfigMaps and Secrets
• Own KbsConfig CR

Security Considerations

Namespace Isolation

• No cross-namespace access: ConfigMaps and Secrets referenced by KbsConfig must be in the same namespace
• Owner references: All resources have owner references to the TrusteeConfig, ensuring proper garbage collection
• RBAC enforcement: Tenants cannot access resources in other namespaces

RBAC Best Practices

1. Principle of Least Privilege: Grant only necessary permissions
2. Namespace-scoped Roles: Use Role (not ClusterRole) for tenant users
3. ServiceAccount for automation: Use ServiceAccounts with appropriate Roles for CI/CD

Example ServiceAccount setup:

kubectl create serviceaccount tenant-deployer -n tenant-a
kubectl create rolebinding tenant-deployer-binding \
  --role=trustee-config-creator \
  --serviceaccount=tenant-a:tenant-deployer \
  -n tenant-a

Pod Security

Trustee pods require privileged security context. Ensure:

• Namespace has pod-security.kubernetes.io/enforce=privileged label
• Or configure PodSecurityPolicy/PodSecurity to allow privileged pods
• Document this requirement clearly to tenants

Resource Management

Resource Quotas

Limit tenant resource consumption:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-quota
  namespace: tenant-a
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 8Gi
    limits.cpu: "8"
    limits.memory: 16Gi
    persistentvolumeclaims: "5"

LimitRanges

Set default resource limits:

apiVersion: v1
kind: LimitRange
metadata:
  name: tenant-limits
  namespace: tenant-a
spec:
  limits:
  - max:
      cpu: "2"
      memory: 4Gi
    min:
      cpu: "100m"
      memory: 128Mi
    type: Container

Troubleshooting

Common Issues

1. Deployment not created

# Check TrusteeConfig status
kubectl get trusteeconfig -n tenant-a -o yaml

# Check operator logs
kubectl logs -n trustee-operator-system deployment/trustee-operator-controller-manager

# Verify namespace labels
kubectl get namespace tenant-a -o yaml | grep pod-security

2. ConfigMap not found errors

Ensure ConfigMaps are in the same namespace:

# List ConfigMaps in tenant namespace
kubectl get configmaps -n tenant-a

# Check KbsConfig references
kubectl get kbsconfig -n tenant-a -o yaml

3. Permission denied

Verify RBAC:

# Check permissions
kubectl auth can-i create trusteeconfigs -n tenant-a --as=alice@example.com

# Review RoleBindings
kubectl get rolebindings -n tenant-a
kubectl describe rolebinding alice-trustee-config-creator -n tenant-a

4. Pod Security admission errors

# Verify namespace labels
kubectl get namespace tenant-a -o jsonpath='{.metadata.labels}' | jq

# Add labels if missing
kubectl label namespace tenant-a pod-security.kubernetes.io/enforce=privileged

Debug Workflow

1. Check TrusteeConfig:

   kubectl get trusteeconfig -n tenant-a -o yaml

2. Check operator logs:

   kubectl logs -n trustee-operator-system -l control-plane=controller-manager --tail=100

3. Check generated resources:

   kubectl get kbsconfig,configmaps,secrets,deployments,services -n tenant-a

4. Check pod status:

   kubectl get pods -n tenant-a
   kubectl describe pod <pod-name> -n tenant-a

Limitations

1. One TrusteeConfig per namespace: Resource names are fixed (trustee-deployment, kbs-service)
2. No cross-namespace references: All ConfigMaps/Secrets must be in the same namespace
3. Privileged pods required: Namespaces must allow privileged pod security context
4. Namespace labels required: Pod Security Admission labels must be set by cluster admin

Migration from Single-Tenant

Existing deployments in trustee-operator-system continue working without changes. To migrate to multi-tenant model:

1. Create dedicated tenant namespace
2. Copy ConfigMaps and Secrets to new namespace
3. Create TrusteeConfig in new namespace
4. Update client applications to use new service endpoint
5. Delete old TrusteeConfig from trustee-operator-system

No operator changes required - multi-tenancy is backward compatible.

#### File: `README.md`

Add to features section:

```markdown
## Features

- Manages trustee (Key Broker Service) lifecycle in Kubernetes
- Supports AllInOneDeployment and MicroservicesDeployment modes
- **Multi-tenancy support**: Each namespace can deploy isolated trustee instances
- Automatic generation of attestation policies and resource policies
- Intel TDX and IBM Secure Execution support
- Disconnected environment support with local certificate caching
- OpenShift proxy configuration support

5. Testing

Unit Tests

Add to internal/controller/kbsconfig_controller_test.go:

// Test multi-namespace reconciliation
func TestKbsConfigMultiNamespace(t *testing.T) {
    // Test that KbsConfig can be reconciled in different namespaces
    // Create KbsConfig in namespace-1
    // Create KbsConfig in namespace-2
    // Verify both create separate Deployments in their respective namespaces
    // Verify no cross-namespace resource access
}

// Test namespace isolation
func TestNamespaceIsolation(t *testing.T) {
    // Test that resources are properly isolated by namespace
    // Create ConfigMap in namespace-1
    // Create KbsConfig in namespace-2 referencing that ConfigMap
    // Verify reconciliation fails with ConfigMap not found error
}

// Test concurrent reconciliation
func TestConcurrentNamespaceReconciliation(t *testing.T) {
    // Test concurrent reconciliation in multiple namespaces
    // Create KbsConfigs in 3 different namespaces simultaneously
    // Verify all reconcile successfully without conflicts
}

E2E Tests

Create test/e2e/multi-tenant/00-install.yaml:

apiVersion: v1
kind: Namespace
metadata:
  name: tenant-test-a
  labels:
    pod-security.kubernetes.io/enforce: privileged
---
apiVersion: v1
kind: Namespace
metadata:
  name: tenant-test-b
  labels:
    pod-security.kubernetes.io/enforce: privileged
---
# Deploy TrusteeConfig in tenant-test-a
apiVersion: confidentialcontainers.org/v1alpha1
kind: TrusteeConfig
metadata:
  name: trustee-config
  namespace: tenant-test-a
spec:
  deploymentType: MicroservicesDeployment
  attestationPolicy:
    type: Permissive
---
# Deploy TrusteeConfig in tenant-test-b
apiVersion: confidentialcontainers.org/v1alpha1
kind: TrusteeConfig
metadata:
  name: trustee-config
  namespace: tenant-test-b
spec:
  deploymentType: MicroservicesDeployment
  attestationPolicy:
    type: Permissive

Create test/e2e/multi-tenant/00-assert.yaml:

# Assert tenant-test-a deployment exists
apiVersion: apps/v1
kind: Deployment
metadata:
  name: trustee-deployment
  namespace: tenant-test-a
status:
  availableReplicas: 1
---
# Assert tenant-test-b deployment exists
apiVersion: apps/v1
kind: Deployment
metadata:
  name: trustee-deployment
  namespace: tenant-test-b
status:
  availableReplicas: 1
---
# Assert tenant-test-a service exists
apiVersion: v1
kind: Service
metadata:
  name: kbs-service
  namespace: tenant-test-a
---
# Assert tenant-test-b service exists
apiVersion: v1
kind: Service
metadata:
  name: kbs-service
  namespace: tenant-test-b

Implementation Order

1. Phase 1: Core Controller Changes

   • Update KbsConfigReconciler in kbsconfig_controller.go
   • Update volume methods in volumes.go
   • Verify compilation

2. Phase 2: Testing

   • Add unit tests
   • Test manually with multiple namespaces
   • Fix any issues discovered

3. Phase 3: RBAC & Documentation

   • Create sample RBAC manifests
   • Update CLAUDE.md
   • Create docs/multi-tenancy.md
   • Update README.md

4. Phase 4: E2E Tests

   • Create E2E test suite
   • Run in CI
   • Verify all tests pass

5. Phase 5: Validation

   • Test in development cluster with 3+ tenant namespaces
   • Verify existing deployments still work
   • Performance testing with multiple tenants

Backward Compatibility

✅ No breaking changes:

• CRD schemas unchanged
• RBAC permissions unchanged
• Existing deployments in trustee-operator-system continue working
• No migration required

Validation Checklist

Before considering implementation complete:

• KbsConfig can be created in any namespace
• TrusteeConfig can be created in any namespace
• Deployments created in correct namespace
• Services created in correct namespace
• ConfigMaps referenced from same namespace only
• Secrets referenced from same namespace only
• Cross-namespace references fail gracefully with clear error
• Deleting TrusteeConfig only deletes resources in same namespace
• Multiple tenant namespaces coexist without conflicts
• Existing deployments in trustee-operator-system still work
• No performance degradation with 5+ tenant namespaces
• All unit tests pass
• All E2E tests pass
• Documentation complete and accurate
• RBAC examples tested

Summary of Files

Modified Files (2)

1. internal/controller/kbsconfig_controller.go - Remove namespace field, update methods
2. internal/controller/volumes.go - Add namespace parameters to volume methods

New Files (8)

3. config/samples/rbac/tenant-role.yaml - Example tenant Role
4. config/samples/rbac/tenant-rolebinding.yaml - Example tenant RoleBinding
5. config/samples/rbac/README.md - RBAC setup guide
6. docs/multi-tenancy.md - Comprehensive multi-tenancy guide
7. test/e2e/multi-tenant/00-install.yaml - E2E test setup
8. test/e2e/multi-tenant/00-assert.yaml - E2E test assertions
9. internal/controller/kbsconfig_controller_test.go - Multi-namespace unit tests (add to existing)
10. Updates to CLAUDE.md and README.md

Risk Assessment

Risk	Likelihood	Impact	Mitigation
Watch performance degradation	Low	Medium	Mapper functions already filter by namespace; monitor metrics
Security isolation breach	Very Low	High	Owner references enforce scoping; RBAC provides isolation
Breaking existing deployments	Very Low	High	Backward compatible; existing predicates removed but behavior preserved
Cross-namespace reference confusion	Medium	Low	Clear error messages; documentation

Next Steps

1. Review and approve this plan
2. Begin implementation with Phase 1 (Core Controller Changes)
3. Iterative testing and refinement
4. Documentation review
5. Release with clear communication about multi-tenancy support