Backward compatibility trustee-operator

plan.md

Multi-Release Backward Compatibility Implementation Plan

Overview

Add support for managing multiple trustee releases through versioned configuration templates. Each TrusteeConfig can select a specific trustee release version, which determines both the configuration templates used and the container image tags deployed.

Design Decisions (Based on User Input)

• ✅ Single TrusteeConfig can select release version via spec.version field
• ✅ Templates organized by trustee release (e.g., config/templates/v0.10.0/)
• ✅ Automatic image selection based on version

Architecture

Template Organization

config/templates/
├── latest/                   # Default templates (current behavior)
│   ├── kbs-config-permissive.toml
│   ├── kbs-config-restricted.toml
│   ├── attestation-policy.rego
│   ├── resource-policy-permissive.rego
│   ├── resource-policy-restrictive.rego
│   ├── rvps-reference-values.json
│   └── tdx-config.json
├── v0.10.0/                  # Trustee v0.10.0 specific templates
│   └── (same 7 files)
├── v0.11.0/                  # Trustee v0.11.0 specific templates
│   └── (same 7 files)
└── versions.json             # Version metadata and image mappings

Version Metadata Format

{
  "supported_versions": ["v0.10.0", "v0.11.0", "latest"],
  "default_version": "latest",
  "version_mappings": {
    "v0.10.0": {
      "kbs_image": "ghcr.io/confidential-containers/key-broker-service:v0.10.0",
      "as_image": "ghcr.io/confidential-containers/attestation-service:v0.10.0",
      "rvps_image": "ghcr.io/confidential-containers/reference-value-provider-service:v0.10.0"
    },
    "latest": {
      "kbs_image": "ghcr.io/confidential-containers/key-broker-service:latest",
      "as_image": "ghcr.io/confidential-containers/attestation-service:latest",
      "rvps_image": "ghcr.io/confidential-containers/reference-value-provider-service:latest"
    }
  }
}

Image Selection Precedence

1. Environment variables (highest priority) - for testing/dev overrides
2. Version-based images from versions.json - production usage
3. Default constants - fallback

Implementation Plan

Phase 1: Foundation

Step 1.1: Add API Version Field

File: api/v1alpha1/kbsconfig_types.go (line ~225)

Add to TrusteeConfigSpec:

// Version specifies the trustee release version to deploy
// Determines which configuration templates and container images to use
// Valid values: "v0.10.0", "v0.11.0", or "latest"
// Defaults to "latest" if not specified
// +optional
// +kubebuilder:validation:Pattern=`^(v[0-9]+\.[0-9]+\.[0-9]+|latest)$`
Version string `json:"version,omitempty"`

Run make generate and make manifests to update CRDs.

Step 1.2: Create Template Directory Structure

• Create config/templates/latest/ directory
• Copy existing 7 template files to latest/
• Create config/templates/versions.json with metadata
• Update Dockerfile if needed (already copies entire directory)

Step 1.3: Create Template Manager Module

New file: internal/controller/template_manager.go

Core responsibilities:

• Load and validate version from versions.json
• Construct version-specific template paths
• Load templates with error handling and fallback to "latest"
• Provide image tag mappings based on version
• Validate requested version is supported

Key functions:

type TemplateManager struct {
    version string
    basePath string
    metadata VersionMetadata
}

func NewTemplateManager(version string) (*TemplateManager, error)
func (tm *TemplateManager) LoadTemplate(templateName string) (string, error)
func (tm *TemplateManager) GetImageVersions() (kbs, as, rvps string, err error)
func (tm *TemplateManager) ValidateVersion() error

New file: internal/controller/template_manager_test.go

• Test version validation
• Test template loading for each version
• Test image mapping
• Test fallback behavior

Phase 2: Controller Integration

Step 2.1: Update TrusteeConfig Controller

File: internal/controller/trusteeconfig_controller.go

Change 1: Add field to reconciler struct (line ~41)

type TrusteeConfigReconciler struct {
    client.Client
    Scheme          *runtime.Scheme
    trusteeConfig   *confidentialcontainersorgv1alpha1.TrusteeConfig
    log             logr.Logger
    namespace       string
    templateManager *TemplateManager  // NEW
}

Change 2: Initialize template manager in buildKbsConfigSpec() (line ~332)

// Get version from TrusteeConfig, default to "latest"
version := r.trusteeConfig.Spec.Version
if version == "" {
    version = "latest"
}

// Create version-aware template manager
templateMgr, err := NewTemplateManager(version)
if err != nil {
    r.log.Error(err, "Failed to create template manager, using latest")
    templateMgr, _ = NewTemplateManager("latest")
}
r.templateManager = templateMgr

Change 3: Update createOrUpdateKbsConfig() to pass version info (line ~144)

// Get image versions from template manager
kbsImage, asImage, rvpsImage, err := r.templateManager.GetImageVersions()

// Create KbsConfig with version annotations
kbsConfig := &confidentialcontainersorgv1alpha1.KbsConfig{
    ObjectMeta: metav1.ObjectMeta{
        Name:      kbsConfigName,
        Namespace: r.namespace,
        Annotations: map[string]string{
            "trustee-operator.confidentialcontainers.org/version": version,
            "trustee-operator.confidentialcontainers.org/kbs-image": kbsImage,
            "trustee-operator.confidentialcontainers.org/as-image": asImage,
            "trustee-operator.confidentialcontainers.org/rvps-image": rvpsImage,
        },
    },
    Spec: spec,
}

Step 2.2: Convert Helper Functions to Methods

Files:

• internal/controller/resource_policy_helper.go (lines 24-59)
• internal/controller/rvps_helper.go (lines 23-34)
• internal/controller/tdx_helper.go (lines 23-34)

Change standalone functions to receiver methods on TrusteeConfigReconciler:

// Before: func generateResourcePolicyRego(profileType string)
// After:  func (r *TrusteeConfigReconciler) generateResourcePolicyRego(profileType string)

// Use template manager:
return r.templateManager.LoadTemplate("resource-policy-restrictive.rego")

Update generateKbsTomlConfig() in trusteeconfig_controller.go (line ~568) similarly.

Phase 3: Image Selection

Step 3.1: Update KbsConfig Controller

File: internal/controller/kbsconfig_controller.go

Add helper function:

func (r *KbsConfigReconciler) getImageForVersion(envVar, defaultImage, versionedImage string) string {
    // 1. Environment variable (highest priority)
    if envImage := os.Getenv(envVar); envImage != "" {
        return envImage
    }

    // 2. Versioned image from annotation (medium priority)
    if versionedImage != "" {
        return versionedImage
    }

    // 3. Default constant (lowest priority)
    return defaultImage
}

Update container builders (lines 602, 635, 670):

func (r *KbsConfigReconciler) buildAsContainer(...) corev1.Container {
    versionedImage := r.kbsConfig.Annotations["trustee-operator.confidentialcontainers.org/as-image"]
    asImageName := r.getImageForVersion("AS_IMAGE_NAME", DefaultAsImageName, versionedImage)
    // ... rest of function
}

Similar changes for buildKbsContainer() and buildRvpsContainer().

Phase 4: Testing

Unit Tests

• template_manager_test.go: Test all template manager functions
• trusteeconfig_controller_test.go: Test version handling, default behavior, image selection
• Test backward compatibility (no version specified → uses latest)

Integration Tests

New directory: test/e2e/multi-version/

Test cases:

1. Deploy with version: v0.10.0, verify v0.10.0 images used
2. Deploy with version: latest, verify latest images
3. Deploy without version field, verify defaults to latest
4. Update version, verify ConfigMaps preserved but deployment updated
5. Invalid version handling

Phase 5: Documentation & Examples

Update Documentation

• CLAUDE.md: Document new version field, template structure, version management
• README.md: Add multi-version support section
• config/samples/: Add examples with version field

Migration Guide

• No action required for existing deployments (defaults to "latest")
• To pin version: Add spec.version: v0.11.0 to TrusteeConfig
• To upgrade versions: Update version field, optionally delete ConfigMaps to regenerate with new templates

Phase 6: Add Versioned Templates

For each trustee release (v0.10.0, v0.11.0, etc.):

1. Create config/templates/vX.Y.Z/ directory
2. Copy 7 template files from that release
3. Update versions.json with version and image mappings
4. Test deployment with that version

Critical Files

File	Changes	Lines
api/v1alpha1/kbsconfig_types.go	Add version field to TrusteeConfigSpec	~225
internal/controller/template_manager.go	NEW: Version-aware template loading	All
internal/controller/trusteeconfig_controller.go	Add templateManager field, initialize it, pass version to KbsConfig	41, 144, 332, 568+
internal/controller/kbsconfig_controller.go	Read version annotations, update image selection	602-698
internal/controller/resource_policy_helper.go	Convert to methods, use template manager	24-59
internal/controller/rvps_helper.go	Convert to methods, use template manager	23-34
internal/controller/tdx_helper.go	Convert to methods, use template manager	23-34
config/templates/versions.json	NEW: Version metadata and image mappings	All

Verification

After implementation:

1. Build & Deploy:

   make docker-build IMG=<registry>/trustee-operator:test
   make docker-push IMG=<registry>/trustee-operator:test
   make deploy IMG=<registry>/trustee-operator:test

2. Test Default Behavior (backward compatibility):

   # Deploy without version field
   kubectl apply -f config/samples/microservices/trusteeconfig_sample.yaml
   # Verify uses "latest" templates and images
   kubectl get deployment -o yaml | grep image:

3. Test Version Selection:

   # Deploy with specific version
   cat <<EOF | kubectl apply -f -
   apiVersion: confidentialcontainers.org/v1alpha1
   kind: TrusteeConfig
   metadata:
     name: test-v0100
   spec:
     version: v0.10.0
     profileType: Permissive
   EOF
   
   # Verify correct images
   kubectl get deployment -o yaml | grep "v0.10.0"

4. Test Version Upgrade:

   # Update version
   kubectl patch trusteeconfig test-v0100 --type=merge -p '{"spec":{"version":"v0.11.0"}}'
   # Verify ConfigMaps preserved
   kubectl get configmap -o yaml | grep "kbs-config"
   # Verify deployment uses new images
   kubectl get deployment -o yaml | grep "v0.11.0"

5. Run Tests:

   make test          # Unit tests
   make test-e2e      # Integration tests

Backward Compatibility

✅ Fully backward compatible:

• Existing TrusteeConfigs without version field continue working
• Default behavior: use "latest" templates and images
• ConfigMaps never regenerated (current preservation logic maintained)
• Environment variable overrides still work
• No breaking changes to API

Overall looks good.
I have few questions to better understand some aspects

• Is my understanding correct that if there is a need for an updated version eg 0.11.1, then a new TrusteeConfig should be created?
• If no version is used (default to latest), then on upgrading the operator, the artefacts for the latest version will get updated ?

These are my thoughts. When a new trustee release is published, there is a new container image(s) and possible changes to trustee config files.

How do we handle this in the trustee-operator?

• new version in the version metadata file (pointing to new image)
• new template directory with updated configuration.

Then we need to release a new operator version, making sure to create all the needed TrusteeConfig (one instance for each supported coco release). We'd also need to have multiple services (and different ports).

My only concern is to create yet another template dir even if there are no breaking changes to configuration files