Nike Custom IngressController Setup.md

Nike Custom IngressController Setup

Revert the current situation

Revert only needed on clusters where you already created a custom IngressController.

1. List the services from openshift-ingress namespace:

oc get svc -n openshift-ingress
NAME                             TYPE           CLUSTER-IP       EXTERNAL-IP                                 PORT(S)                      AGE
router-default                   LoadBalancer   xxx.xxx.xxx.xxx  xxxxxxxxxxxxxxxxx.lb.appdomain.cloud        80:xxxxx/TCP,443:xxxxx/TCP   3h28m

2. Pick the loadbalancer hostname from router-default, and update back the default domain with it (default domain has 0000 in it), eg.:

ibmcloud ks ingress domain update -c xxxxxxxxxxxxxxxxxxxx --hostname xxxxxxxxxxxxxxxxx.lb.appdomain.cloud --domain xxx-xxx-xxx-xxxxx-xxxxxxxxxxxxxxxxxxxxx-0000.us-south.containers.appdomain.cloud

3. Wait for the domain is updated and the dig command gives back the right hostname:

dig xxx-xxx-xxx-xxxxx-xxxxxxxxxxxxxxxxxxxxx-0000.us-south.containers.appdomain.cloud +short                                                                                      

xxxxxxxxxxxxxxxxx.lb.appdomain.cloud.
xx.xxx.xxx.xx
xxx.xxx.xxx.xxx

4. Delete the router-default-custom service from openshift-ingress namespace:

oc delete svc -n openshift-ingress router-default-custom

5. Delete the custom-ingress ingresscontroller from openshift-ingress-operator namespace:

oc delete ingresscontroller -n openshift-ingress-operator custom-ingress

6. Delete the router-custom-ingress deployment from openshift-ingress namespace if it is still exist:

oc delete deployment -n openshift-ingress router-custom-ingress

Create custom ingress controller

1. Create a LoadBalancer Service manually in the openshift-ingress namespace:

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: public
    service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-idle-connection-timeout: "910"
  labels:
    app: router
    router: router-custom-ingress
  name: router-custom-ingress
  namespace: openshift-ingress
spec:
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: http
  - name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    ingresscontroller.operator.openshift.io/deployment-ingresscontroller: custom-ingress
  type: LoadBalancer

2. Wait until the LoadBalancer Service has an address assigned:

$ kubectl get services -n openshift-ingress
NAME                    TYPE           CLUSTER-IP      EXTERNAL-IP                            PORT(S)                      AGE
router-custom-ingress   LoadBalancer   xxx.xx.xxx.xx   xxxxxxxxxxxxxxxxx.lb.appdomain.cloud   80:xxxxx/TCP,443:xxxxx/TCP   9h

3. Create a new Ingress domain with the new Loadbalancer address:

ibmcloud ks ingress domain create -c xxxxxxxxxxxxxxxxxxxx --hostname xxxxxxxxxxxxxxxxx.lb.appdomain.cloud --secret-namespace openshift-ingress --domain custom-ingress-domain-example.us-south.containers.appdomain.cloud
OK

Creating custom-ingress-domain-example.us-south.containers.appdomain.cloud

Note:

• if you want to avoid extra steps --secret-namespace openshift-ingress parameter is mandatory in this case!

4. Wait until the domain is ready and the secret has been created:

➜ ic ks ingress domain ls -c xxxxxxxxxxxxxxxxxxxx
OK
Domain                                                                                                  Target(s)                                   Default   Provider   Secret Status   Status
xxx-xxx-xxx-xxxxx-xxxxxxxxxxxxxxxxxxxxx-0000.us-south.containers.appdomain.cloud                        xxxxxxxxxxxxxxxxx.lb.appdomain.cloud        yes       akamai     created         OK
custom-ingress-domain-example.us-south.containers.appdomain.cloud                                       xxxxxxxxxxxxxxxxx.lb.appdomain.cloud        no        akamai     created         OK

5. Check the secret for your domain is ready, and it is in the openshift-ingress namespace:

ic ks ingress secret ls -c xxxxxxxxxxxxxxxxxxxx
OK
Name                                           Namespace           CRN        Expires On                 Domain                                                                             Status    Type
xxx-xxx-xxx-xxxxx-xxxxxxxxxxxxxxxxxxxxx-0000   ibm-cert-store      crn:xxxx   2025-11-12T09:46:49+0000   xxx-xxx-xxx-xxxxx-xxxxxxxxxxxxxxxxxxxxx-0000.us-south.containers.appdomain.cloud   created   TLS
xxx-xxx-xxx-xxxxx-xxxxxxxxxxxxxxxxxxxxx-0000   openshift-ingress   crn:xxxx   2025-11-12T09:46:49+0000   xxx-xxx-xxx-xxxxx-xxxxxxxxxxxxxxxxxxxxx-0000.us-south.containers.appdomain.cloud   created   TLS
custom-ingress-domain-example                  ibm-cert-store      crn:xxxx   2025-11-12T13:06:07+0000   custom-ingress-domain-example.us-south.containers.appdomain.cloud                  created   TLS
custom-ingress-domain-example                  openshift-ingress   crn:xxxx   2025-11-12T13:06:07+0000   custom-ingress-domain-example.us-south.containers.appdomain.cloud                  created   TLS

oc get secret -n openshift-ingress custom-ingress-domain-example
NAME                           TYPE                DATA   AGE
custom-ingress-domain-example  kubernetes.io/tls   2      10m

6. Create the custom-ingress IngressController resource based on this YAML:

apiVersion: operator.openshift.io/v1
kind: IngressController
metadata:
  name: custom-ingress
  namespace: openshift-ingress-operator
spec:
  defaultCertificate:
    name: custom-ingress-domain-szombi
  domain: custom-ingress-domain-szombi.us-south.stg.containers.appdomain.cloud
  endpointPublishingStrategy:
    type: Internal
  logging:
    access:
      destination:
        type: Container
      httpLogFormat: '{"pod":"%H","timestamp":"%t","http_version":"%HV","http_method":"%HM","request_uri": "%HU",   "status_code":%ST,"termination_state":"%ts","tls_version":"%sslv","tls_cipher":"%sslc","client_ip":"%ci",  "client_port":%cp,"idle_time":%Ti,"response_time":%Tr,"session_duration":%Tt,"upstream_connect_time":%Tc, "backend_name":"%b","backend_server":"%s","server_ip":"%si","server_port":%sp,"retries":%rc,"bytes_uploaded":%U, "bytes_read":%B,"host":"%[capture.req.hdr(0)]","cf-ray":"%[capture.req.hdr(1)]"}'
      httpCaptureHeaders:
        request:
        - maxLength: 256
          name: Host
        - maxLength: 32
          name: CF-RAY
  tuningOptions:
    reloadInterval: 0s
    clientTimeout: 905s
    serverTimeout: 905s

7. Verify that the router-custom-ingress IngressController pods are running in the openshift-ingress namespace:

kubectl get pods -n openshift-ingress
NAME                                     READY   STATUS    RESTARTS   AGE
router-custom-ingress-666744d9b5-6ht6s   2/2     Running   0          90m
router-custom-ingress-666744d9b5-72vlm   2/2     Running   0          90m
router-default-6d66df9bc7-b52h7          1/1     Running   0          4h9m
router-default-6d66df9bc7-k7wrr          1/1     Running   0          4h9m

Handle route resources

We provide two methods:

• method 1 will cause disruption in the traffic, but they can modify the original route resources
• method 2 won't cause disruption in the traffic, but they have to temporarily duplicate

The user can decide which way is more suitable for them.

Change host in original route resources - method 1

⚠️ The operation described below will cause disruption in the traffic. Make sure to execute it when no traffic is routed to the cluster! ⚠️

8. Change the spec.host field in route resource which is associated with your application to use the new Ingress domain:

spec:
  host: custom-ingress-domain-example.us-south.containers.appdomain.cloud

⚠️ Note: This point will cause disruption in the traffic until you don't update CIS entry in next step and the CIS changes don't propagate! ⚠️

9. Update your CIS entry to point to the new ingress domain instead of the default one

10. You can validate the requests if you check the logs of the logs container in the new custom router pods.

Create additional route resources - method 2

8. Copy all of you Route resource for your application, and the modified routes should contains the new Ingress domain in the spec.host Example:

Example script:

# !/bin/bash
# Variables
OLD_ROUTE="my-app-svc2"
NAMESPACE="default"
NEW_NAME="my-app-svc2-custom"
NEW_HOST="my-app-svc2.custom-ingress-domain-example.us-south.containers.appdomain.cloud"

# Get, modify, and apply
oc get route "$OLD_ROUTE" -n "$NAMESPACE" -o yaml \
  | yq eval "
      .metadata.name = \"$NEW_NAME\" |
      .spec.host = \"$NEW_HOST\" |
      del(.metadata.uid, .metadata.resourceVersion, .metadata.creationTimestamp, .metadata.managedFields, .status)
    " - \
  | oc apply -f -

Example result:

k get route -n default
NAME                 HOST/PORT                                                                                                                      PATH   SERVICES      PORT   TERMINATION   WILDCARD
my-app-svc2          my-app-svc2.xxx-xxx-xxx-xxxxx-xxxxxxxxxxxxxxxxxxxxx-0000.us-south.containers.appdomain.cloud ... 1 more                               my-app-svc2   8080                 None
my-app-svc2-custom   my-app-svc2.custom-ingress-domain-example.us-south.containers.appdomain.cloud ... 1 more                                              my-app-svc2   8080                 None

9. At this point you can reach you application with your default and the modified route (for routes configured with spec.tls.termination: edge). In my example: With my original route:

curl -I https://my-app-svc2.xxx-xxx-xxx-xxxxx-xxxxxxxxxxxxxxxxxxxxx-0000.us-south.containers.appdomain.cloud/
HTTP/1.1 200 OK

With my custom route:

curl -I https://my-app-svc2.custom-ingress-domain-example.us-south.containers.appdomain.cloud/
HTTP/1.1 200 OK

And you can validate the requests if you check the logs of the logs container in the new custom router pods.

10. For routes configured with spec.tls.termination: passthrough you will need to take extra steps. You need to ensure that the new domain's secret is available in your application's namespace.

• If you are using Secret Manager integration to manage these certs: You can run: ibmcloud ks ingress secret create -c <clusterID> --name <secret-name> --namespace <target-namespace> --cert-crn <crn of your new domain's secret>.

  You can run ibmcloud ks ingress secret ls -c <clusterID> --show-crn to list your secrets and find the crn of your new domain's secret.

  Related docs: https://cloud.ibm.com/docs/containers?topic=containers-secrets#tls-default

• If you are not using Secret Manager: you will need an automation in place to sync your new domain's secret to your namespace.

11. Update the spec.forceFlowHTTPS.secretName field for your Appconnect Integration Server with your secret from the previous step. More details (here)[https://www.ibm.com/docs/en/app-connect/13.0.x?topic=resources-integration-server-reference#crvalues__title__1].

12. Update your CIS entry to point to the new ingress domain instead of the default one. This change shouldn't cause disruption in the traffic, since your application is reachable with the old domain and the new one.

13. You can validate the requests if you check the logs of the logs container in the new custom router pods.

14. If you validated the traffic, and your application is using the new domain (and the new route) you can remove your old route resources.