- Had to run the script twice to chmod a new install script breaking out the IDP install. No changes to logic, just calling a different backend script.
$ export ENABLE_KEYCLOAK_IDP=true
./scripts/deploy-rhoai-stable.sh
## Installing prerequisites
* Installing cert-manager operator...
namespace/cert-manager-operator created
operatorgroup.operators.coreos.com/cert-manager-operator created
subscription.operators.coreos.com/openshift-cert-manager-operator created
* Waiting for Subscription cert-manager-operator/openshift-cert-manager-operator to start setup...
subscription.operators.coreos.com/openshift-cert-manager-operator condition met
* Waiting for Subscription setup to finish setup. CSV = cert-manager-operator.v1.18.0 ...
clusterserviceversion.operators.coreos.com/cert-manager-operator.v1.18.0 condition met
* Installing LWS operator...
namespace/openshift-lws-operator created
operatorgroup.operators.coreos.com/leader-worker-set created
subscription.operators.coreos.com/leader-worker-set created
* Waiting for Subscription openshift-lws-operator/leader-worker-set to start setup...
subscription.operators.coreos.com/leader-worker-set condition met
* Waiting for Subscription setup to finish setup. CSV = leader-worker-set.v1.0.0 ...
clusterserviceversion.operators.coreos.com/leader-worker-set.v1.0.0 condition met
* Setting up LWS instance and letting it deploy asynchronously.
leaderworkersetoperator.operator.openshift.io/cluster created
* Initializing Gateway API provider...
gatewayclass.gateway.networking.k8s.io/openshift-default created
* Waiting for GatewayClass openshift-default to transition to Accepted status...
gatewayclass.gateway.networking.k8s.io/openshift-default condition met
* Installing RHCL operator...
namespace/kuadrant-system created
operatorgroup.operators.coreos.com/kuadrant-operator-group created
subscription.operators.coreos.com/kuadrant-operator created
* Waiting for Subscription kuadrant-system/kuadrant-operator to start setup...
subscription.operators.coreos.com/kuadrant-operator condition met
* Waiting for Subscription setup to finish setup. CSV = rhcl-operator.v1.2.1 ...
clusterserviceversion.operators.coreos.com/rhcl-operator.v1.2.1 condition met
* Setting up RHCL instance...
kuadrant.kuadrant.io/kuadrant created
* Installing RHOAI v3 operator...
gateway.gateway.networking.k8s.io/openshift-ai-inference created
namespace/redhat-ods-operator created
operatorgroup.operators.coreos.com/rhoai3-operatorgroup created
subscription.operators.coreos.com/rhoai3-operator created
* Waiting for Subscription redhat-ods-operator/rhoai3-operator to start setup...
subscription.operators.coreos.com/rhoai3-operator condition met
* Waiting for Subscription setup to finish setup. CSV = rhods-operator.3.0.0 ...
clusterserviceversion.operators.coreos.com/rhods-operator.3.0.0 condition met
* Setting up RHOAI instance and letting it deploy asynchronously.
datasciencecluster.datasciencecluster.opendatahub.io/default-dsc created
## Installing Models-as-a-Service
* Cluster domain: apps.ci-ln-v8c9mib-76ef8.aws-4.ci.openshift.org
* Cluster audience: https://kubernetes.default.svc
* Keycloak IDP enabled: true
* TLS certificate: router-certs-default
* Keycloak issuer: https://keycloak.apps.ci-ln-v8c9mib-76ef8.aws-4.ci.openshift.org/realms/maas
* Keycloak client: maas-cli
* Deploying Keycloak...
namespace/keycloak-system created
deployment.apps/keycloak created
service/keycloak created
route.route.openshift.io/keycloak created
Waiting for deployment "keycloak" rollout to finish: 0 of 1 updated replicas are available...
deployment "keycloak" successfully rolled out
* Configuring Keycloak realm from manifests...
configmap/maas-realm-config created
job.batch/keycloak-realm-import created
namespace/maas-api created
namespace/opendatahub condition met
serviceaccount/maas-api serverside-applied
clusterrole.rbac.authorization.k8s.io/maas-api serverside-applied
clusterrolebinding.rbac.authorization.k8s.io/maas-api serverside-applied
configmap/maas-parameters serverside-applied
configmap/tier-to-group-mapping serverside-applied
service/maas-api serverside-applied
deployment.apps/maas-api serverside-applied
telemetrypolicy.extensions.kuadrant.io/user-group serverside-applied
gateway.gateway.networking.k8s.io/maas-default-gateway serverside-applied
httproute.gateway.networking.k8s.io/maas-api-route serverside-applied
authpolicy.kuadrant.io/maas-api-auth-policy serverside-applied
authpolicy.kuadrant.io/gateway-auth-policy serverside-applied
ratelimitpolicy.kuadrant.io/gateway-rate-limits serverside-applied
tokenratelimitpolicy.kuadrant.io/gateway-token-rate-limits serverside-applied
servicemonitor.monitoring.coreos.com/limitador-metrics serverside-applied
networkpolicy.networking.k8s.io/maas-authorino-allow serverside-applied
* Waiting for gateway deployment to be created...
* Gateway deployment found, reducing CPU requirements for scheduling...
* Configuring Keycloak IDP...
./scripts/deploy-rhoai-stable.sh: line 563: /home/brent/idp/models-as-a-service/scripts/deploy-keycloak-idp.sh: Permission denied
brent@ip-172-31-33-128:~/idp/models-as-a-service$ chmod +x /home/brent/idp/models-as-a-service/scripts/deploy-keycloak-idp.sh
brent@ip-172-31-33-128:~/idp/models-as-a-service$ ./scripts/deploy-rhoai-stable.sh
## Installing prerequisites
* The cert-manager operator is present in the cluster. Skipping installation.
* The LWS operator is present in the cluster. Skipping installation.
* Initializing Gateway API provider...
gatewayclass.gateway.networking.k8s.io/openshift-default unchanged
* Waiting for GatewayClass openshift-default to transition to Accepted status...
gatewayclass.gateway.networking.k8s.io/openshift-default condition met
* The RHCL operator is present in the cluster. Skipping installation.
WARNING: Creating an instance of RHCL is also skipped.
* The RHOAI operator is present in the cluster. Skipping installation.
## Installing Models-as-a-Service
* Cluster domain: apps.ci-ln-v8c9mib-76ef8.aws-4.ci.openshift.org
* Cluster audience: https://kubernetes.default.svc
* Keycloak IDP enabled: true
* TLS certificate: router-certs-default
* Keycloak issuer: https://keycloak.apps.ci-ln-v8c9mib-76ef8.aws-4.ci.openshift.org/realms/maas
* Keycloak client: maas-cli
* Deploying Keycloak...
namespace/keycloak-system unchanged
deployment.apps/keycloak configured
service/keycloak unchanged
route.route.openshift.io/keycloak unchanged
deployment "keycloak" successfully rolled out
* Configuring Keycloak realm from manifests...
configmap/maas-realm-config unchanged
job.batch "keycloak-realm-import" deleted from keycloak-system namespace
job.batch/keycloak-realm-import created
namespace/maas-api unchanged
namespace/opendatahub condition met
serviceaccount/maas-api serverside-applied
clusterrole.rbac.authorization.k8s.io/maas-api serverside-applied
clusterrolebinding.rbac.authorization.k8s.io/maas-api serverside-applied
configmap/maas-parameters serverside-applied
configmap/tier-to-group-mapping serverside-applied
service/maas-api serverside-applied
deployment.apps/maas-api serverside-applied
telemetrypolicy.extensions.kuadrant.io/user-group serverside-applied
gateway.gateway.networking.k8s.io/maas-default-gateway serverside-applied
httproute.gateway.networking.k8s.io/maas-api-route serverside-applied
authpolicy.kuadrant.io/maas-api-auth-policy serverside-applied
authpolicy.kuadrant.io/gateway-auth-policy serverside-applied
ratelimitpolicy.kuadrant.io/gateway-rate-limits serverside-applied
tokenratelimitpolicy.kuadrant.io/gateway-token-rate-limits serverside-applied
servicemonitor.monitoring.coreos.com/limitador-metrics serverside-applied
networkpolicy.networking.k8s.io/maas-authorino-allow serverside-applied
* Waiting for gateway deployment to be created...
* Gateway deployment found, reducing CPU requirements for scheduling...
* Configuring Keycloak IDP...
Deploying Keycloak tier mapping ConfigMap...
configmap/tier-to-group-mapping serverside-applied
networkpolicy.networking.k8s.io/maas-allow-gateway serverside-applied
configmap/tier-to-group-mapping serverside-applied
networkpolicy.networking.k8s.io/maas-allow-gateway serverside-applied
Applying AuthPolicy for Keycloak OIDC...
authpolicy.kuadrant.io/maas-api-auth-policy serverside-applied
✅ AuthPolicy applied
Excluding /maas-api paths from Gateway AuthPolicy...
authpolicy.kuadrant.io/gateway-auth-policy patched
Setting MaaS API image for Keycloak IDP...
* Creating OpenShift Route for Gateway...
route.route.openshift.io/maas-gateway-route created
* Waiting for KServe webhook to be ready (up to 2 minutes)...
* KServe webhook ready
=========================================
Deployment is complete.
Next Steps:
1. Deploy a sample model:
kubectl create namespace llm
kustomize build 'https://github.com/opendatahub-io/models-as-a-service.git/docs/samples/models/simulator?ref=main' | kubectl apply -f -
2. Get Gateway endpoint:
CLUSTER_DOMAIN=$(kubectl get ingresses.config.openshift.io cluster -o jsonpath='{.spec.domain}')
HOST="maas.${CLUSTER_DOMAIN}"
3. Get authentication token:
KEYCLOAK_URL="https://keycloak.${CLUSTER_DOMAIN}"
KEYCLOAK_REALM="maas"
KEYCLOAK_CLIENT_ID="maas-cli"
KEYCLOAK_CLIENT_SECRET="maas-cli-secret"
KEYCLOAK_USERNAME="freeuser1"
KEYCLOAK_PASSWORD="password123"
KEYCLOAK_ACCESS_TOKEN=$(curl -sSk -H "Content-Type: application/x-www-form-urlencoded" -d "grant_type=password" -d "client_id=${KEYCLOAK_CLIENT_ID}" -d "client_secret=${KEYCLOAK_CLIENT_SECRET}" -d "username=${KEYCLOAK_USERNAME}" -d "password=${KEYCLOAK_PASSWORD}" "${KEYCLOAK_URL}/realms/${KEYCLOAK_REALM}/protocol/openid-connect/token" | jq -r .access_token)
TOKEN=$(curl -sSk -H "Authorization: Bearer ${KEYCLOAK_ACCESS_TOKEN}" -H "Content-Type: application/json" -X POST -d '{"expiration": "24h"}' "https://${HOST}/maas-api/v1/tokens" | jq -r .token)
4. List models (Keycloak token) and call inference (MaaS token):
MODELS=$(curl -sSk "https://${HOST}/maas-api/v1/models" -H "Content-Type: application/json" -H "Authorization: Bearer ${KEYCLOAK_ACCESS_TOKEN}")
MODEL_NAME=$(echo $MODELS | jq -r '.data[0].id')
MODEL_URL=$(echo $MODELS | jq -r '.data[0].url')
curl -sSk -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" -d "{\"model\": \"${MODEL_NAME}\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}], \"max_tokens\": 50}" "${MODEL_URL}/v1/chat/completions"
5. Test authorization limiting (no token 401 error):
curl -sSk -H "Content-Type: application/json" -d "{\"model\": \"${MODEL_NAME}\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}], \"max_tokens\": 50}" "${MODEL_URL}/v1/chat/completions" -v
6. Test rate limiting (200 OK followed by 429 Rate Limit Exceeded after about 4 requests):
for i in {1..16}; do curl -sSk -o /dev/null -w "%{http_code}\n" -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" -d "{\"model\": \"${MODEL_NAME}\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}], \"max_tokens\": 50}" "${MODEL_URL}/v1/chat/completions"; done
7. Run validation script (Runs all the checks again):
curl https://raw.githubusercontent.com/opendatahub-io/models-as-a-service/refs/heads/main/scripts/validate-deployment.sh | sh -v -
8. Check metrics generation:
kubectl port-forward -n kuadrant-system svc/limitador-limitador 8080:8080 &
curl http://localhost:8080/metrics | grep -E '(authorized_hits|authorized_calls|limited_calls)'
9. Access Prometheus to view metrics:
kubectl port-forward -n openshift-monitoring svc/prometheus-k8s 9090:9091 &
# Open http://localhost:9090 in browser and search for: authorized_hits, authorized_calls, limited_calls
$ CLUSTER_DOMAIN=$(kubectl get ingresses.config.openshift.io cluster -o jsonpath='{.spec.domain}')
HOST="maas.${CLUSTER_DOMAIN}"
brent@ip-172-31-33-128:~/idp/models-as-a-service$ KEYCLOAK_URL="https://keycloak.${CLUSTER_DOMAIN}"
KEYCLOAK_REALM="maas"
KEYCLOAK_CLIENT_ID="maas-cli"
KEYCLOAK_CLIENT_SECRET="maas-cli-secret"
KEYCLOAK_USERNAME="freeuser1"
KEYCLOAK_PASSWORD="password123"
KEYCLOAK_ACCESS_TOKEN=$(curl -sSk -H "Content-Type: application/x-www-form-urlencoded" -d "grant_type=password" -d "client_id=${KEYCLOAK_CLIENT_ID}" -d "client_secret=${KEYCLOAK_CLIENT_SECRET}" -d "username=${KEYCLOAK_USERNAME}" -d "password=${KEYCLOAK_PASSWORD}" "${KEYCLOAK_URL}/realms/${KEYCLOAK_REALM}/protocol/openid-connect/token" | jq -r .access_token)
TOKEN=$(curl -sSk -H "Authorization: Bearer ${KEYCLOAK_ACCESS_TOKEN}" -H "Content-Type: application/json" -X POST -d '{"expiration": "24h"}' "https://${HOST}/maas-api/v1/tokens" | jq -r .token)
brent@ip-172-31-33-128:~/idp/models-as-a-service$ echo $TOKEN
eyJhbGciOiJSUzI1NiIsImtpZCI6InVIM0VqNlBIYWEyQ3F6Z1IxWTRZYVZLTDZoT0NVTmhGNTJwQW9PcUFUejQifQ.eyJhdWQiOlsibWFhcy1kZWZhdWx0LWdhdGV3YXktc2EiXSwiZXhwIjoxNzY4NTk2MzIwLCJpYXQiOjE3Njg1MDk5MjAsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2YyIsImp0aSI6IjY3MGFiZTg1LTZiYjItNDVjYi05MjMyLTNjZmVmZTQ2YWMzNyIsImt1YmVybmV0ZXMuaW8iOnsibmFtZXNwYWNlIjoibWFhcy1kZWZhdWx0LWdhdGV3YXktdGllci1mcmVlIiwic2VydmljZWFjY291bnQiOnsibmFtZSI6ImZyZWV1c2VyMS02NTc0OGZmMSIsInVpZCI6IjYzODM1NGRkLWI2MDYtNGU3NS04Y2Q1LWRmYzdlYjVkMDQ4ZSJ9fSwibmJmIjoxNzY4NTA5OTIwLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6bWFhcy1kZWZhdWx0LWdhdGV3YXktdGllci1mcmVlOmZyZWV1c2VyMS02NTc0OGZmMSJ9.BntCM0nHISbHJOs2rpGXf7qE3EEidK5pYZeUzDnc3WHpVbmY45bZjPPnl_1OwO7RY9_G32xGW9NdZKtQw4QdAud6doZxcGgpo-7n49D_JcD8DPOQfbg5-Eii78k43OmaxVVNsmIVlSsGpvpwQ23Dtc-7UTtbwEfd83h5l-0wVshiT06AeOvpm4bYHwQYYtCVxAH8_jIj8Dd5I_ZxZMFuGrxKmM9KhcVoQWEsPkqHiJjQhRtU6_MuTRKqizJS17IPLXPcWRuwS7TiY6pA4hVHGX6JymhzI_UmbK5hfqNExJ8GrfCRLSxbbdfCL54eCkqVZciU0j-rbqy_3KvaJUpamW_ncaXY-x4jxIC5OjhyvpE3dMq3NaMBNkc2gqDg3RUpeWt4s8Ov4ow_tE1_m9F77ZKixj6gIvi81fxeP8rPa_tF1oXjaXcpM2t3d0Y0f2LSEy8DNNpuPS2ljwTx8ZJKqsMqhpl56_vwcWcWR9bRtBNu-_8hfQaOfiIkTvhhSvuo3NdTE5NiIBMNvS-UQCN6m1TMVOHa64SmB72Gol2iEdAskGVVFvPFR5b-qhV9FvD0_yUpINfEAnOEI4e9CzejPDtuol4BF1V5ytEO3e0CPAs9o8ooVedrVelQ1imzrlnbzKu9kweaJkYpKYeQJY06gcHlZ_gztaG_EZF5I7Q1Z38
brent@ip-172-31-33-128:~/idp/models-as-a-service$ MODELS=$(curl -sSk "https://${HOST}/maas-api/v1/models" -H "Content-Type: application/json" -H "Authorization: Bearer ${KEYCLOAK_ACCESS_TOKEN}")
MODEL_NAME=$(echo $MODELS | jq -r '.data[0].id')
MODEL_URL=$(echo $MODELS | jq -r '.data[0].url')
curl -sSk -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" -d "{\"model\": \"${MODEL_NAME}\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}], \"max_tokens\": 50}" "${MODEL_URL}/v1/chat/completions"
{"id":"chatcmpl-7dd1ab6a-b010-425c-ae50-4037374b07b9","created":1768509933,"model":"facebook/opt-125m","usage":{"prompt_tokens":1,"completion_tokens":7,"total_tokens":8},"object":"chat.completion","do_remote_decode":false,"do_remote_prefill":false,"remote_block_ids":null,"remote_engine_id":"","remote_host":"","remote_port":0,"choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"I am your AI assistant, how "}}]
