This design describes the architecture for deploying a Rails application on OVH Kubernetes with separate containers for the web server, delayed job workers, and Kafka consumers. The solution uses Docker multi-stage builds to create separate container images for each process type from a single Dockerfile. Each build target contains the appropriate CMD to start its specific process in the foreground with proper signal handling, health checks, and JSON logging for Logz.io integration.
The previous deployment model had a disconnected build and deploy process:
Build Process:
- Developer merges code to master
.github/workflows/cd.ymlbuildsplaywrighttarget → pushed to GHCR (never used)- Operator manually SSHs to build server in
storecove-app-dockerdirectory - Runs
./build-deploy -s true -a true -b masterscript which:- Clones fresh copy of datajust repo
- Builds using
storecove-app-docker/production/Dockerfile - Copies assets to S3 CDN
- Pushes to AWS ECR
Deploy Process:
- Script runs
aws ecs update-service --force-new-deploymentfor 3 services - ECS pulls latest image from ECR
- Starts new tasks with monolithic container
Problems with this approach:
- Manual intervention required
- CI builds wasted (never deployed)
- Different Dockerfile for production vs. CI
- Build-from-scratch on every deploy (slow)
- Assets managed separately on S3
The new model unifies build and deploy into one automated workflow:
Build & Deploy Process:
- Developer merges code to master
.github/workflows/deploy.ymlautomatically triggers- Builds 5 Docker targets from
datajust/Dockerfile - Pushes all images to OVH Container Registry
- Runs migrations using
railstarget - Applies Kubernetes manifests
- Kubernetes performs rolling update
Benefits:
- ✅ Fully automated
- ✅ Same Dockerfile for all environments
- ✅ Images built once, used everywhere
- ✅ Assets served from container
- ✅ Faster builds (layer caching)
- ✅ No manual SSH required
| Component | Status | Replacement |
|---|---|---|
storecove-app-docker/production/Dockerfile |
Deprecated | datajust/Dockerfile with multiple targets |
storecove-app-docker/production/build-deploy |
Deprecated | .github/workflows/deploy.yml |
.github/workflows/cd.yml GHCR images |
Deprecated for prod | .github/workflows/deploy.yml OVH images |
| Manual ECS service restart | Deprecated | Automatic Kubernetes rolling update |
| S3 CDN for assets | Deprecated | Assets served from container |
| Container-level cron (whenever gem) | Deprecated | Kubernetes CronJobs |
graph TB
subgraph "OVH Kubernetes Cluster"
subgraph "Web Tier"
WEB1[Rails Server Pod 1]
WEB2[Rails Server Pod 2]
WEB3[Rails Server Pod N]
end
subgraph "Worker Tier"
DJ1[Worker Primary Pod 1]
DJ2[Worker Primary Pod N]
DJ3[Worker Secondary Pod 1]
DJ4[Worker Secondary Pod N]
end
subgraph "Kafka Consumer Tier"
KS[Sending Status Consumer]
KN[New Document Consumer]
KR[Received Status Consumer]
end
subgraph "Scheduled Tasks"
CRON[Kubernetes CronJobs]
end
subgraph "Logging"
FB[Fluent Bit DaemonSet]
end
ING[Ingress Controller]
SVC[Kubernetes Service]
end
DB[(Database)]
KAFKA[Kafka Brokers]
LOGZ[Logz.io]
ROLLBAR[Rollbar]
ING --> SVC
SVC --> WEB1
SVC --> WEB2
SVC --> WEB3
WEB1 --> DB
DJ1 --> DB
DJ2 --> DB
DJ3 --> DB
DJ4 --> DB
CRON --> DB
KS --> KAFKA
KN --> KAFKA
KR --> KAFKA
FB --> LOGZ
WEB1 -.-> FB
DJ1 -.-> FB
KS -.-> FB
flowchart TD
subgraph "Docker Build"
BASE[Base Stage] --> APP[App Base Stage]
APP --> RAILS[rails target]
APP --> WORKER[worker target]
APP --> KS[kafka-sending-status target]
APP --> KN[kafka-new-document target]
APP --> KR[kafka-received-status target]
end
subgraph "Container Registry"
RAILS --> IMG_RAILS[storecove-app:rails-latest]
WORKER --> IMG_WORKER[storecove-app:worker-latest]
KS --> IMG_KS[storecove-app:kafka-sending-status-latest]
KN --> IMG_KN[storecove-app:kafka-new-document-latest]
KR --> IMG_KR[storecove-app:kafka-received-status-latest]
end
subgraph "Kubernetes Deployments"
IMG_RAILS --> DEP_RAILS[rails-server Deployment]
IMG_WORKER --> DEP_WORKER1[worker-primary Deployment]
IMG_WORKER --> DEP_WORKER2[worker-secondary Deployment]
IMG_KS --> DEP_KS[kafka-sending-status Deployment]
IMG_KN --> DEP_KN[kafka-new-document Deployment]
IMG_KR --> DEP_KR[kafka-received-status Deployment]
end
subgraph "Kubernetes CronJobs"
IMG_RAILS --> CRON1[scheduled-task-1 CronJob]
IMG_RAILS --> CRON2[scheduled-task-2 CronJob]
IMG_RAILS --> CRONN[scheduled-task-N CronJob]
end
The Dockerfile uses multi-stage builds to create optimized images for each component type from a shared base.
# syntax=docker/dockerfile:1-labs
# Base stage with all dependencies
FROM ubuntu:focal AS base
ARG BUNDLER_VERSION=2.6.8
ENV BUNDLER_VERSION=${BUNDLER_VERSION}
ENV DEBIAN_FRONTEND=noninteractive
ENV BUNDLE_PATH=/cache/bundle
ENV YARN_CACHE_FOLDER=/cache/yarn
ENV BUNDLE_SILENCE_ROOT_WARNING=1
SHELL ["/bin/bash", "-l", "-c"]
# ... (existing base setup: apt packages, RVM, Ruby, Node.js, etc.) ...
WORKDIR /app
# Ruby dependencies stage
FROM base AS ruby-deps
USER app
COPY --chown=app:sudo Gemfile Gemfile.lock ./
RUN bash -lc "bundle install"
# Node dependencies stage
FROM base AS node-deps
USER app
COPY --chown=app:sudo package.json yarn.lock ./
RUN bash -lc "yarn install --frozen-lockfile"
# Application base with all code and assets
FROM base AS app-base
USER app
COPY --chown=app:sudo Gemfile Gemfile.lock ./
COPY --from=ruby-deps /cache/bundle /cache/bundle
COPY --chown=app:sudo package.json yarn.lock ./
COPY --from=node-deps /cache/yarn /cache/yarn
COPY --chown=app:sudo . .
RUN yarn install --frozen-lockfile
ENV SECRET_KEY_BASE_DUMMY=1
RUN bash -lc "bundle exec rails assets:precompile"
ENV SECRET_KEY_BASE_DUMMY=0
# Create log directory
RUN mkdir -p /app/log
# ===== Rails Server Target =====
FROM app-base AS rails
EXPOSE 3000
ENV RAILS_SERVE_STATIC_FILES=true
ENV RAILS_LOG_TO_STDOUT=true
ENV PROCESS_TARGET=server
CMD ["bash", "-lc", "bundle exec rails server -b 0.0.0.0 -p 3000"]
# ===== Delayed Job Worker Target =====
# Pool configuration passed via DELAYED_JOB_POOLS environment variable
# Example: DELAYED_JOB_POOLS="--pool=mail:1 --pool=slack:2"
FROM app-base AS worker
EXPOSE 3001
ENV PROCESS_TARGET=worker
ENV DELAYED_JOB_POOLS=""
ENV DELAYED_JOB_TIMEOUT=280
COPY --chown=app:sudo scripts/health_server.rb /scripts/health_server.rb
CMD ["bash", "-lc", "HEALTH_PORT=3001 ruby /scripts/health_server.rb & exec bundle exec bin/delayed_job run --timeout=${DELAYED_JOB_TIMEOUT} $DELAYED_JOB_POOLS"]
# ===== Kafka Sending Status Consumer =====
FROM app-base AS kafka-sending-status
EXPOSE 3002
ENV PROCESS_TARGET=kafka-sending-status
COPY --chown=app:sudo scripts/health_server.rb /scripts/health_server.rb
CMD ["bash", "-lc", "HEALTH_PORT=3002 ruby /scripts/health_server.rb & exec bundle exec racecar --group-id \"$KAFKA_SENDINGSTATUSUPDATE_CONSUMER_GROUP_ID\" --sasl-username \"$KAFKA_SENDINGSTATUSUPDATE_CONSUMER_USERNAME\" --sasl-password \"$KAFKA_SENDINGSTATUSUPDATE_CONSUMER_PASSWORD\" Kafka::Consumers::SendingActionStatusUpdateConsumer"]
# ===== Kafka New Document Consumer =====
FROM app-base AS kafka-new-document
EXPOSE 3003
ENV PROCESS_TARGET=kafka-new-document
COPY --chown=app:sudo scripts/health_server.rb /scripts/health_server.rb
CMD ["bash", "-lc", "HEALTH_PORT=3003 ruby /scripts/health_server.rb & exec bundle exec racecar --group-id \"$KAFKA_NEWDOCUMENTNOTIFICATION_CONSUMER_GROUP\" --sasl-username \"$KAFKA_NEWDOCUMENTNOTIFICATION_CONSUMER_USERNAME\" --sasl-password \"$KAFKA_NEWDOCUMENTNOTIFICATION_CONSUMER_PASSWORD\" Kafka::Consumers::NewDocumentNotificationConsumer"]
# ===== Kafka Received Status Consumer =====
FROM app-base AS kafka-received-status
EXPOSE 3004
ENV PROCESS_TARGET=kafka-received-status
COPY --chown=app:sudo scripts/health_server.rb /scripts/health_server.rb
CMD ["bash", "-lc", "HEALTH_PORT=3004 ruby /scripts/health_server.rb & exec bundle exec racecar --group-id \"$KAFKA_RECEIVEDDOCUMENTSTATUS_CONSUMER_GROUP\" --sasl-username \"$KAFKA_RECEIVEDDOCUMENTSTATUS_CONSUMER_USERNAME\" --sasl-password \"$KAFKA_RECEIVEDDOCUMENTSTATUS_CONSUMER_PASSWORD\" Kafka::Consumers::ReceivedDocumentStatusConsumer"]The worker Docker target uses a configurable DELAYED_JOB_POOLS environment variable, allowing different Kubernetes deployments to run different queue pools from the same image.
| Deployment | DELAYED_JOB_POOLS Value |
|---|---|
| worker-primary | --pool=mail:1 --pool=inboundpeppol,inboundpeppolemail,inboundsftp,inboundublemail,inboundpartneremail:4 --pool=ses_notifications,ses_mail,sar_mail,edi_smtp,edi_as2,ses_mail_in_out:2 --pool=vatcalc_out_out_live,vatcalc_out_out_pilot:1 --pool=analyze_action,invoice_analyzer,slack,apply_action:1 --pool=document_submissions:2 |
| worker-secondary | --pool=smp_phoss:8 --pool=aruba_out_out_prod,aruba_out_out_pilot,aruba_out_out_webhooks_pilot,aruba_out_out_webhooks_prod:1 --pool=chargebee_webhook_events,exactsales_webhook_events,storecove_webhook_events:1 --pool=outgoing_webhooks,outgoing_webhooks_sandbox:4 --pool=outgoing_webhooks_asia,outgoing_webhooks_sandbox_asia:4 --pool=exact_worker,snelstart_worker,sftp_worker,as2_worker:1 --pool=received_documents,aruba_in_in_webhooks:1 --pool=storecove_api_self:3 --pool=active_storage_analysis,active_storage_mirror,active_storage_preview,active_storage_purge:1 --pool=kafka_sending_actions_status_update,kafka_received_document_status,kafka_new_document_notification:12 --pool=meta_events,exceptions,aruba_admin:1 --pool=customer_reporting:1 --pool=my_lhdnm_poller:6 |
This allows:
- Independent scaling of pool groups
- Single Docker image for all workers
- Easy adjustment of pool assignments via K8s manifests
The Rails application uses Puma in single-process, multi-threaded mode (workers commented out in config/puma.rb). This is intentional for the Kubernetes deployment:
- Horizontal Scaling: Multiple pods provide process-level isolation and fault tolerance
- Simpler Failure Mode: If a pod crashes, only one replica is affected
- Resource Predictability: Each pod uses consistent resources (no worker forking)
- Thread Pool: Each pod uses 5 threads (configurable via
RAILS_MAX_THREADS)
Production Configuration:
# config/puma.rb
threads 5, 5 # Default: 5 threads per pod
# workers disabled - scaling via Kubernetes replicas insteadEnvironment Variables:
RAILS_MAX_THREADS- Max threads per pod (default: 5)RAILS_MIN_THREADS- Min threads per pod (default: 5)WEB_CONCURRENCY- Not used (workers disabled)DB_POOL- ActiveRecord connection pool size (should match RAILS_MAX_THREADS)
ActiveRecord Connection Pooling:
The ActiveRecord connection pool size should match the Puma thread count to avoid connection exhaustion. In config/database.yml:
production:
primary:
adapter: mysql2
pool: <%= ENV.fetch("DB_POOL") { ENV.fetch("RAILS_MAX_THREADS") { 5 } } %>
# ... other settings ...For the Rails server with 5 threads per pod and 2-10 replicas, total connections = 5 threads × 10 pods = 50 connections maximum.
Racecar consumers must log to STDOUT for Fluent Bit collection:
# config/initializers/racecar.rb (update for Kubernetes)
Racecar.configure do |config|
# ... existing config ...
# Change from file logging to STDOUT
config.logfile = STDOUT
# Use Rails logger for consistent JSON formatting
config.logger = Rails.logger if Rails.logger
# Offset commit configuration for graceful shutdown
config.offset_commit_interval = 10 # Commit every 10 seconds (default)
config.offset_commit_threshold = 0 # Or commit after every message for max safety
# ... rest of config ...
endSIGTERM Handling: Racecar handles SIGTERM gracefully by default, committing offsets before shutdown.
CMD with bash -lc and exec:
The design uses CMD ["bash", "-lc", "exec bundle exec <command>"] which:
- Starts bash as PID 1
- The
execkeyword replaces bash with the actual process - The actual process (Puma, delayed_job, racecar) receives SIGTERM directly
- All three processes handle SIGTERM gracefully by default:
- Puma: Stops accepting new connections, completes in-flight requests
- delayed_job: Completes current job within timeout, or leaves in queue
- Racecar: Commits offsets and disconnects cleanly
Health Server Background Process:
The health server runs as a background process (&) and won't receive SIGTERM propagation. This is acceptable because:
- When the main process exits (delayed_job or racecar), the container exits
- Kubernetes detects the container exit and restarts it
- The health server is supplementary; container exit is the primary failure detection
Container Exit on Process Failure: If the main process crashes:
- Container exits with non-zero code
- Kubernetes detects exit via container state
- Liveness probe subsequently fails
- Kubernetes restarts the container per the restart policy
A lightweight WEBrick server for worker and Kafka consumer health checks:
#!/usr/bin/env ruby
require 'webrick'
require 'json'
PROCESS_TARGET = ENV.fetch('PROCESS_TARGET', 'unknown')
HEALTH_PORT = ENV.fetch('HEALTH_PORT', 3001).to_i
# Only load Rails for workers that need DB checks
if PROCESS_TARGET.start_with?('worker')
require_relative '/app/config/environment'
end
server = WEBrick::HTTPServer.new(Port: HEALTH_PORT, Logger: WEBrick::Log.new("/dev/null"), AccessLog: [])
server.mount_proc '/health' do |req, res|
begin
if PROCESS_TARGET.start_with?('worker')
# Workers check database connectivity
ActiveRecord::Base.connection.execute("SELECT 1")
end
# Kafka consumers just check process is alive (per requirements)
res.status = 200
res.content_type = 'application/json'
res.body = {
status: 'healthy',
process_target: PROCESS_TARGET,
pod_name: ENV.fetch('POD_NAME', 'unknown'),
namespace: ENV.fetch('POD_NAMESPACE', 'default'),
timestamp: Time.now.iso8601
}.to_json
rescue => e
res.status = 503
res.content_type = 'application/json'
res.body = { status: 'unhealthy', process_target: PROCESS_TARGET, error: e.message, timestamp: Time.now.iso8601 }.to_json
end
end
server.mount_proc '/ready' do |req, res|
res.status = 200
res.content_type = 'application/json'
res.body = { status: 'ready', process_target: PROCESS_TARGET }.to_json
end
trap('INT') { server.shutdown }
trap('TERM') { server.shutdown }
server.startFor the web server, health checks are handled by a Rails controller. Note: Liveness checks process health only (no DB), while readiness checks DB connectivity.
# app/controllers/health_controller.rb
class HealthController < ApplicationController
skip_before_action :authenticate_user!, raise: false
# Liveness: Is the process alive? (Don't check DB - restarting won't help if DB is down)
def liveness
render json: { status: 'alive', process_target: ENV.fetch('PROCESS_TARGET', 'server'), timestamp: Time.current.iso8601 }, status: :ok
end
# Readiness: Can it serve traffic? (Check DB connectivity)
def readiness
ActiveRecord::Base.connection.execute("SELECT 1")
render json: {
status: 'ready',
process_target: ENV.fetch('PROCESS_TARGET', 'server'),
pod_name: ENV.fetch('POD_NAME', 'unknown'),
namespace: ENV.fetch('POD_NAMESPACE', 'default'),
timestamp: Time.current.iso8601
}, status: :ok
rescue => e
render json: { status: 'not_ready', error: e.message }, status: :service_unavailable
end
end# config/routes.rb (add these routes)
get '/health/liveness' => 'health#liveness'
get '/health/readiness' => 'health#readiness'# Gemfile (add)
gem 'lograge'# config/environments/production.rb (add)
config.lograge.enabled = true
config.lograge.formatter = Lograge::Formatters::Json.new
config.lograge.custom_options = lambda do |event|
{
process_target: ENV.fetch('PROCESS_TARGET', 'server'),
pod_name: ENV.fetch('POD_NAME', 'unknown'),
namespace: ENV.fetch('POD_NAMESPACE', 'default')
}
end| Variable | Required | Default | Description |
|---|---|---|---|
PROCESS_TARGET |
No | Set by target | Process type identifier (server, worker-primary, worker-secondary, kafka-*) |
HEALTH_PORT |
No | 3001-3004 | Port for health check server (workers/kafka) |
DELAYED_JOB_POOLS |
Conditional | "" | Pool arguments for delayed_job (required for worker target) |
DELAYED_JOB_TIMEOUT |
No | 280 | Seconds to wait for job completion on SIGTERM |
RAILS_ENV |
Yes | - | Rails environment |
RAILS_LOG_TO_STDOUT |
No | true | Enable logging to stdout |
RAILS_SERVE_STATIC_FILES |
No | true | Enable static file serving from Puma |
RAILS_MAX_THREADS |
No | 5 | Maximum Puma threads per pod |
RAILS_MIN_THREADS |
No | 5 | Minimum Puma threads per pod |
DB_POOL |
No | 5 | ActiveRecord connection pool size (should match RAILS_MAX_THREADS) |
DATABASE_URL |
Yes | - | Database connection string |
KAFKA_* |
Conditional | - | Kafka credentials (required for kafka-* targets) |
LOGZIO_TOKEN |
Yes | - | Logz.io shipping token (via Secret) |
POD_NAME |
No | unknown | Kubernetes pod name (from downward API) |
POD_NAMESPACE |
No | default | Kubernetes namespace (from downward API) |
| Secret Name | Keys | Used By |
|---|---|---|
| storecove-app-db-credentials | DATABASE_HOST, DATABASE_PORT, DATABASE_USERNAME, DATABASE_PASSWORD, DATABASE_NAME | All |
| storecove-app-master-key | RAILS_MASTER_KEY | All |
| storecove-app-aws-credentials | AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, AWS_*_BUCKET | All |
| storecove-app-valkey-credentials | VALKEY_HOST, VALKEY_PORT, VALKEY_USERNAME, VALKEY_PASSWORD | Rails, Workers |
| storecove-app-queue-credentials | SQS_*_QUEUE URLs (bounces, complaints, deliveries, partner, peppol, receive, sftp) | Workers |
| storecove-app-kafka-credentials | KAFKA_CONSUMER, KAFKA_PRODUCER | Kafka consumers |
| storecove-app-logzio | LOGZIO_TOKEN | Fluent Bit |
| storecove-app-rollbar | ROLLBAR_ACCESS_TOKEN | GitHub Actions |
| storecove-app-email-credentials | EMAIL_PROVIDER_USERNAME, EMAIL_PROVIDER_PASSWORD | Rails, Workers |
| storecove-app-billing-credentials | CHARGEBEE_API_KEY, CHARGEBEE_SITE, STRIPE_SECRET_KEY | Rails, Workers |
| storecove-app-peppol-credentials | PEPPOL_SHOP_ID, DEFAULT_ACCESSPOINT_* | Rails, Workers |
| storecove-app-webhooks-credentials | WEBHOOKS_ENCRYPT_KEY, WEBHOOKS_ENCRYPT_IV | Rails, Workers |
| storecove-app-intercom-credentials | INTERCOM_APP_ID, INTERCOM_API_SECRET, INTERCOM_API_ACCESS_TOKEN | Rails |
| mysql-ca-cert | ca-cert.pem | All (mounted as volume) |
| Component | Health Port | Endpoint | Notes |
|---|---|---|---|
| Rails Server | 3000 | /health/liveness, /health/readiness | Via Rails controller |
| Worker Primary | 3001 | /health | Via WEBrick health_server.rb |
| Worker Secondary | 3001 | /health | Via WEBrick health_server.rb |
| Kafka Sending Status | 3002 | /health | Via WEBrick health_server.rb |
| Kafka New Document | 3003 | /health | Via WEBrick health_server.rb |
| Kafka Received Status | 3004 | /health | Via WEBrick health_server.rb |
| Component | CPU Request | CPU Limit | Memory Request | Memory Limit | Replicas | terminationGracePeriodSeconds |
|---|---|---|---|---|---|---|
| Rails Server | 500m | 2000m | 1Gi | 4Gi | 2-10 (HPA) | 30 |
| Worker Primary | 250m | 1000m | 512Mi | 2Gi | 2-5 | 300 |
| Worker Secondary | 250m | 2000m | 512Mi | 4Gi | 2-5 | 300 |
| Kafka Consumer (each) | 100m | 500m | 256Mi | 1Gi | 1-3 | 60 |
| CronJob (each) | 100m | 500m | 256Mi | 1Gi | N/A | N/A |
Each Docker build target is built and pushed separately with appropriate tags:
# Build Rails server target
- name: Build and push Rails server
uses: docker/build-push-action@v6
with:
context: .
push: true
target: rails
tags: |
${{ vars.OVH_REGISTRY_URL }}/storecove-app:rails-${{ github.sha }}
${{ vars.OVH_REGISTRY_URL }}/storecove-app:rails-latest
cache-from: type=gha
cache-to: type=gha,mode=max
# Build Worker target
- name: Build and push Worker
uses: docker/build-push-action@v6
with:
context: .
push: true
target: worker
tags: |
${{ vars.OVH_REGISTRY_URL }}/storecove-app:worker-${{ github.sha }}
${{ vars.OVH_REGISTRY_URL }}/storecove-app:worker-latest
cache-from: type=gha
cache-to: type=gha,mode=max
# Build Kafka consumer targets
- name: Build and push Kafka Sending Status
uses: docker/build-push-action@v6
with:
context: .
push: true
target: kafka-sending-status
tags: |
${{ vars.OVH_REGISTRY_URL }}/storecove-app:kafka-sending-status-${{ github.sha }}
${{ vars.OVH_REGISTRY_URL }}/storecove-app:kafka-sending-status-latest
cache-from: type=gha
cache-to: type=gha,mode=max
# Repeat for kafka-new-document and kafka-received-statusThe build strategy uses Docker BuildKit with GitHub Actions cache for faster builds:
Cache Strategy:
cache-from: type=gha- Pull cache layers from previous buildscache-to: type=gha,mode=max- Store all layers for future builds- Shared layers between targets (base, ruby-deps, node-deps, app-base) are cached once
Build Performance:
- First build: ~15-20 minutes (all layers)
- Subsequent builds (code changes only): ~2-5 minutes (app-base rebuilt)
- Subsequent builds (dependency changes): ~10-12 minutes (ruby-deps/node-deps rebuilt)
Parallel Builds: Consider building targets in parallel using GitHub Actions matrix strategy:
strategy:
matrix:
target: [rails, worker, kafka-sending-status, kafka-new-document, kafka-received-status]
steps:
- uses: docker/build-push-action@v6
with:
target: ${{ matrix.target }}
tags: ${{ vars.OVH_REGISTRY_URL }}/storecove-app:${{ matrix.target }}-${{ github.sha }}This reduces total build time from ~15 minutes sequential to ~5 minutes parallel (limited by slowest target).
# Run database migrations
- name: Run database migrations
run: |
kubectl run migration-${{ github.sha }} \
--image=${{ vars.OVH_REGISTRY_URL }}/storecove-app:rails-${{ github.sha }} \
--restart=Never \
--rm \
--wait \
--command -- bash -lc "bundle exec rails db:migrate"
# Apply Kubernetes manifests
- name: Apply Kubernetes manifests
run: |
export IMAGE_TAG=${{ github.sha }}
envsubst < k8s/rails-server.yaml | kubectl apply -f -
envsubst < k8s/worker-primary.yaml | kubectl apply -f -
envsubst < k8s/worker-secondary.yaml | kubectl apply -f -
envsubst < k8s/kafka-sending-status.yaml | kubectl apply -f -
envsubst < k8s/kafka-new-document.yaml | kubectl apply -f -
envsubst < k8s/kafka-received-status.yaml | kubectl apply -f -
kubectl apply -f k8s/cronjobs/
kubectl apply -f k8s/ingress.yaml
# Notify Rollbar of deployment
- name: Notify Rollbar
if: success()
run: |
curl -X POST https://api.rollbar.com/api/1/deploy/ \
-H "Content-Type: application/json" \
-d '{
"access_token": "${{ secrets.ROLLBAR_ACCESS_TOKEN }}",
"environment": "production",
"revision": "${{ github.sha }}",
"local_username": "${{ github.actor }}",
"comment": "Deployed via GitHub Actions"
}'apiVersion: apps/v1
kind: Deployment
metadata:
name: rails-server
labels:
app: storecove
component: server
spec:
replicas: 2
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 0
maxSurge: 1
selector:
matchLabels:
app: storecove
component: server
template:
metadata:
labels:
app: storecove
component: server
spec:
terminationGracePeriodSeconds: 30
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
containers:
- name: rails
image: ${OVH_REGISTRY_URL}/storecove-app:rails-latest
imagePullPolicy: Always
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: false
capabilities:
drop:
- ALL
env:
- name: RAILS_ENV
value: "production"
- name: RAILS_SERVE_STATIC_FILES
value: "true"
- name: PROCESS_TARGET
value: "server"
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: MYSQL_SSL_CA
value: "/etc/ssl/mysql/ca-cert.pem"
envFrom:
- secretRef:
name: storecove-app-db-credentials
- secretRef:
name: storecove-app-master-key
- secretRef:
name: storecove-app-aws-credentials
- secretRef:
name: storecove-app-valkey-credentials
- secretRef:
name: storecove-app-email-credentials
- secretRef:
name: storecove-app-billing-credentials
- secretRef:
name: storecove-app-peppol-credentials
- secretRef:
name: storecove-app-webhooks-credentials
- secretRef:
name: storecove-app-intercom-credentials
- secretRef:
name: storecove-app-rollbar-credentials
volumeMounts:
- name: mysql-ca
mountPath: /etc/ssl/mysql
readOnly: true
ports:
- containerPort: 3000
name: http
livenessProbe:
httpGet:
path: /health/liveness
port: 3000
initialDelaySeconds: 30
periodSeconds: 10
failureThreshold: 3
timeoutSeconds: 5
readinessProbe:
httpGet:
path: /health/readiness
port: 3000
initialDelaySeconds: 30
periodSeconds: 10
failureThreshold: 3
timeoutSeconds: 5
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: 2000m
memory: 4Gi
volumes:
- name: mysql-ca
secret:
secretName: mysql-ca-cert
---
apiVersion: v1
kind: Service
metadata:
name: rails-server
spec:
selector:
app: storecove
component: server
ports:
- port: 80
targetPort: 3000Important: The Ingress NGINX Controller is scheduled for retirement in March 2026.
- Verify OVH's actual ingress controller type before production deployment
- If OVH uses nginx-ingress, plan migration to Gateway API by Q2 2026
- The annotations below assume nginx-ingress; update if OVH uses a different controller
OVH Production Subdomains:
app.fr.storecove.com- Main application (2M body size limit)api.fr.storecove.com- API endpoint (100M body size limit)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: storecove-app-ingress
annotations:
# Body size limits
nginx.ingress.kubernetes.io/proxy-body-size: "2m"
# Timeouts for long-running requests
nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
nginx.ingress.kubernetes.io/proxy-send-timeout: "300"
# Security headers
nginx.ingress.kubernetes.io/server-snippet: |
more_clear_headers "X-Powered-By";
more_clear_headers "Server";
# TLS
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
ingressClassName: nginx
tls:
- hosts:
- app.fr.storecove.com
secretName: storecove-app-tls
rules:
- host: app.fr.storecove.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: rails-server
port:
number: 80
---
# Separate ingress for API subdomain with larger body size
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: storecove-api-ingress
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: "100m"
nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
nginx.ingress.kubernetes.io/proxy-send-timeout: "300"
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
ingressClassName: nginx
tls:
- hosts:
- api.fr.storecove.com
secretName: storecove-api-tls
rules:
- host: api.fr.storecove.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: rails-server
port:
number: 80apiVersion: apps/v1
kind: Deployment
metadata:
name: worker-primary
labels:
app: storecove
component: worker-primary
spec:
replicas: 2
selector:
matchLabels:
app: storecove
component: worker-primary
template:
metadata:
labels:
app: storecove
component: worker-primary
spec:
terminationGracePeriodSeconds: 300
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
containers:
- name: worker
image: ${OVH_REGISTRY_URL}/storecove-app:worker-latest
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: false
capabilities:
drop:
- ALL
env:
- name: RAILS_ENV
value: "production"
- name: PROCESS_TARGET
value: "worker-primary"
- name: DELAYED_JOB_POOLS
value: "--pool=mail:1 --pool=inboundpeppol,inboundpeppolemail,inboundsftp,inboundublemail,inboundpartneremail:4 --pool=ses_notifications,ses_mail,sar_mail,edi_smtp,edi_as2,ses_mail_in_out:2 --pool=vatcalc_out_out_live,vatcalc_out_out_pilot:1 --pool=analyze_action,invoice_analyzer,slack,apply_action:1 --pool=document_submissions:2"
- name: DELAYED_JOB_TIMEOUT
value: "280"
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: MYSQL_SSL_CA
value: "/etc/ssl/mysql/ca-cert.pem"
envFrom:
- secretRef:
name: storecove-app-db-credentials
- secretRef:
name: storecove-app-master-key
- secretRef:
name: storecove-app-aws-credentials
- secretRef:
name: storecove-app-valkey-credentials
- secretRef:
name: storecove-app-queue-credentials
- secretRef:
name: storecove-app-email-credentials
- secretRef:
name: storecove-app-billing-credentials
- secretRef:
name: storecove-app-peppol-credentials
- secretRef:
name: storecove-app-webhooks-credentials
volumeMounts:
- name: mysql-ca
mountPath: /etc/ssl/mysql
readOnly: true
ports:
- containerPort: 3001
name: health
livenessProbe:
httpGet:
path: /health
port: 3001
initialDelaySeconds: 30
periodSeconds: 30
failureThreshold: 3
timeoutSeconds: 10
resources:
requests:
cpu: 250m
memory: 512Mi
limits:
cpu: 1000m
memory: 2Gi
volumes:
- name: mysql-ca
secret:
secretName: mysql-ca-certapiVersion: apps/v1
kind: Deployment
metadata:
name: worker-secondary
labels:
app: storecove
component: worker-secondary
spec:
replicas: 2
selector:
matchLabels:
app: storecove
component: worker-secondary
template:
metadata:
labels:
app: storecove
component: worker-secondary
spec:
terminationGracePeriodSeconds: 300
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
containers:
- name: worker
image: ${OVH_REGISTRY_URL}/storecove-app:worker-latest
imagePullPolicy: Always
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: false
capabilities:
drop:
- ALL
env:
- name: RAILS_ENV
value: "production"
- name: PROCESS_TARGET
value: "worker-secondary"
- name: DELAYED_JOB_POOLS
value: "--pool=smp_phoss:8 --pool=aruba_out_out_prod,aruba_out_out_pilot,aruba_out_out_webhooks_pilot,aruba_out_out_webhooks_prod:1 --pool=chargebee_webhook_events,exactsales_webhook_events,storecove_webhook_events:1 --pool=outgoing_webhooks,outgoing_webhooks_sandbox:4 --pool=outgoing_webhooks_asia,outgoing_webhooks_sandbox_asia:4 --pool=exact_worker,snelstart_worker,sftp_worker,as2_worker:1 --pool=received_documents,aruba_in_in_webhooks:1 --pool=storecove_api_self:3 --pool=active_storage_analysis,active_storage_mirror,active_storage_preview,active_storage_purge:1 --pool=kafka_sending_actions_status_update,kafka_received_document_status,kafka_new_document_notification:12 --pool=meta_events,exceptions,aruba_admin:1 --pool=customer_reporting:1 --pool=my_lhdnm_poller:6"
- name: DELAYED_JOB_TIMEOUT
value: "280"
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: MYSQL_SSL_CA
value: "/etc/ssl/mysql/ca-cert.pem"
envFrom:
- secretRef:
name: storecove-app-db-credentials
- secretRef:
name: storecove-app-master-key
- secretRef:
name: storecove-app-aws-credentials
- secretRef:
name: storecove-app-valkey-credentials
- secretRef:
name: storecove-app-queue-credentials
- secretRef:
name: storecove-app-email-credentials
- secretRef:
name: storecove-app-billing-credentials
- secretRef:
name: storecove-app-peppol-credentials
- secretRef:
name: storecove-app-webhooks-credentials
volumeMounts:
- name: mysql-ca
mountPath: /etc/ssl/mysql
readOnly: true
ports:
- containerPort: 3001
name: health
livenessProbe:
httpGet:
path: /health
port: 3001
initialDelaySeconds: 30
periodSeconds: 30
failureThreshold: 3
timeoutSeconds: 10
resources:
requests:
cpu: 250m
memory: 512Mi
limits:
cpu: 2000m
memory: 4Gi
volumes:
- name: mysql-ca
secret:
secretName: mysql-ca-certapiVersion: apps/v1
kind: Deployment
metadata:
name: kafka-sending-status
labels:
app: storecove
component: kafka-sending-status
spec:
replicas: 1
selector:
matchLabels:
app: storecove
component: kafka-sending-status
template:
metadata:
labels:
app: storecove
component: kafka-sending-status
spec:
terminationGracePeriodSeconds: 60
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
containers:
- name: consumer
image: ${OVH_REGISTRY_URL}/storecove-app:kafka-sending-status-latest
imagePullPolicy: Always
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: false
capabilities:
drop:
- ALL
env:
- name: RAILS_ENV
value: "production"
- name: PROCESS_TARGET
value: "kafka-sending-status"
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: MYSQL_SSL_CA
value: "/etc/ssl/mysql/ca-cert.pem"
envFrom:
- secretRef:
name: storecove-app-db-credentials
- secretRef:
name: storecove-app-master-key
- secretRef:
name: storecove-app-kafka-credentials
volumeMounts:
- name: mysql-ca
mountPath: /etc/ssl/mysql
readOnly: true
ports:
- containerPort: 3002
name: health
livenessProbe:
httpGet:
path: /health
port: 3002
initialDelaySeconds: 30
periodSeconds: 30
failureThreshold: 3
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 1Gi
volumes:
- name: mysql-ca
secret:
secretName: mysql-ca-certNote: The kafka-new-document and kafka-received-status deployments follow the same pattern, with their respective ports (3003, 3004) and PROCESS_TARGET values.
Scheduled tasks are implemented as Kubernetes CronJobs using the rails Docker target. Each CronJob runs independently with the full Rails environment.
# Example: Daily reporting task
apiVersion: batch/v1
kind: CronJob
metadata:
name: daily-report
labels:
app: storecove
component: cronjob
spec:
schedule: "0 6 * * *" # 6 AM UTC daily
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
jobTemplate:
spec:
activeDeadlineSeconds: 3600 # Job must complete within 1 hour
backoffLimit: 2
template:
spec:
restartPolicy: OnFailure
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
containers:
- name: rails
image: ${OVH_REGISTRY_URL}/storecove-app:rails-latest
imagePullPolicy: Always
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: false
capabilities:
drop:
- ALL
command: ["bash", "-lc", "bundle exec rake reports:daily"]
env:
- name: RAILS_ENV
value: "production"
- name: PROCESS_TARGET
value: "cronjob-daily-report"
- name: MYSQL_SSL_CA
value: "/etc/ssl/mysql/ca-cert.pem"
envFrom:
- secretRef:
name: storecove-app-db-credentials
- secretRef:
name: storecove-app-master-key
- secretRef:
name: storecove-app-aws-credentials
- secretRef:
name: storecove-app-valkey-credentials
volumeMounts:
- name: mysql-ca
mountPath: /etc/ssl/mysql
readOnly: true
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 1Gi
volumes:
- name: mysql-ca
secret:
secretName: mysql-ca-certTasks currently defined in config/schedule.rb (whenever gem) must be migrated to individual CronJob manifests:
| Task Description | CronJob Name | Schedule | Command |
|---|---|---|---|
| Customer reports | customer-reports | 0 6 * * * |
rake customer_reporting:schedule_reports |
| SaaS org reporting (monthly) | saas-organizations | 30 8 1 * * |
rake saas:organizations_global && rake saas:organizations_asia && rake saas:organizations_pacific |
| Peppol end users reporting | peppol-end-users | 0 23 2 * * |
rake peppol_reporting:peppol_reporting_end_users |
| Peppol transactions reporting | peppol-transactions | 0 1 3 * * |
rake peppol_reporting:peppol_reporting_transactions |
| Peppol SG/IRAS reporting | peppol-sg-monthly | 30 5 1 * * |
rake peppol_reporting:identifiers_in_out_sg && rake peppol_reporting:reporting_sg_iras_sla_sandbox && rake peppol_reporting:reporting_sg_iras_sla_live |
| AWS SES bounce rates | aws-ses-bounce-rates | 30 4 * * 1 |
rake aws_ses_reporting:bounce_rates_sending && rake aws_ses_reporting:bounce_rates_administrations |
| Kafka sending/clearing updates | kafka-sending-clearing | */10 * * * * |
rake kafka:produce_invoice_submission_action_update_requests_sending && rake kafka:produce_invoice_submission_action_update_requests_clearing |
| Kafka new docs hourly | kafka-new-docs-hourly | 0 * * * * |
rake kafka:produce_new_documents_request_hourly |
| Kafka new docs daily | kafka-new-docs-daily | 0 0 * * * |
rake kafka:produce_new_documents_request_daily |
| Clean delayed jobs queue | clean-delayed-jobs | */5 * * * * |
rake railsdb:clean_delayed_jobs_inboundpeppol |
| CorpPass/MyKYC detection | corppass-mykyc-detect | */5 * * * * |
rake corppass:detect[sandbox] && rake corppass:detect[live] && rake mykyc:detect[sandbox] && rake mykyc:detect[live] |
| Reconcile Chargebee | reconcile-chargebee | 15 7 * * 6 |
rake saas:reconcile_chargebee |
| Check invalid identifiers | identifiers-invalid | 30 7 * * 6 |
rake identifiers:invalid |
| SMP reconciliation | smp-reconcile | 0 8 * * 6 |
rake smp:reconcile && rake smp:reconcile_sg |
| Email worker | email-worker | 0 * * * * |
rails runner "C5::EmailWorker.new.perform" |
| Invoice analyzer | invoice-analyzer | 0 * * * * |
rails runner "InvoiceAnalyzerJob.perform_later" |
Total: 16 CronJobs replacing container-level cron managed by whenever gem.
Fluent Bit requires RBAC permissions to access Kubernetes metadata for log enrichment.
# ServiceAccount for Fluent Bit
apiVersion: v1
kind: ServiceAccount
metadata:
name: fluent-bit
namespace: logging
---
# ClusterRole with permissions to read pod metadata
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: fluent-bit
rules:
- apiGroups: [""]
resources:
- namespaces
- pods
verbs: ["get", "list", "watch"]
---
# ClusterRoleBinding to bind the role to the service account
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: fluent-bit
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: fluent-bit
subjects:
- kind: ServiceAccount
name: fluent-bit
namespace: logging
---
# DaemonSet for Fluent Bit
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluent-bit
namespace: logging
spec:
selector:
matchLabels:
app: fluent-bit
template:
metadata:
labels:
app: fluent-bit
spec:
serviceAccountName: fluent-bit
containers:
- name: fluent-bit
image: fluent/fluent-bit:latest
securityContext:
runAsNonRoot: false
privileged: false
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
env:
- name: LOGZIO_TOKEN
valueFrom:
secretKeyRef:
name: storecove-app-logzio
key: token
volumeMounts:
- name: varlog
mountPath: /var/log
readOnly: true
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: config
mountPath: /fluent-bit/etc/
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: config
configMap:
name: fluent-bit-config
---
apiVersion: v1
kind: ConfigMap
metadata:
name: fluent-bit-config
namespace: logging
data:
fluent-bit.conf: |
[SERVICE]
Flush 1
Log_Level info
Parsers_File parsers.conf
[INPUT]
Name tail
Path /var/log/containers/storecove*.log
Parser docker
Tag kube.*
Refresh_Interval 5
Mem_Buf_Limit 5MB
[FILTER]
Name kubernetes
Match kube.*
Kube_URL https://kubernetes.default.svc:443
Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
Merge_Log On
K8S-Logging.Parser On
[OUTPUT]
Name http
Match *
Host listener.logz.io
Port 8071
URI /?token=${LOGZIO_TOKEN}&type=kubernetes
Format json_lines
tls On
tls.verify On
parsers.conf: |
[PARSER]
Name docker
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%LA property is a characteristic or behavior that should hold true across all valid executions of a system—essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.
For any Docker build target (rails, worker, kafka-*), building and running the image SHALL start only the process specified by that target.
Validates: Requirements 1.2, 1.3, 1.4, 1.5, 1.6
For any Docker build target, the started process SHALL be the main process (PID 1 or direct child of PID 1) and SHALL not daemonize.
Validates: Requirements 1.7
For any worker or Kafka consumer target, the health check server SHALL be listening on its port before the main process starts consuming work.
Validates: Requirements 1.9
For any log entry output by any process type (server, worker, kafka-*), the log entry SHALL be valid JSON that can be parsed without error.
Validates: Requirements 2.1, 2.2, 2.3
For any JSON log entry, the entry SHALL contain the fields: timestamp, level, process_target, pod_name, namespace, and message.
Validates: Requirements 2.4, 2.5
For any log entry, the entry SHALL NOT contain values of environment variables whose names contain PASSWORD, SECRET, KEY, or TOKEN.
Validates: Requirements 2.8, 7.5
For any SIGTERM signal sent to a container, the main process SHALL begin graceful shutdown within 1 second.
Validates: Requirements 5.4
For any health check request, the response SHALL be returned within 5 seconds.
Validates: Requirements 3.11
For any Rails server, the liveness endpoint SHALL return 200 even when the database is unreachable, while the readiness endpoint SHALL return 503.
Validates: Requirements 3.1, 3.2, 3.3
| Error Condition | Behavior | Exit Code |
|---|---|---|
| Missing required env var | Log error with variable name, exit | 1 |
| Process fails to start | Log error with details, exit | 1 |
| Health server fails to bind | Log error, continue (main process may still work) | - |
| Component | Error Condition | HTTP Status | Response Body |
|---|---|---|---|
| Rails Server (readiness) | DB connection failed | 503 | {"status":"not_ready","error":"..."} |
| Worker | DB connection failed | 503 | {"status":"unhealthy","error":"..."} |
| Worker | Process not running | 503 | {"status":"unhealthy","error":"process not found"} |
| Kafka Consumer | Process crashed | 503 | {"status":"unhealthy","error":"..."} |
| Component | terminationGracePeriodSeconds | Rationale |
|---|---|---|
| Rails Server | 30 | Typical HTTP request timeout |
| Delayed Job Worker | 300 | Jobs may take several minutes |
| Kafka Consumer | 60 | Offset commit and disconnect |
Unit tests verify specific examples and edge cases:
-
Health Check Tests
- Test liveness returns 200 when process running
- Test readiness returns 200 when DB connected
- Test readiness returns 503 when DB disconnected
-
Logging Tests
- Test log output is valid JSON
- Test required fields are present
- Test sensitive data is not logged
Property-based tests verify universal properties across many inputs using a PBT library (e.g., RSpec property testing for Ruby).
Each property test should run minimum 100 iterations.
Property Test 1: JSON Log Validity
- Generate various log scenarios
- Verify all output is parseable JSON
- Feature: kubernetes-rails-deployment, Property 4: JSON Log Format Validity
Property Test 2: Required Fields Presence
- Generate log entries
- Verify all contain required fields
- Feature: kubernetes-rails-deployment, Property 5: Required Log Fields Presence
Property Test 3: Sensitive Data Exclusion
- Generate log entries with various env vars set
- Verify no sensitive values appear in logs
- Feature: kubernetes-rails-deployment, Property 6: Sensitive Data Exclusion from Logs
Integration tests verify components work together:
-
Container Build Tests
- Build each target
- Verify correct process starts
- Verify health endpoint responds
-
Kubernetes Manifest Validation
- Use
kubectl --dry-runto validate manifests - Verify all required fields present
- Verify probe configurations correct
- Use
-
Log Shipping Tests
- Start container with Fluent Bit
- Generate logs
- Verify logs appear in Logz.io (or mock endpoint)