Skip to content

Instantly share code, notes, and snippets.

@rptaylor

rptaylor/v1.0.3-retry-leaderelection-v3 logs

Created January 15, 2026 00:00
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save rptaylor/f16dcae2bf7b21305f0e8c95aaeeb8f9 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save rptaylor/f16dcae2bf7b21305f0e8c95aaeeb8f9 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
v1.0.3-retry-leaderelection-v3 logs
E0114 23:40:56.918621 1 leaderelection.go:441] Failed to update lock optimistically: Put "https://10.232.0.1:443/apis/coordination.k8s.io/v1/namespaces/traefik/leases/kubevip-traefik?timeout=10s": context deadline exceeded, falling back to slow path
E0114 23:40:56.919212 1 leaderelection.go:448] error retrieving resource lock traefik/kubevip-traefik: client rate limiter Wait returned an error: context deadline exceeded
I0114 23:40:56.919311 1 leaderelection.go:297] failed to renew lease traefik/kubevip-traefik: context deadline exceeded
2026/01/14 23:40:57 INFO leadership lost service=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079 leader=cluster-dev-k8s-node-p02
2026/01/14 23:40:57 DEBUG service is active - delete after leadership was lost service=traefik
2026/01/14 23:40:57 DEBUG [service] lookup "target UID"=06f68e17-4979-413b-9f0e-3c87767f5079 "found UID"=06f68e17-4979-413b-9f0e-3c87767f5079 name=traefik namespace=traefik
2026/01/14 23:40:57 DEBUG setting HasEndpoints ip=10.5.7.90 value=false
2026/01/14 23:40:57 DEBUG ending layer 2 update ip=10.5.7.90 interface=eth0 ms=3000
2026/01/14 23:40:57 INFO [ARP manager] removing ARP/NDP instance name=10.5.7.90/32-eth0
2026/01/14 23:40:57 DEBUG type: 25, route: {Ifindex: 2 Dst: 10.5.7.90/32 Src: 10.5.7.90 Gw: <nil> Flags: [] Table: 255 Realm: 0}
2026/01/14 23:40:57 INFO [LOADBALANCER] Stopping load balancers name=traefik
2026/01/14 23:40:57 INFO [VIP] Deleting VIP ip=10.5.7.90
2026/01/14 23:40:57 INFO Removed instance from manager uid=06f68e17-4979-413b-9f0e-3c87767f5079 name=traefik "remaining advertised services"=0
2026/01/14 23:40:57 INFO stopping leader election service=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079
2026/01/14 23:40:57 DEBUG leaderelection finished - exiting and restarting service=traefik
2026/01/14 23:40:57 INFO new leader election service=traefik namespace=traefik lock_name=kubevip-traefik host_id=cluster-dev-k8s-node-p02
I0114 23:40:57.358304 1 leaderelection.go:257] attempting to acquire leader lease traefik/kubevip-traefik...
I0114 23:40:57.373218 1 leaderelection.go:271] successfully acquired lease traefik/kubevip-traefik
2026/01/14 23:40:57 DEBUG [STARTING] Service Sync namespace=traefik name=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079
2026/01/14 23:40:57 DEBUG no matching service instance found service=traefik namespace=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079 addresses=[10.5.7.90] hostnames=[]
2026/01/14 23:40:57 DEBUG [service] add namespace=traefik name=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079
2026/01/14 23:40:57 INFO NewInstance used instanceAddresses=[10.5.7.90] instanceHostnames=[]
2026/01/14 23:40:57 DEBUG service security enabled=false
2026/01/14 23:40:57 INFO (svcs) adding VIP ip=10.5.7.90 interface=eth0 namespace=traefik name=traefik
2026/01/14 23:40:57 DEBUG starting loadbalancer for service name=traefik namespace=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079
2026/01/14 23:40:57 DEBUG StartLoadBalancerService networks=1
2026/01/14 23:40:57 DEBUG current ip to process ip=10.5.7.90 mask=32
2026/01/14 23:40:57 DEBUG config flags enable_routing_table=false enable_leader_election=false enable_services_election=true
2026/01/14 23:40:57 INFO successful add IP
2026/01/14 23:40:57 DEBUG type: 24, route: {Ifindex: 2 Dst: 10.5.7.90/32 Src: 10.5.7.90 Gw: <nil> Flags: [] Table: 255 Realm: 0}
2026/01/14 23:40:57 DEBUG [service] update namespace=traefik name=traefik
2026/01/14 23:40:57 INFO layer 2 broadcaster starting IP=10.5.7.90 device=eth0
2026/01/14 23:40:57 DEBUG layer 2 update ip=10.5.7.90 interface=eth0 ms=3000
2026/01/14 23:40:57 INFO [ARP manager] inserting ARP/NDP instance name=10.5.7.90/32-eth0
2026/01/14 23:40:57 DEBUG LB status current="[{IP:10.5.7.90 Hostname: IPMode:<nil> Ports:[{Port:9000 Protocol:TCP Error:<nil>} {Port:80 Protocol:TCP Error:<nil>} {Port:443 Protocol:TCP Error:<nil>}]}]" new="[{IP:10.5.7.90 Hostname: IPMode:<nil> Ports:[{Port:9000 Protocol:TCP Error:<nil>} {Port:80 Protocol:TCP Error:<nil>} {Port:443 Protocol:TCP Error:<nil>}]}]"
2026/01/14 23:40:57 INFO [service] service=traefik namespace=traefik "synchronised in"=11ms
2026/01/14 23:40:57 DEBUG [service] add label to node namespace=traefik name=traefik
2026/01/14 23:40:57 DEBUG patching node node=cluster-dev-k8s-node-p02 patch="[{\"op\":\"add\",\"path\":\"/metadata/labels/service-provided.kube-vip.io~1traefik.traefik\",\"value\":\"10.5.7.90\"}]" operation=add labels=map[service-provided.kube-vip.io/traefik.traefik:10.5.7.90] clientSetNil=false
2026/01/14 23:40:57 DEBUG updated node=cluster-dev-k8s-node-p02 labels="map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:cluster-dev-k8s-node-p02 kubernetes.io/os:linux rcs.uvic.ca/node-type:service-node service-provided.kube-vip.io/traefik.traefik:10.5.7.90 topology.cinder.csi.openstack.org/zone:Persistent_02]"
2026/01/14 23:40:57 DEBUG [FINISHED] Service Sync namespace=traefik name=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079
2026/01/14 23:41:10 DEBUG type: 24, route: {Ifindex: 2 Dst: fd85:ee78:d8a6:8607::300/120 Src: <nil> Gw: 2607:f8f1:c11:7000:f816:3eff:fe52:4aa Flags: [] Table: 254 Realm: 0}
...
2026/01/14 23:43:59 DEBUG type: 24, route: {Ifindex: 5 Dst: fd85:ee78:d8a6:8607::800/120 Src: fd85:ee78:d8a6:8607::8e3 Gw: <nil> Flags: [] Table: 254 Realm: 0}
E0114 23:44:09.411269 1 leaderelection.go:441] Failed to update lock optimistically: Put "https://10.232.0.1:443/apis/coordination.k8s.io/v1/namespaces/traefik/leases/kubevip-traefik?timeout=10s": context deadline exceeded, falling back to slow path
E0114 23:44:09.411486 1 leaderelection.go:448] error retrieving resource lock traefik/kubevip-traefik: client rate limiter Wait returned an error: context deadline exceeded
I0114 23:44:09.411524 1 leaderelection.go:297] failed to renew lease traefik/kubevip-traefik: context deadline exceeded
2026/01/14 23:44:10 INFO leadership lost service=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079 leader=cluster-dev-k8s-node-p02
2026/01/14 23:44:10 DEBUG service is active - delete after leadership was lost service=traefik
2026/01/14 23:44:10 DEBUG [service] lookup "target UID"=06f68e17-4979-413b-9f0e-3c87767f5079 "found UID"=06f68e17-4979-413b-9f0e-3c87767f5079 name=traefik namespace=traefik
2026/01/14 23:44:10 DEBUG setting HasEndpoints ip=10.5.7.90 value=false
2026/01/14 23:44:10 DEBUG ending layer 2 update ip=10.5.7.90 interface=eth0 ms=3000
2026/01/14 23:44:10 INFO [ARP manager] removing ARP/NDP instance name=10.5.7.90/32-eth0
2026/01/14 23:44:10 DEBUG type: 25, route: {Ifindex: 2 Dst: 10.5.7.90/32 Src: 10.5.7.90 Gw: <nil> Flags: [] Table: 255 Realm: 0}
2026/01/14 23:44:10 INFO [LOADBALANCER] Stopping load balancers name=traefik
2026/01/14 23:44:10 INFO [VIP] Deleting VIP ip=10.5.7.90
2026/01/14 23:44:10 INFO Removed instance from manager uid=06f68e17-4979-413b-9f0e-3c87767f5079 name=traefik "remaining advertised services"=0
2026/01/14 23:44:10 INFO stopping leader election service=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079
2026/01/14 23:44:10 DEBUG leaderelection finished - exiting and restarting service=traefik
2026/01/14 23:44:10 DEBUG service lease cancelled, exiting watch service=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079
...
2026/01/14 23:50:06 DEBUG found endpoint provider=endpointslices ip=10.224.8.101 nodename=cluster-dev-k8s-node-p02
2026/01/14 23:50:06 DEBUG watcher provider=endpointslices "service name"=traefik namespace=traefik endpoints=1 "last endpoint"=10.224.8.101 "active leader election"=false
2026/01/14 23:50:06 INFO new leader election service=traefik namespace=traefik lock_name=kubevip-traefik host_id=cluster-dev-k8s-node-p02
I0114 23:50:06.559445 1 leaderelection.go:257] attempting to acquire leader lease traefik/kubevip-traefik...
I0114 23:50:06.591559 1 leaderelection.go:271] successfully acquired lease traefik/kubevip-traefik
2026/01/14 23:50:06 DEBUG [STARTING] Service Sync namespace=traefik name=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079
2026/01/14 23:50:06 DEBUG no matching service instance found service=traefik namespace=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079 addresses=[10.5.7.90] hostnames=[]
2026/01/14 23:50:06 DEBUG [service] add namespace=traefik name=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079
2026/01/14 23:50:06 INFO NewInstance used instanceAddresses=[10.5.7.90] instanceHostnames=[]
2026/01/14 23:50:06 DEBUG service security enabled=false
2026/01/14 23:50:06 INFO (svcs) adding VIP ip=10.5.7.90 interface=eth0 namespace=traefik name=traefik
2026/01/14 23:50:06 DEBUG starting loadbalancer for service name=traefik namespace=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079
2026/01/14 23:50:06 DEBUG StartLoadBalancerService networks=1
2026/01/14 23:50:06 DEBUG current ip to process ip=10.5.7.90 mask=32
2026/01/14 23:50:06 DEBUG config flags enable_routing_table=false enable_leader_election=false enable_services_election=true
2026/01/14 23:50:06 INFO successful add IP
2026/01/14 23:50:06 DEBUG [service] update namespace=traefik name=traefik
2026/01/14 23:50:06 DEBUG type: 24, route: {Ifindex: 2 Dst: 10.5.7.90/32 Src: 10.5.7.90 Gw: <nil> Flags: [] Table: 255 Realm: 0}
2026/01/14 23:50:06 INFO layer 2 broadcaster starting IP=10.5.7.90 device=eth0
2026/01/14 23:50:06 DEBUG layer 2 update ip=10.5.7.90 interface=eth0 ms=3000
2026/01/14 23:50:06 INFO [ARP manager] inserting ARP/NDP instance name=10.5.7.90/32-eth0
2026/01/14 23:50:06 DEBUG LB status current="[{IP:10.5.7.90 Hostname: IPMode:<nil> Ports:[{Port:9000 Protocol:TCP Error:<nil>} {Port:80 Protocol:TCP Error:<nil>} {Port:443 Protocol:TCP Error:<nil>}]}]" new="[{IP:10.5.7.90 Hostname: IPMode:<nil> Ports:[{Port:9000 Protocol:TCP Error:<nil>} {Port:80 Protocol:TCP Error:<nil>} {Port:443 Protocol:TCP Error:<nil>}]}]"
2026/01/14 23:50:06 INFO [service] service=traefik namespace=traefik "synchronised in"=242ms
2026/01/14 23:50:06 DEBUG [service] add label to node namespace=traefik name=traefik
2026/01/14 23:50:06 DEBUG patching node node=cluster-dev-k8s-node-p02 patch="[{\"op\":\"add\",\"path\":\"/metadata/labels/service-provided.kube-vip.io~1traefik.traefik\",\"value\":\"10.5.7.90\"}]" operation=add labels=map[service-provided.kube-vip.io/traefik.traefik:10.5.7.90] clientSetNil=false
2026/01/14 23:50:07 DEBUG updated node=cluster-dev-k8s-node-p02 labels="map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:cluster-dev-k8s-node-p02 kubernetes.io/os:linux rcs.uvic.ca/node-type:service-node service-provided.kube-vip.io/traefik.traefik:10.5.7.90 topology.cinder.csi.openstack.org/zone:Persistent_02]"
2026/01/14 23:50:07 DEBUG [FINISHED] Service Sync namespace=traefik name=traefik uid=06f68e17-4979-413b-9f0e-3c87767f5079
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment