tracing docker-image-resource checks

idea.md

Suppose the following resource in pipeline p is slow to check:

- name: image
  type: docker-image
  source:
    repository: artifactory.internal/concourse-docker/cf-mgmt
    tag: '05.12.2020'
    insecure_registries: 
    - artifactory.internal
    username: ((concourse-rm-un))
    password: ((concourse-rm-pw))

Let's try something to trace the execution of this resource's checks, to see where the slowness may be happening.

1. Open a terminal and intercept a check container for this resource with fly intercept --check p/image.
2. In the resulting shell, download delve:

   apt update
   apt install -y unzip
   curl -Lo delve.zip https://bintray.com/jetbrains/golang/download_file?file_path=com%2Fjetbrains%2Fdelve%2F1.1.35%2Fdelve-1.1.35.zip
   unzip delve.zip
   chmod +x dlv/linux/dlv
   

3. Following https://stac47.github.io/go/delve/debug/2019/11/06/debug-stdin-go-with-delve.html, create a long-lived fifo:

   mkfifo fifo
   sleep infinity > fifo &
   

4. Start the check program, attaching the fifo as stdin, and make note of the PID:

   $ /opt/resource/check < fifo &
   [2] 1336
   

5. Attach delve and trace every function in the main package:

   $ dlv/linux/dlv trace 'main.*' -p 1336
   

6. Open another terminal and intercept the same container with fly intercept --check p/image and write the check request payload to the fifo:

   echo '{
     "source":{
       "repository":"artifactory.internal/concourse-docker/cf-mgmt",
       "tag":"05.12.2020",
       "insecure_registries":["artifactory.internal"],
       "username":"<real username here>",
       "password":"<real password here>"
     }
   }' > fifo
   

7. Watch the tracepoints get logged in your other terminal. If the slowness is pronounced enough, you should be able to see where the tracepoints pause.