get all GET URLs of all network requests made on a web page from the Chrome Dev Tools (from an exported HAR)

HOWTO.md

Setup

Using Homebrew on Mac OS X:

brew install jq

Add these aliases to your profile (e.g., ~/.zshrc, ~/.bashrc, ~/.profile, etc.):

alias getUrls='pbpaste | jq ".log.entries" | tee >(jq --raw-output "[.[] | select(.request.method == \"GET\")|.request.url] | .[]")'
alias getUrlsTime='pbpaste | jq ".log.entries" | tee >(jq --raw-output "[.[] | select(.request.method == \"GET\")|.request.url,\"blocked :\"+(.timings.blocked|tostring),\"wait: \" + (.timings.wait|tostring), \"receive: \" + (.timings.receive|tostring)] | .[]")'
alias getUrlsTimeMD=' pbpaste | jq ".log.entries" | tee >(jq --raw-output "[.[] | select(.request.method == \"GET\")| [\"\",(.request.url|split(\"/noah/\")|.[1]),(.timings.blocked|floor|tostring),(.timings.wait|floor|tostring),(.timings.receive|floor|tostring),\"\"]|join(\"|\")]| .[]")'
alias getUrlsTimeCSV=' pbpaste | jq ".log.entries" | tee >(jq --raw-output "[.[] | select(.request.method == \"GET\")| [(.request.url|split(\"/noah/\")|.[1]),(.timings.blocked|floor),(.timings.wait|floor),(.timings.receive|floor)]]| .[] | @csv")'

Usage

1. Open the Chrome Developer Tools (Command+Option+i)
2. Click the Network tab
3. Reload the page (Command+r)
4. In the Network tab, hover over any area in the table that is not a link (e.g., the Status column)
5. Right click (Control+click) to open the context menu and click Copy All as HAR
6. Open your favourite command-line interface (e.g., Terminal)
7. Enter getUrls to print the URLs or getUrlsTime to also print timing data or getUrlsTimeMD to produce a markdown-table or getURLsTimeCSV to produce a CSV output