Parallel downloads with xargs

Download 5 URLs in parallel and print status codes

cat urls.txt | xargs -P 5 -I{} curl -sSL -o /dev/null -w '%{url_effective} %{http_code}\n' {}
tool
curl21 commands
tags
curl

More curl commands

  • WebSocket handshake test
    curl -i -N -H 'Upgrade: websocket' -H 'Connection: Upgrade' -H 'Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw==' -H 'Sec-WebSocket-Version: 13' http://localhost:8080/ws

    Verify WebSocket upgrade handshake responds with 101

  • Rate-limit download speed
    curl --limit-rate 1M -O https://example.com/bigfile.iso

    Limit download bandwidth to 1 MB/s

  • POST JSON payload
    curl -s -X POST https://api.example.com/v1/resource -H 'Content-Type: application/json' -H 'Authorization: Bearer <token>' -d '{"key":"value"}' | jq .

    Send a JSON POST request with Bearer auth

  • Follow redirects with verbose
    curl -sLv https://example.com 2>&1 | grep -E '^[<>*]'

    Show full request/response headers including redirects

  • Time a request
    curl -so /dev/null -w 'DNS: %{time_namelookup}s\nConnect: %{time_connect}s\nTTFB: %{time_starttransfer}s\nTotal: %{time_total}s\n' https://example.com

    Measure DNS, connect, and TTFB timing

all 21 curl commands