Home Software Linux curl/wget

Command Line Tools for Transferring Data

curl

Get verbose information:

$ curl -v -I ⟨URL⟩

Download remote file:

$ curl --no-clobber -C -R -O -H "User-Agent:" ⟨URL⟩

--no-clobber : do not overwrite files that already exist,
-C : resume getting a partially-downloaded file
-O : write output to a file named as the remote file
-H "User-Agent:" : send empty User-Agent header. The default header value is 'curl/[version]'.

wget

Wget - GNU Project - Free Software Foundation

Wget Manual

user-agent sent by default:

$ wget -qO- https://httpbin.org/user-agent
{
  "user-agent": "Wget/1.25.0"
}

Some websites may block wget default user-agent to prevent automated access. A custom user agent can be set to make wget look like a regular browser. Add to ~/.wgetrc:

user_agent = Mozilla/5.0 (Windows NT 10.0; Win64; x64)

Download remote file:

$ wget -S -c ⟨URL⟩

-c, --continue : resume getting a partially-downloaded file

-S, --server-response : print server response

Don't download anything:

wget -S --spider ⟨URL⟩