Following Redirects With Curl

By | 2020-11-12

The curl command is a great tool with a lot of uses. When used for HTTP, it doesn’t follow redirects. However, following redirects with curl is easy.

If you are in a hurry, the short answer is the -L option:

$  curl http://tylersguides.com > tylersguides.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
$ head -1 tylersguides.html
$ curl -L  http://tylersguides.com > tylersguides.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 26082    0 26082    0     0  14060      0 --:--:--  0:00:01 --:--:-- 34454
$ head -1 tylersguides.html
<!DOCTYPE html>

A common use for curl is downloading files or web pages via command line, especially scripts. Sometimes this ends up with results like this:

$ curl http://tylersguides.com > tylersguides.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
$ head tylersguides.html 
$ 

Notice how the resulting file is empty? This is because when I tried to download my site’s home page with plain HTTP, the web server sends an HTTP redirect to the same URL, except with HTTPS. If you suspect you are being redirected, curl can show you the HTTP headers with the -I option.

$ curl -I http://tylersguides.com
HTTP/1.1 301 Moved Permanently
Date: Wed, 11 Nov 2020 13:51:48 GMT
Server: Apache
X-Redirect-By: WordPress
Upgrade: h2,h2c
Connection: Upgrade
Location: https://tylersguides.com/
host-header: c2hhcmVkLmJsdWVob3N0LmNvbQ==
Content-Type: text/html; charset=UTF-8

$ 

As you can see, the web server responded with a redirect. You could simply use the URL from the Location header with another curl command, but sometimes you will run into situations where you get multiple redirects.

To have curl automatically follow the redirect, use the -L option:

$ curl -I http://tylersguides.com
HTTP/1.1 301 Moved Permanently
Date: Wed, 11 Nov 2020 13:51:48 GMT
Server: Apache
X-Redirect-By: WordPress
Upgrade: h2,h2c
Connection: Upgrade
Location: https://tylersguides.com/
host-header: c2hhcmVkLmJsdWVob3N0LmNvbQ==
Content-Type: text/html; charset=UTF-8

$  curl http://tylersguides.com > tylersguides.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
$ head -1 tylersguides.html
$ curl -L  http://tylersguides.com > tylersguides.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 26082    0 26082    0     0  14060      0 --:--:--  0:00:01 --:--:-- 34454
$ head -1 tylersguides.html
<!DOCTYPE html>

If you are writing a script to automate a transfer, you may wish to get rid of the status information. Use the -s option to suppress it:

$ curl -sL  http://tylersguides.com > tylersguides.html

References