Protecting Downloads From Hotlinking – The Soft Way
The Story
Once upon a time, on a Web site 23 hops away from my home PC, there was a free software download that required registration. An email with the download page URL was sent to the visitor after registration. The download page contained usage instructions and a direct, static URL of the download in the form http://host/download/file.
Someone had registered and published the latter URL in a public directory, so people started to download the file directly, without seeing even the download page, let alone the registration form.
The URL of the download was changed, but the story repeated itself in a couple of weeks.
The Problem
The typical countermeasure is to generate a unique download URL that would only work for the given visitor. However, the question is how to identify the visitor, who may have filled the registration form on a mobile device then read the confirmation email on a PC, may have referrer and/or cookies blocked or not propagated to the download manager, or may be connecting from behind a large proxy with multiple external IPs.
It was decided that the protection mechanism should not get in the way of visitors that properly registered – the download had to work for them no matter what. In particular, this meant that the filename part of the URL had to be the desired name of the downloaded file.
The Implementation Restrictions
The file is large (circa 100 MB), so having it served entirely by a PHP or CGI script might hit memory and/or execution time limits. The server’s ability to process multiple requests simultaneously would have been jeopardized as well. The X-Sendfile header could have been the answer, but the Web server was Apache 2.2, which only supports that header through a third-party module. Using a third-party module is undesirable, and so is cluttering a Web server directory with temporary symbolic links.
The Solution
- Create a data directory in the server’s variable data on
/var. - In the data directory, create a marker file with a random name, and a symbolic link with a fixed name, pointing to that file.
- Once a day, generate a new marker file and move the symbolic link over to it. Delay removal of the previous marker file for another day using a secondary symbolic link.
-
When generating temporary URLs, retrieve the name of the current marker file file via the symbolic link and inject that name into the URL, for instance:
http://host/download/marker/file - Add a URL rewrite rule to the Apache config for the site that would match URLs with markers, check if the given marker exists in the data directory, and if yes, internally rewrite the URL to point to the secret directory containing the protected file.
Implemented this way, a temporary URL will be good for at least 24 but not more than 48 hours.
The Implementation
It is recommended to replace /var/tmp with the pathname of a dedicated marker directory.
Script to manage marker files:
Add this script to crontab, then run it once to initialize the primary marker. Optionally pass the marker directory as argument:
#!/bin/sh
BASEDIR=${1-/var/tmp}
CURRLINK=${BASEDIR}/curr
PREVLINK=${BASEDIR}/prev
getLinkTarget () {
LINKTARGET=
if [ ! -e $1 ]
then
if [ -L $1 ]
then
echo "$1 points to a non-existing file!"
exit 1
else
# $1 does not exist at all, that's OK
return
fi
fi
if [ ! -L $1 ]
then
echo "$1 is not a symbolic link!"
exit 1
fi
LINKTARGET=`readlink $1`
# No need to test LINKTARGET for existence as -e $1 above would have returned false
if [ "`dirname ${LINKTARGET}`" != "${BASEDIR}" ]
then
echo "$1 points to a location outside of ${BASEDIR}"
exit 1
fi
if [ ! -f ${LINKTARGET} ]
then
echo "$1 points to something other than a regular file!"
exit 1
fi
}
getLinkTarget ${CURRLINK}
CURRFILE=${LINKTARGET}
getLinkTarget ${PREVLINK}
PREVFILE=${LINKTARGET}
[ "${PREVFILE}" != "" ] && rm ${PREVFILE}
[ "${CURRFILE}" != "" ] && ln -sf ${CURRFILE} ${PREVLINK}
ln -sf `mktemp ${BASEDIR}/XXXXXXXX` ${CURRLINK}Apache config:
Assuming that the mod_rewrite module is already in use, you need to add just three lines:
RewriteCond %{REQUEST_URI} ^/(.*)magic/(.*)/(.*)$
RewriteCond /var/tmp/%2 -f
RewriteRule . /%1/secret-directory/%3 [L]where magic is a fixed fake directory name that does not exist anywhere in your web site, but just indicates that the following element of the path is the marker, and secret-directory is the real location of protected files.
Example:
RewriteCond %{REQUEST_URI} ^/(.*)/dkH893oW/(.*)/(.*)$
RewriteCond /var/tmp/%2 -f
RewriteRule . /%1/precious-downloads/%3 [L]If you are not very familiar with the almighty Apache URL rewrite engine, here is what is going on here:
The first RewriteCond directive tells Apache to check if the requested URI contains the directory /dkH893oW/, followed by another directory. If it does, the second RewriteCond jumps into action and checks if a (marker) file matching the name of that second directory exists in /var/tmp. If the marker exists, the RewriteRule directive makes Apache build a new URI with those two directories replaced with precious-downloads and stop processing rewrite rules. (“Internally” means “without redirecting the browser”, as opposed to so called 3xx redirects, so the visitor has no means to obtain the rewritten URL.)
If you are wondering what else Apache URL rewrites can do, you may find many more mod_rewrite tips and tricks at AskApache.com.
PHP function to read the current marker:
<?php
define('TEMPLINKDIR','/var/tmp');
function getTempLinkMarker() {
$currLink = TEMPLINKDIR.'/curr';
if (!file_exists($currLink))
trigger_error("templink marker does not exist", E_USER_ERROR);
if (!is_link($currLink))
trigger_error("templink marker is not a symbolic link", E_USER_ERROR);
$currFile = readLink($currLink);
if (!file_exists($currFile))
trigger_error("templink marker points to a non-existent file", E_USER_ERROR);
return basename($currFile);
}
?>