I was working with Twilio for another project and got the idea that it might be fun if I could send pictures and text messages to a magic phone number and have them appear as updates to a static web site. The idea turned into a thing where anyone could send a text to a magic number and start a site of their own.
I usually use more modern languages now, but I'm a user and admirer of Perl from way back and I figured it might be a good fit for at least prototyping such a simple idea. I already had httpd set up to serve Perl CGI scripts, so it was a quick way to get started. I guess if you're looking to build something similar you'll probably use different technology, but the ideas should be relevant at least.
I didn't want to pull in the whole world for a little script, so I tried to stick to the Perl core modules for dependencies.
#!/usr/bin/perl use Digest::SHA qw(hmac_sha1_base64 sha1_base64); use File::Temp qw(tempdir); use HTTP::Tiny; use POSIX qw(strftime);
CGI is how we used to write web apps back before web apps came with their own
web servers. The web server will run our script when it receives a request for
it. By convention, this will usually be a request for an URL like
http://example.com/cgi-bin/script but other configurations are
possible depending on the web server. The web server will pass the request body
(if any) to the script as its standard input and will expect the script's
standard output to provide (at least some of) the headers and the body of the
response. The request headers are passed to the script through environment
variables.
I was saddened to read that CGI.pm had been deprecated and was no longer
included in the Perl core modules. The thing I read suggested using one of the
modern web frameworks, but I chose to go a different way. I wrote a little
library that would get request parameters from either an
application/x-www-form-urlencoded request
body (which is commonly used for POST requests) or from the QUERY_STRING
environment variable (which we would expect for GET requests). I didn't fuss
with chunked encoding or multipart/form-data request bodies, but
you can if you want. The request parameters end up in a hash
%params by name and value. Less processed parameters are available
as @params or $params as an escape hatch in case the
usual processing is inappropriate for some particular case.
# Get parameter string from body (neither chunked encoding nor
# multipart/form-data are supported) if present or from query string otherwise
our $params;
if ($ENV{'CONTENT_LENGTH'} && $ENV{'CONTENT_TYPE'} =~
m:^application/x-www-form-urlencoded:) {
read(STDIN, $params, $ENV{'CONTENT_LENGTH'});
} else {
$params = $ENV{'QUERY_STRING'};
}
# Split parameters from string on ampersand or semicolon
our @params = split(/[&;]/, $params);
# Split each parameter into key/value and decode
our %params;
foreach my $param (@params) {
# Pluses can be decoded into spaces before splitting, but hex
# decode should wait since it could make a spurious equals sign
$param =~ s/\+/ /g;
# Split on the first equals sign
my ($key, $value) = split('=', $param, 2);
# Hex decode key and value separately
$key =~ s/%(..)/pack('c', hex($1))/ge;
$value =~ s/%(..)/pack('c', hex($1))/ge;
# If a key is given more than once, only the last value is kept
$params{$key} = $value;
}
I also included a little function in my CGI library to send an “internal server error” response and terminate the script. I use this as a convenience for handling exceptional situations in my script.
sub cgi_die {
my $message = shift || 'Unspecified error';
print "Status: 500\n\n$message\n";
exit;
}
I already had a Twilio account set up, so I just made a new project, added a number to it, and put $20 in to get started. If you're looking to follow in my footsteps, getting set up with Twilio and reading their documentation may well be the longest part of your journey. I can tell they try to make it as simple as possible, but there's just so darn many things you can do with their service… a good problem to have, I guess.
When texts come in to my number, Twilio sends my web server requests for my script. I set up the URL for my script when I configured the number. Twilio signs their requests and I figured I should check this signature so my Internet friends can't prank me by sending their own requests to update my site. Twilio expects the responses to their requests to be in something called TwiML, which is XML. Check out their documentation to find out all of the cool things you can do with TwiML.
I made another little library with a couple of convenience functions: one to check Twilio's signature and another to wrap a TwiML response with appropriate headers and XML stuff. Twilio signs their requests using my authorization token, which I have my web server pass to my script in an environment variable.
sub twilio_signature_valid {
my $data = "https://$ENV{SERVER_NAME}$ENV{REQUEST_URI}";
foreach my $key (sort keys(%params)) {
$data .= "$key$params{$key}";
}
my $hash = hmac_sha1_base64($data, $ENV{TWILIO_AUTH_TOKEN}) . '=';
return $hash eq $ENV{HTTP_X_TWILIO_SIGNATURE};
}
sub twiml {
print <<'END';
Status: 200
Content-type: text/xml
<?xml version="1.0" encoding="UTF-8"?>
<Response>
END
print join("\n", @_) . "\n</Response>\n";
exit;
}
I didn't want the files or directories my script would create to be writeable by any user except the web server, so I set the file mode creation mask early in my script. I also checked that signature we were just talking about.
umask 07022 or cgi_die("Could not set umask: $!");
twilio_signature_valid() or cgi_die('Could not validate signature');
I realized that I didn't want every message received by my magic number to go on my site: I only wanted messages from me to go there. I came around to the idea that maybe it would be cooler if each sender's messages would go on to their own site that they could share with other folks. Ope! My fun little project was already growing.
I wrote a bit of code to turn the sender's number (given by Twilio as the From parameter) into a sender ID (by way of salted hash) and use that to look up a directory that holds their site. For new senders, a directory is made up on the spot and associated with their ID.
# Site directory from sender phone number or channel address
my $senders_dir = 'senders';
my $sender = sha1_base64($params{From}, $ENV{SENDER_SALT});
$sender =~ y:+/:-_:;
my ($site, $site_dir);
if (-e "$senders_dir/$sender") {
open SH, '<', "$senders_dir/$sender" or
cgi_die("Could not open sender: $!");
flock SH, 1 or cgi_die("Could not lock sender: $!");
$site = <SH>;
close SH or cgi_die("Could not close sender: $!");
$site_dir = "../$site";
} else {
$site_dir = tempdir('BLXXXXXX', DIR => '..') or
cgi_die("Could not make site dir: $!");
chmod 0755, $site_dir or cgi_die("Could not chmod site dir: $!");
$site = (split '/', $site_dir)[-1];
open SH, '>', "$senders_dir/$sender" or
cgi_die("Could not open sender: $!");
flock SH, 2 or cgi_die("Could not lock sender: $!");
print SH $site;
close SH or cgi_die("Could not close sender: $!");
}
I wanted to have pictures. Twilio doesn't send the pictures along with the request (thankfully), but they do send URLs you can use to get the pictures if you want. The Perl core modules include an HTTP client called HTTP::Tiny which is well-documented and a delight to use.
# Mirror message media to site directory
my $http = HTTP::Tiny->new();
for (my $i = 0; $i < $params{NumMedia}; $i++) {
$params{"MediaContentType$i"} eq 'image/jpeg' or next;
my $filename = "$site_dir/$params{MessageSid}-$i.jpeg";
my $result = $http->mirror($params{"MediaUrl$i"}, $filename) or
cgi_die("Could not mirror media: $result->{content}");
}
I also wanted to have words. Twilio does send the text of the message along with the request, so I just went ahead and saved it.
# Save message to site directory
open MH, '>', "$site_dir/$params{MessageSid}.txt"
or cgi_die("Could not open message: $!");
flock MH, 2 or cgi_die("Could not lock message: $!");
print MH $params{Body};
close MH or cgi_die("Could not close message: $!");
With the new pictures and/or words in hand, my script was ready to build (or
re-build) a page for them to live on. I didn't want to serve a half-built page,
so I started out by making a file to work in called new-index.html.
I planned to rename it into place as index.html when it was
finished.
# Open new index with exclusive lock
open IH, '>', "$site_dir/new-index.html"
or cgi_die("Could not open new index: $!");
flock IH, 2 or cgi_die("Could not lock new index: $!");
Usual stuff for the beginning of the page:
print IH <<"END"; <!DOCTYPE html> <meta charset=utf-8> <meta name=viewport content="initial-scale=1"> <title>$site</title> <link rel=stylesheet href=../style.css> END
I thought it would be nice to have the most recent messages at the top of the page, so I got a list of messages in reverse-order of when they were received.
# List messages by mtime descending
my @message_files = sort { (stat $b)[9] <=> (stat $a)[9] }
glob("$site_dir/*.txt");
I thought I'd like to show when each message was received, so I formatted the timestamp into something more human-readable and put it on the page. I put an image tag for each picture that came with the message, then got the message text and put it on the page (escaping any HTML, of course).
foreach my $message_file (@message_files) {
# Message file mtime, formatted for humans
my @parts = localtime((stat $message_file)[9]);
$parts[4]++;
$parts[5] += 1900;
$mtime = sprintf '%5$d/%4$d/%6$d %3$d:%2$02d', @parts;
print IH "<h1><span>$mtime</span></h1>\n";
# An image tag for each picture
my $basename = (split '/', $message_file)[-1] =~ s/\.txt$//r;
for my $media_file (glob "$site_dir/$basename-*") {
$media_file = (split '/', $media_file)[-1];
print IH "<p><img src=\"$media_file\">\n";
}
# Slurp message lines with shared lock
open MH, '<', $message_file or
cgi_die("Could not open message file: $!");
flock MH, 1 or cgi_die("Could not lock message file: $!");
my @message_lines = <MH>;
close MH or cgi_die("Could not close message file: $!");
# Message lines with HTML escaped
my %entities = ('&' => '&', '<' => '<',
'>' => '>');
foreach (@message_lines) { s/([&<>])/$entities{$1}/g }
print IH '<p class=message>' . join("<br>\n", @message_lines) . "\n\n";
}
I finished out the page with a little blurb and some contact information.
print IH << 'END'; <address> This independent <a href=/>SMS static site</a> is hosted by<br> Parks Digital LLC <support@parksdigital.com> </address> END
With the page safely built, I could then rename it into place to be served.
# Rename, release lock, and close
rename "$site_dir/new-index.html", "$site_dir/index.html"
or cgi_die("Could not rename new index: $!");
close IH or cgi_die("Could not close new index: $!");
I wanted to get a text message response to let me know that my message had been received and my site updated. And also to let me know what the URL was since it would be randomly selected. So for a response to Twilio I gave a little bit of TwiML asking them to send a message in reply to the sender giving them the address of their site and encouraging them to share it.
twiml(
'<Message>Added to',
"https://$ENV{SERVER_NAME}/$site",
'-- share it!</Message>'
);
This was a fun project to do and I've been enjoying updating my site from time to time. I like that it's there when I want it, but not demanding my attention.
When I shared this project on Hacker News, a few folks asked if the source was available. In chatting with them, it sounded like they were interested in learning how I built this so they could build something a bit different. In light of that, I thought an article about what I did and why might be more helpful than just a code-dump.
I hope that this has been helpful and interesting for you! If this is the kind of thing you're into, you might enjoy my other articles.
Aaron D. Parks