New Features in the new Send to Dropbox

The day is finally here! Almost 3 years of nights and weekends went into the back end and new website for Send to Dropbox. Here’s a brief change log.

  • Date formats can now be customized to your liking.
  • You can now choose to overwrite existing files, or index their filename. e.g. foo.text => foo (1).text.
  • Automatic unzipping of archives has been fixed.
  • There’s now a support ticketing system to better handle issues that may arise.
  • The backend has been re-architected to increase efficiency and minimize downtime. No more waiting hours for your attachments.
  • The new website no longer random stalls at a blank screen when signing in.
  • The new website now correctly refreshes your authorization token.
  • Accounts can now be fully deleted.

Along with these changes, we’re excited to now offer Pro account subscriptions. Pro accounts are $3 a month or $29 a year. Here’s what you get.

  • Completely customizable address. No more random characters at the end. Get the full address you want!
  • Create multiple addresses with their own unique settings.
  • Email whitelists! You can now set up filters to only accept attachments from addresses that match your filters, which helps keep your Dropbox safe!
  • Priority support. Your support requests will move to the front of the line and be handled first over basic account requests.

A lot of work went into the new accounts, and we made a special effort to not take anything away from existing users by enforcing bandwidth/email caps or promoting free features to pro.

Furthermore, we’ve received a lot of donations since Send to Dropbox started, and you’ll be happy to know that those donations have not gone unnoticed. If you’ve donated to us in the past, you can sign up for a pro subscription, and redeem your donation for a 2x credit towards your account. Since all subscriptions come with a 1 day trial, you can upgrade your account and immediately apply your donation without your card being charged. To redeem your donation, simply click on “Your Account” and then click the “Subscription” tab. At the bottom of the form, you just need to enter the PayPal email address you donated with to redeem your credit.

Thanks for your continued support and a brighter future for Send to Dropbox.

Tags: Dropbox Email Pro

How Node.js Streams saved Send to Dropbox

Around three years ago, I started Send to Dropbox as a side project, written in PHP. At the time, I was a decent front-end programmer, but a bit of a novice when it came to literally anything else (infrastructure, scaling, protocols, RFCs etc). So with my Dropbox API key in hand, I wrote a hokey and barely working first version of Send to Dropbox. It was bad. Really bad.

The backend consisted of a couple hundred lines of PHP that would check a local Dovecot email server, loop over all of the messages, distribute the attachments, and then delete them. This script ran once a minute via cron job. As you can assume, there were problems. For starters, PHP typically puts a 30 second cap on the execution of any script. Easy enough to fix right?

set_time_limit(3600);

Boom! Fixed. Not really. This worked fine while word of Send to Dropbox was making its way around Twitter. However, shortly thereafter, Send to Dropbox made its way to the top of Hacker News. The site became wildly popular in a short amount of time, and all of the sudden my backend script was running longer than 60 seconds on each execution. This created a problem of having multiple instances of the script execute at the same time, stepping on each other’s toes and sending attachments twice. Again, no problem, I’ll just create a file when the script starts, and delete the file when the scripts completes. A semaphore if you will. That worked for a bit, until the site got so popular that all the mailboxes filled up, filling up the server and causing the script to crash. Ugh, what to do now.

Typically, having lots and lots of traffic and users is a good problem to have right? Perhaps. But this was a side project. I wrote this after staying up like two nights in a row.

This is where I started. Fast-forward to 2013. Enter Node.js. Enter streams. Now, there were a lot of different back ends between the initial prototype in June of 2010 and now, and I’ll spare you the details, but needless to say, I’ve learned a lot. Since the original prototype, I’ve written an OAuth and Dropbox library from scratch, a RFC compliant streaming email parser, a streaming SMTP server and an entire new front end. So why streams? What did Node.js streams afford me that I couldn’t get in PHP or Ruby?

One thing that was abundantly clear after some time is that I don’t want to be responsible for what is sent through my service. Send to Dropbox’s door is wide open, so-to-speak. You can send any kind of file through, and it gets processed. You can imagine then, how this may be a liability. Node’s streams allow me to never have to write an email to disk. The moment your outgoing email server starts transferring an email to Send to Dropbox’s SMTP server, it is parsed and piped directly through to Dropbox’s API. Everything is done in memory. This allows me to not have to directly store any email content at any time. (Phew!)

Another thing that became an issue with past implementations was memory usage. I was running this service on a Linode 512, and it was routinely running into memory issues from loading and parsing large emails (30MB+) in a single process. Streams mitigated this issue by parsing and streaming emails in real time. In other words, its now entirely possible that the start of a large email has been streamed to Dropbox and garbage collected before the end of the email even reaches Send to Dropbox’s servers. This was great! After switching to a streaming architecture, I saw my memory usage drop from hitting the limit of 400ish MB and crashing, to humming along smoothly for days at around 150MB.

Overall, Node.js was a perfect choice for the backend of Send to Dropbox. The ability to handle a lot of I/O elegantly was a Godsend for the service, which would have likely gone belly up otherwise. Thank you to everyone who hung in there with me during the rockier and less stable past of Send to Dropbox, and look forward to a 99.9% uptime future.