Heroku investigation (DRAFT IN PROGRESS)

Heroku Overview

https://devcenter.heroku.com/articles/dyno-types

Web workers

Current AWS:

• Run on one t2.medium EC2, which is 4G of RAM, and 2 virtual cores

• Running 10 web workers (passenger). Looking at passenger-stats, each workers RAM usage can range from 115M to 250M

Considerations

Would probably switch to use puma instead of passenger. Somewhat more standard also somewhat more flexible. One reason we weren’t using it is has slightly more management overhead we hadn’t figure dout, in a way that isn’t an issue on heroku. Puma lets you run multiple processes each of which has multiple threads, which can give you more total workers in less RAM. Although there are limitations, may allow us to get by with LESS RAM, but for now we’ll estimate similar RAM needs.

Estimated Heroku

To get to 4G RAM would be 4 “standard 2x” dynos, each of which has 1GB RAM and (maybe?) 4 vcpus. We could probably run 3-4 non-threaded workers in each, for 12-16 workers, actually more than we have now? This may be over-provisioned with capacity, but we could possibly use more capacity than we have on current AWS, and we’ll estimate this for now.

Background Job Workers

Current AWS

One t2.large EC2 which is 8GB of RAM, and 2 virtual cores. Running 12 seperate jobs workers, some reserved for specialty queues.

Considerations

• This is actually a pretty huge amount of RAM, expensive on heroku

• One reason we needed this much RAM is that our “package all pages into a PDF” routine takes a lot of RAM that scales with number of pages – since there's not really an affordable way to get 8GB of RAM on one dyno on heroku anyway – we may just have to do development work to maintain the “all pages as PDF” feature regardless, figuring out how to make it use constant RAM, or eliminating the feature.

• We would possibly also (eventually?) switch from resque to sidekiq, which, using multi-threaded concurrency, can in some cases have more workers in less RAM. (Very similar to passenger to puma switch)

• But for now just for a rough estimate, we’ll estimate we really do need 8GB of RAM, split between various dynos

Estimated Heroku

8 standard-2x each with 1GB of RAM, we could probably run at least 16 workers, if not 24 or 32. We might still not be able to run our current all-page-PDF routine without figuring out how to make it use constant RAM. if we go tiny on staging, it will be very slow to ingest, is that ok? For now, we’ll say yes, but still do at least a standard-2X