Not another cloud billing horror story
There are plenty of horror stories with obscene bills from cloud providers. Fortunately, I am not in that category, but I don’t want to become complacent either.
Old infrastructure
GetSite started with a hybrid infrastructure. Some services were deployed on Hetzner with Rancher, and others were split between AWS and MongoDB.
On the MongoDB cloud, I was using Atlas and ran a serverless instance for the database. The invoices started to ramp up, but not in direct proportion with the traffic we were getting. At first, it was a couple of monthly bucks and very quickly increased to $40 / mo. As you might have guessed, this was a strong motivator for moving from Mongo.
Before making the move, I tried to figure out what inflated the invoices so much. The database instance was getting 20 req/s at all times, and had no idea why. The official support was quite unresponsive and not helpful in this regard.
Mind you, this spike in requests started about a month ago, however, the traffic to GetSite didn’t increase. Something’s fishy if you ask me, but I didn’t want to spend more time looking into it.
On AWS, I had a few Elastic Beanstalk instances that were quite expensive compared with Hetzner. The instances were part of an autoscaling group.
Did I mention I was using Rancher 1 for the self-hosted services? It is a great piece of software, but it was discontinued a while back. Rancher 2 is very complex to use and geared towards enterprise users and big clusters.
New infrastructure
Enter Coolify, the successor to Rancher. I have been trying to find a replacement for Rancher for a few years. All container managers (Portainer included) seem to be tailored to the enterprise. I read about Coolify a few months ago but wasn’t convinced. Their website is not doing them justice.
A couple of weeks ago, after the inflated MongoDB invoice, I decided to jump in and install Coolify on a Hetzner server. The onboarding process is a bit unintuitive, but it grew on me as I started using it.
I moved all services from AWS, MongoDB, Rancher, and some old MySQL and Redis servers to Coolify in a day. Spent another day setting up workers, load balancers, backups, and security, and testing everything out. All of this while keeping 100% uptime (ok… maybe like 99.99%).
Costs
I cut costs down to about $40 / month for the whole infrastructure. That’s how much the last MongoDB invoice was, for reference. While bringing the cost down, I also beefed up the server capacity. The current setup has plenty of spare room for future growth.