Running a database in the cloud can be convenient, but it can also become expensive quickly. For the Prosopo team, MongoDB Atlas was initially a fast, reliable way to run a cloud database, but as our data grew, so did the bills. In the last year, we realized that we were spending thousands of dollars per month on infrastructure that we can run more efficiently ourselves. We explored various options, but ultimately decided to move our MongoDB deployment to Hetzner, a cost-effective cloud provider.
Here’s how we managed to cut our costs by 90% without compromising performance or reliability.
When we first started building, MongoDB Atlas was an easy choice. Everything was set up for us, and we could focus on building our application without worrying about database management. The free tier was enough for our initial needs, and as we grew, it became as easy as a few clicks. However, scaling up came with a heavy price. We went from paying $0 per month for a small database to over $3,000 per month for a few hundred GB of data. The cost breakdown before our stay was approximately as follows:
| Service | monthly cost | |
|---|---|---|
| Atlas M40 Instance – AWS | $1000 | |
| Atlas Continuous Cloud Backup Storage | $700 | |
| Atlas AWS Data Transfer (Same Region) | $10 | |
| Atlas AWS Data Transfer (various regions) | $1 | |
| Atlas AWS Data Transfer (Internet) | $1,000 | |
| Total + VAT | $3000+ |
Those of you with a more keen eye may have noticed the huge costs associated with data transfer over the Internet – it’s as much as the server itself! We are building Prosopo to be resilient to outages, such as the recent massive AWS outage, so we use several different cloud providers. This means that a lot of our database traffic goes over the Internet, which is very expensive on MongoDB Atlas, because it runs on AWS (there are other options available, but we chose AWS).
When your entire stack is on AWS, data transfer costs are minimal, as shown by the “Same Region” line item above. However, this architecture creates centralization and single points of failure, which we want to avoid.
Oh, and even though we were paying so much, we didn’t have access to any support! That’s a different payment plan.
Hetzner offered a compelling alternative:
- High-performance dedicated servers At a fraction of AWS pricing
- Predictable, flat-rate pricing no surprise bill
- Strong European data center presencethat is consistent with our privacy requirements
- No data transfer fees between our application server and the database
We decided the only way to get control of our costs was to move to a self-hosted solution. MongoDB was running in Atlas Replica Set mode, but we opted to set up a much better machine with a larger amount of RAM (256GB) and faster SSDs to handle our workload. In the future, if we want to go back to a distributed setup, we can always add more nodes.
The server in question costs $160 per monthWhich is a huge savings compared to our previous costs.
Our Atlas instance had about 500GB of data, but the product does not rely on this data in real time – it is used to generate recognition rules using ML modeling, and the rules are applied to our CAPTCHA providers in a follow-up step. The reason for this architecture stems from our blockchain beginnings.
We were able to take a mongodump of the database, restore it to the new Hetzner server, and then use the script to sync any changes that occurred during the migration window.
Setting up our Hetzner server wasn’t just a matter of turning on MongoDB. We ran Proxmox on the host machine, created an Ubuntu VM, and deployed MongoDB in Docker. Networking added another layer of complexity: the VM sits on a private subnet (10.0.0.x) with the host handling DHCP, NAT, and port forwarding. We also use Trafic as a reverse proxy to handle SSL and route traffic from the outside world to MongoDB, which saves us from wrestling with MongoDB’s own SSL certificates. Although this setup gives us flexibility and security, it requires a little more technical know-how than fully managed solutions.
When you migrate from MongoAtlas to a self-hosted solution, you take on more responsibility for managing your database. You need to make sure it is secure, backed up, monitored, and can be rebuilt in case of failure or need for additional servers. We used the following tools:
Monitoring in MongoDB Atlas was very useful. This highlighted the times when we were opening unlimited connections due to connection leaks in our application code. However, MongoDB Atlas won’t let us kill connection using admin shell commandsThis meant that we reached our connection limit and we had no way to restart the server. We had to shut down our services to free the connection. With our own server, we can now monitor and manage connections directly through OpenObserve and, importantly, the MongoDB shell.
Backups in MongoDB Atlas were also convenient but expensive. now we use mongodumpA cron, and a storage box from Hetzner to take daily backups of our databases. The backup is compressed and encrypted before being sent to the remote storage box.
We opted for the DIY approach to our servers and databases, which reduced our costs significantly and gave us more control over our infrastructure. However, this approach is not for everyone. We’re still a small team, and the limited number of servers we run is manageable.
The performance we are seeing from our new standalone MongoDB server is much better than the Atlas setup. This is simply because the specifications are very high and the index can be kept mostly in memory. We know there is something called vector search in MongoDB Atlas but we weren’t using it, so it wasn’t a factor in our decision.
For reference, the details look like this:
| specifications | Atlas M40 example | Hetzner Dedicated Servers |
|---|---|---|
| CPU | 8 vCPU | 8 core Intel Xeon W-2145 |
| to hit | 16 GB | 256 GB |
| storage | 380gb ssd | 4 x 1TB NVMe SSD RAID1 |
It’s more likely that as we grow, we’ll start to see value in hosted solutions again, but for now we’re quite happy with the new setup. Hopefully this experience will help inform others starting their cloud journey – for a little bit of pain you can save a lot of money!