How we became a managed hosting customer
Whenever I tell someone that we are currently using a managed hosting service, people are eager to know more about the how and why. This post should give you an overview over the “why”.
When I first started working at vaamo, the IT infrastructure consisted of two physical servers (running OpenNebula) which hosted a couple of VMs. Alltogether I’d say about 25 VMs + two physical servers. All instances were managed by Chef which was a relatively new tool for me since I have only been working with Puppet for a couple of years. Considering that background, Chef was quite overwhelming, as you can imagine; so for the first couple of weeks I felt like this about every two hours:
Chef seemed to be relatively nice in the beginning (new stuff is often cool, until you know enough to understand the downsides) but after realizing that it needs far more services to be functional than vaamo needs for their core services, I promised myself to keep the infrastructure as simple as possible. Here and there I need to remind myself of this (or get reminded by Timo) but most of the time it’s working like that whenever I evaluate a new piece of software:
- Can I automate the setup of the tool? (I want to be certain that I can bootstrap said service even at 2 a.m.)
- Is it stable? (Ironic reminder: Github stars == stability)
- Is the effort worth the benefit? (A complex tool which does an acceptable job (examples would be possible but not necessary) but looks like a train wreck from all angles isn’t what I want to keep me from sleeping)
At this point you might have guessed that I made some awful experiences with being woken up in the middle of the night in the jobs I had before. Interestingly enough this is also quite a topic at the current monitorama conference.
Complexity and growth will find a way into your infrastructure sooner or later and it won’t get better if you need to take care of multiple HA setups which often aren’t primary dependencies of your companies’ core services and therefore are not part of the company goal. But I’m straying from the main topic. Let’s continue.
Shortly after I got the confidence to actually do changes in Chef on a daily basis, we needed to expand the infrastructure. This need for a larger infrastructure came mainly from one decision: to be able to support multitenancy on an infrastructure level.
The implications for the operations part are mostly clear:
- We need to clone stuff
- Everything we configure needs to be under version control (since we must be absolutely sure about the state of the changeable parts)
- We need to keep certain parts (configuration files, deployables) mutable …
or let’s say:
- The changing parts must not be changed manually but only by an automated process
which doesn’t sound too much like immutable infrastructure (which is something we aim for but currently cannot implement). I’m straying off again …
So, what do you do if you basically have no time? After we evaluated our options (AWS, DigitalOcean, Rackspace, VMWare, OpenStack … all of them managed and unmanaged), we chose to go with managed hosting in an environment which we don’t have to administrate (no monitoring alarms for empty RAID controller batteries anymore \o/) and containing nothing but our core-/auxiliary-services (like fileserver and database), monitoring/metrics and - of course - configuration management (BTW: We aren’t using Chef anymore).
Reaching this conclusion cost us quite some time and a lot of talking, both internally and also with external companies which provide hosting at various locations. At the end we picked Hostserver and were quite happy with the way they approached and helped us to master this migration without any major incidents.
But why exactly did we choose to go with Hostserver? For a couple of reasons:
First and foremost: They’re a company located in Germany which also hosts their infrastructure in data centers located in Germany. Some of our tenants are quite eager to keep their stuff on German soil (which is something a part of me absolutely agrees with). If you work for such a company you’ll know it’s not that easy to host mission critical systems on any kind of US cloud provider.
Another reason was the feeling they gave us at every step of the migration. Having competent people on the other side of the phone isn’t something you get for free these days.
However, from the first day on, it was clear that it is a temporary solution since managed hosting is way more expensive than just hosting a couple of VMs on any kind of cloud provider. Also the process of changing the infrastructure is quite heavy since we communicate via Tickets/eMails most of the time.
We wanted to use the time with Hostserver to gain information and mature our services/infrastructure to a level where we can host it again on places where VMs might not live forever. We are currently in the process of building the new infrastructure and maybe this is worth another blogpost.
If you want to chat about what you just read:
malte-codecraft at vaamo.de or twitter