You deploy the application on your production environment and… surprise: doesn’t work!
Antipattern: start from your dev environment and scale up adding more servers, load balancers, db replicas etc…
While this seems a standard way to design and release a new application, it will just postpone the issues the you will have when you will scale up your infrastructure to production.
Let me clarify: at a first glance, this can be confused with “over design”, and we all know this is wrong, but you know that application will run in a production environment, serving real users, so you have to design starting from your production environment, and solve the issues that arise here first (fail fast and learn fast, as always).
The recipe: scale down from production
Here is a list of design considerations, emerged from experiences at my company.
One of our application, based on PHP+MongoDB, run on “standard” setup for a production environment: one domain name is resolved to 2 different IPs, allocated to 2 load balancers running HAProxy; the load balancers direct the traffic to 2 or more web servers running Apache+PHP, connected to a 3 nodes MongoDB replica set (no need for a shard yet). All the machines are virtual machines, built using RighScale templates.
Using this methodology, can “scale down” the infrastructure simply collapsing the 3 layers (LB, FrontEnd, BackEnd) in less virtual machines, still using the same full stack: for instance, our “Integration Tests” environment runs the full stack on a single machine, but HAProxy and a MongoDB replica set are included, even if not required. The apparently complexity of this setup is simplyfied by using “Server Templates” and virtual machines, than can be instantiated from scratch with just a simple click of a button.
The development environment is a virtual machine too, running on each of our laptop; we are working to have the dev virtual machine built from a RightScale template too, but at least we can share a “master” image to all the evs when something changes in the setup (an article on that coming soon…)
This practice saved our life a few times:
- Starting from v.1.3, the MongoDB driver for PHP requires a different configuration for replica set: we discovered that BEFORE going in production enviroment (yes… we should read the change log…)
- We have cron jobs that must run as “environment singleton” (just job running in a given enviroment): again, we discoverd this need BEFORE going in production and addressed it.
- Because we have multiple front end web servers in production, we need to address the “session management”: the final decision was to design the application “sessionless”. If we would have followed the classic approch, we should have addressed the “session management” in prodution enviroment in some way: storing sessions on DB, using server affinity in load balancer, etc… in summary: a more complex infrastructure.
Conclusions
Starting from design your production enviroment and scale it down is a practice to anticipate the troubles, and solve them at “development” stage.
Usign modern technologies like server virtualization and automated infrastructure provisioning, simplify this practice.
