Broken Promises

As you evaluate different cloud providers, it is important to understand the different concepts providers can use to deploy multi-tenancy. Different concepts facilitate—or limit—the way in which a provider can respond to changes in the service needs of clients.

General Purpose Clouds:

For example, some vendors design their clouds as commodities. They focus on providing low cost access to computing power in flat, homogenous environments. This type of general purpose cloud can scale quickly and easily to support large numbers of similar users. As they become saturated, however, you may begin to see variations in performance, as some users expand their usage and experience spikes that place constraints on all other uses.

Performance variations can affect computing power, storage and I/O or network traffic. Most providers already have solved performance problems associated with sharing VM RAM and CPU power, and most have deployed one or more of the many solutions for storage and I/O performance issues. Consequently, network performance is usually the first noticeable bottleneck. While it is important to know how your provider will handle performance variations wherever they appear, it is especially important to know how network issues will be handled. 

The Concern: Network Latency:

Networks experience varying levels of latency based on where the users and their data reside and how much bandwidth has been allocated each user. The easiest solution to network issues within a cloud is to physically separate heavy users from lighter users. This means moving the heavy user to a private cloud where resources can be adjusted to meet the requirements of peak periods, more users and new applications. 

The Answer: Scalability and SLA:

To reduce your risk of incurring more costs from your cloud provider, look for an enterprise provider that has scalability at every level of the cloud—SaaS, PaaS and IaaS. And look, too, for a provider offering a Service Level Agreement that addresses the performance requirements for the services most important to your business. These are the attributes of an enterprise level provider with the elasticity to meet your future needs.

One of the largest benefits that an application developer can get out of a cloud-based infrastructure is the opportunity to design for variable scale.  Specifically, you can start off small (with a limited number of virtual machines, using limited host resources), and then expand your environment as usage grows.  Conversely, you are able to shrink your infrastructure consumption during non-peak times. While some of this flexibility can be applied to existing legacy applications, the real win can be for newly developed systems.   

To get this benefit, there are some fundamental architectural principals that need to be followed:  loose coupling of system components, distributed system design and automated application installation / configuration. A solid architecture should reasonably scale from fitting the entire application onto a single VM to sharing it among hundreds (or even thousands) of VMs supporting the users. 

To achieve the loose coupling and distributed design goals, you need to decompose the architecture into units of functionality and think through how they will distribute work within the system.  Each component should be designed to support multiple instances of that application service within the environment. By doing this, you can load balance the application load as you need to scale. 

This decomposition should happen at all layers. It does no good to scale out web servers if a singular application server will become the performance bottleneck.  And definitely be sure to think through a scaling strategy for your databases.  If you plan on using a traditional relational database platform (RDBMS), consider setting up your identity columns in a way that will support future distribution of load through sharding techniques.  Another alternative is to use multiple read replicas, with a single write-enabled database instance. If you plan on going the route of NoSQL, be sure that you understand the scaling dynamics of the selected platform.

Achieving automated application installation and configuration builds on your distributed design. The key to ensuring that you can do achieve this architectural goal is to classify virtual machines into roles.  Role definitions will let you relate one server to the other servers in the environment. Using a “web server” role as an example, perhaps any server in that role needs to know what database server to connect to.  And just to relate this idea back to the point about determining a database scaling design, that database target might be different for different web servers. 

Once you have a good understanding of how you plan to deploy a highly-distributed version of your system, it's time to automate your installation and configuration. These are critical tasks if you want to achieve value from a dynamic infrastructure environment, because you need to match the speed that you can install and configure an application with the speed that you can provision new infrastructure.  Your software should be installable via command line, and you should look at different options to automate the configuration of the installed applications.  

While you may want to take these concepts to the extreme, my best advice for a new application architecture is to start simple.  Let these ideas guide your design, but remember, you’re main goal is to get the new application deployed for your users!