Broken Promises

As you evaluate different cloud providers, it is important to understand the different concepts providers can use to deploy multi-tenancy. Different concepts facilitate—or limit—the way in which a provider can respond to changes in the service needs of clients.

General Purpose Clouds:

For example, some vendors design their clouds as commodities. They focus on providing low cost access to computing power in flat, homogenous environments. This type of general purpose cloud can scale quickly and easily to support large numbers of similar users. As they become saturated, however, you may begin to see variations in performance, as some users expand their usage and experience spikes that place constraints on all other uses.

Performance variations can affect computing power, storage and I/O or network traffic. Most providers already have solved performance problems associated with sharing VM RAM and CPU power, and most have deployed one or more of the many solutions for storage and I/O performance issues. Consequently, network performance is usually the first noticeable bottleneck. While it is important to know how your provider will handle performance variations wherever they appear, it is especially important to know how network issues will be handled. 

The Concern: Network Latency:

Networks experience varying levels of latency based on where the users and their data reside and how much bandwidth has been allocated each user. The easiest solution to network issues within a cloud is to physically separate heavy users from lighter users. This means moving the heavy user to a private cloud where resources can be adjusted to meet the requirements of peak periods, more users and new applications. 

The Answer: Scalability and SLA:

To reduce your risk of incurring more costs from your cloud provider, look for an enterprise provider that has scalability at every level of the cloud—SaaS, PaaS and IaaS. And look, too, for a provider offering a Service Level Agreement that addresses the performance requirements for the services most important to your business. These are the attributes of an enterprise level provider with the elasticity to meet your future needs.

One of the largest benefits that an application developer can get out of a cloud-based infrastructure is the opportunity to design for variable scale.  Specifically, you can start off small (with a limited number of virtual machines, using limited host resources), and then expand your environment as usage grows.  Conversely, you are able to shrink your infrastructure consumption during non-peak times. While some of this flexibility can be applied to existing legacy applications, the real win can be for newly developed systems.   

To get this benefit, there are some fundamental architectural principals that need to be followed:  loose coupling of system components, distributed system design and automated application installation / configuration. A solid architecture should reasonably scale from fitting the entire application onto a single VM to sharing it among hundreds (or even thousands) of VMs supporting the users. 

To achieve the loose coupling and distributed design goals, you need to decompose the architecture into units of functionality and think through how they will distribute work within the system.  Each component should be designed to support multiple instances of that application service within the environment. By doing this, you can load balance the application load as you need to scale. 

This decomposition should happen at all layers. It does no good to scale out web servers if a singular application server will become the performance bottleneck.  And definitely be sure to think through a scaling strategy for your databases.  If you plan on using a traditional relational database platform (RDBMS), consider setting up your identity columns in a way that will support future distribution of load through sharding techniques.  Another alternative is to use multiple read replicas, with a single write-enabled database instance. If you plan on going the route of NoSQL, be sure that you understand the scaling dynamics of the selected platform.

Achieving automated application installation and configuration builds on your distributed design. The key to ensuring that you can do achieve this architectural goal is to classify virtual machines into roles.  Role definitions will let you relate one server to the other servers in the environment. Using a “web server” role as an example, perhaps any server in that role needs to know what database server to connect to.  And just to relate this idea back to the point about determining a database scaling design, that database target might be different for different web servers. 

Once you have a good understanding of how you plan to deploy a highly-distributed version of your system, it's time to automate your installation and configuration. These are critical tasks if you want to achieve value from a dynamic infrastructure environment, because you need to match the speed that you can install and configure an application with the speed that you can provision new infrastructure.  Your software should be installable via command line, and you should look at different options to automate the configuration of the installed applications.  

While you may want to take these concepts to the extreme, my best advice for a new application architecture is to start simple.  Let these ideas guide your design, but remember, you’re main goal is to get the new application deployed for your users!

There's been a ton of commotion recently with everyone getting excited about Google's App Engine and Amazon's EC2. Rackspace has been getting into the mix as well with Mosso. It's pretty exciting stuff, and has the potential to take a significant market share from the more traditional hosting models (shared, dedicated, managed, etc...).

Lew Moorman (from Rackspace) argues that there will always be a place for the other models. I actually think that he's right in the short to medium term, but fundamentally, computing power has become a commodity. It's getting to the point where individuals and forward thinking organizations are abstracting the details of hosting their applications away from themselves, in effect reducing the level of system administration knowledge that they will have to keep on staff. Like it or not, that's not going to stop.

Programming language evolution is a perfect example of how this abstraction moves certain technical knowledge domains out of the mainstream and back into being a speciality. How many programmers do you know that actually use any sort of assembly language programming in their work today? I'll assume not many. Instead, programming has been abstracted away from the hardware and into the domain of solving user problems.

Looking at the hosting market as it stands today, we are going to see some serious change over the next decade (if not sooner). Today, many companies are outsourcing the server and network administration work to an ever decreasing number of service providers (isn't consolidation fun?). This is the same principle as the programming example: companies are abstracting the detailed infrastructure work away from their staff. But in many ways, it isn't really abstract as much as the executive suite would like to think. Most enterprises still outsource in "server units" or "network device units", ensuring that they have to think in terms of physical or logical servers.

Now look at "cloud hosting". It's another abstraction layer for application infrastructure, outsourcing most of the challenges of the physical systems completely to the providers.

So here's where it's going... if cloud hosting providers can solve a couple of key problems, then they are in a position to help enterprise buyers move to this new model. Issue number 1? Security. In this world of industry competition, government regulation and consumer data protection concerns, this is of paramount concern to enterprise buyers - which is why it should matter to cloud hosting companies. These hosting companies should start to have a dialog with their various technology partners and potential customers about the following topics: Data Privacy, Authentication, Authorization, Logical "Tier-ed" Separation of Application Layers, Private Network Transport. The sooner this conversation starts, the sooner the problems can be solved.

NOTE: This posting talks about the hosting business... which my employer is actively involved in. The ideas expressed above are my own, and have absolutely nothing to do with the company I work for.