Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

Benchmarking Amazon EC2: The wacky world of cloud performance

Peter Wayner | May 2, 2013
Before turning to the world of cloud computing, let's pause to remember the crazy days of the 1970s when the science of the assembly line wasn't well-understood and consumers discovered that each purchase was something of a gamble. This was perhaps most true at the car dealer, where the quality of new cars was so random that buyers began to demand to know the day a car rolled off the assembly line.

Time of day? Yes, the engineer said. He really wanted to know whether we were paying attention to when we run the tests. Sure, operating systems are an obvious source of performance differences, and using the latest drivers and patches always makes sense. But he also wanted to know the time of day.

This was new. CPUs have clocks that tick extremely quickly, but they'll stamp out computation at the same rate morning, noon, and night. Was it really important to watch the time? Yes, he said. In other words, the cloud machines may be more like a '70s-era Detroit assembly line than a Swiss watch.

Cloud Camaros and Corvettes

To find out just how variable the cloud might be, I fired up instances on Amazon's EC2. I chose Amazon because it's one of the industry's leaders with one of the most sophisticated clouds, but the conclusions here are probably just as true for any of the other clouds -- they're all playing the same game. They're building a superfast rack of computers and finding a clever way to share it with many users. Even though the sales literature makes it seem as if you're buying another computer like the one sitting on your desk, it's usually closer to buying into a time-share for a condo at the beach.

At this point, I should pause and put aside these glass-half-empty allusions to Detroit and real estate time-shares. A gracious defender of the cloud would say that sometimes you get what you pay for, but often you get lucky. On a bad day, you could end up sharing a CPU with 10 crazy guys trying to get rich minting Bitcoins by melting the processor; on a good day, the CPU will give you many extra cycles for free. You could end up sharing a machine with 10 grandmothers who only update the Web pages with a new copy of the church bulletin each Sunday. In other words, the CPU glass is not only half-full, but the bartender will often top off your tumbler with the last remnants of the bottle too.

The results were surprising. I purchased two kinds of machines: the low-end T1 Micro instance and the medium version called appropriatedly an M1 Medium. The Micro is a test machine with 613MB of RAM and a promise of "up to two EC2 Compute Units (for short periodic bursts)." The Medium comes with 3.75GB of RAM and a promise of one "virtual core with two EC2 Compute Units." The Micro is listed at 2 cents per hour and the Medium costs 12 cents per hour. Both of these prices are the list for an "on demand" instance available to anyone walking in the door. Lower prices are available for those who make a longer-term commitment with a reservation.


Previous Page  1  2  3  4  5  Next Page 

Sign up for CIO Asia eNewsletters.