Thursday, October 26, 2006

Putting My Gartner Hat On

It's back! It's back! Everyone seems to be exclaiming the return of a boom for the data center industry. has a nice article on the subject as well as a brief one at Nicholas Carr's blog.

My inbox is flooded with stories of companies expanding, building new data centers and record growth. On top of this there have been many stories on how there is a shortage of generators due to so many new facilities being built. Waiting periods of up to a year!!

I am by no means an expert, but here is my take on all of this. We have a record number of "small" companies building data centers that are either the right size for them or all that they can afford to build. Meanwhile the guys that really know how to do this (HP, Google) are not necessarily building 'more' data centers, but consolidating into a smaller number of HUGE data centers. Combine this with virtualization really catching on and automation helping out the "lights-out data center" and I really think we'll see all of these small data centers moving into mammoth mega-data centers in the next 5-10 years. With real estate and power being such big factors, I just think that economies of scale will come in to play and make it more economical for the mega centers to flourish over the small ones. Servers are becoming more and more hands off and it really won't matter where they are as long as you have good bandwidth and remote administration tools.

With journalists reporting on another 'bubble' in the industry happening and having lived through the last bubble I tend to get cautious with so much development going on. A sound business plan and the ability to adapt to fast-paced changes is crucial to survival.


Anonymous said...

I disagree. The "datacenter industry is divided into three sectors. Private facilities run by companies large enough to 'DIY' (your Google example). Colocation facilities occupy the middle market, and they can range in scale from large multi-facility companies like Equinix, to small regional or niche players such as my employer. At the small end are companies with their own server closet to computer-room-sized installation. These could be on their own site, or even located at a colo facility.

The situation we see at the moment is recovery from the over-supply condition in the middle market that started in 1999 and peaked in 2001. There just wasn't enough datacenter floor space prior to 1999, and by 2001 there was way too much. Classic Econ 101 oversupply condition. This caused prices in the middle market to collapse as the players remaining standing after so many fell over dead rushed to fill their spaces, or suffer a similar fate.

The IT industry however, has continued to grow, at least in terms of datacenter needs, at the same rate it has always grown. 1999 was an anomaly, where the industry saw ~200% growth, but other than that one year, the demand for datacenter space has grown at a remarkably linear rate. Only now the spaces in the mid-market are reaching capacity, and the lack of capital in the datacenter market to grow electrical and HVAC infrastructure at the same rate as the linear IT industry growth has come around to bite the datacenter industry in the keister. JUST as the demand for more space crosses over the supply availability curve.

We were all very gun shy about building or expanding facilities between 2001 and now. For good reason. Those of us lucky to move/upgrade our facilities in the down market will do OK on this supply side. Those who sat on their spaces, and sold them at a loss to stay in business will be hurt by it.

Perhaps your "mega-datacenter" future will come to pass 15 years from now, but somehow I doubt it. In every industry there have been happy niches for small and medium players, and I suspect that will be true for the datacenter business as well. The concept of computers being 'self-sufficient' and able to live in completely "lights out" facilities is laughable. That is only a reality for companies with the capital to invest in massively redundant systems that can tolerate the failure of 25% of their components and continue to operate. Software is far too failure-prone to be "lights out" in reality and small-to-medium businesses will never be flush enough with cash to build massively redundant systems. As such, there should always be a market for mid-sized colos with technologically savvy staff to manage IT infrastructure for such businesses.


John said...


Excellent observations....and I agree with many of the things you are saying.

I think I tend to discount or forget the really small operation that is maybe 3-5 racks worth of equipment.
This type of operation doesn't need to build their own data center, but they miss out on the N+1 availability that a colo offers. It's that decision point for a small to medium size business where they have outgrown their current infrastructure and they have to decide on spending all of the money on a new center, or to simply outsource it to a colo.

As far as the lights-out part I try to be optimistic. At the facility where I work a company has approximately 15-20 racks worth of equipment with KVM and remote reboot abilities. This combined with a fat pipe means the only reason they need to physically touch their racks is to remove or add servers. Now---I have also worked at a place where trips to the "computer room" were very frequent and necessary.

I like your economy analysis and think you are probably right on. The IT industry is certainly and interesting one to work in (I sure enjoy it).
Thanks for your comments.