Tuesday, April 01, 2008

Miichael Manos Keynote at Data Center World

The Michael Manos keynote this morning at Data Center World was inspiring. Michael doesn't read off of note cards and you can tell that he not only knows this material inside and out, but that he is passionate about it. I gained a number of things from this talk -- including him mentioning that their new Chicago facility will have upwards of 200 shipping containers pieced together like the RV campsites like I have talked about in previous posts.

Here are my notes from his talk:

  • Microsoft challenges: 15x growth on servers, 9x growth on egress, 15x growth on power, 3x growth on number of data centers.
  • 30% of the audience had active programs to monitor and measure power in their data center
  • Next to nobody in the audience knew what their carbon emissions were for their data center
  • Sustainability regulation 'is' coming ; not if, but when
  • How many are working on green programs? 30% (audience)
  • Industry challenges/minefields:
  1. sustainability reporting & efficiency reporting
  2. data center inventory globally becoming a challenge
  3. increasing power densities at the rack level
  4. power costs
  5. green-washing
  6. expertise shortage
  7. organizational structures
  8. increasing capital cost barriers
  9. innovation hoarding
  10. heterogeneity versus homogeneity mindsets.
  • The 'old' power draw was 4kW / rack ; now is 8-12 kW / rack ; some implementations 36kW / rack
  • $11-$18mil for a tier2 / tier2+ data center
  • Cost of a data center:
    • 82% construction costs : mechanical/electrical
    • land: 2%, core&shell cost 9%, architectural: 7%
  • 40,000 to 80,000 systems inside a typical 500,000 sq ft data center
  • Staffing at a Microsoft data center: 35 people across 24x7x365 shifts.
  • Boarding process: trucks pull up with pre-populated racks, drive into colo room, connect and go
  • Driving automation is key ; have to have good operational practices.
  • 35 factors for site selection. Including: Internet population, Internet peering/network, mobile users, power pricing, power capacity, environmental, construction costs, tax climate, IT labor availability, corporate citizenship -- this gives them their composite heat map. Quincy wasn't necessarily an ideal site until they went to the county level on a second pass and determined its benefits.
  • Metrics: PUE vs DCIE : Microsoft uses both ; dcie shows the overhead ; goal, for either is ONE
  • Microsoft sets aggressive pue targets internally.; they try to half the distance to a PUE goal by one every two years.
  • Scribe (their software tool) ; shows/measures PUE, DCIE, total racks, total servers, kW power consumed, critical load power, etc.
  • They have 1 million points to monitor/measure across data centers
  • They have been measuring PUE for 4+ years ; an old data center of theirs started at 2.4 PUE and is at 1.9'ish in aug. 2007 ; very little if any capital improvements in this data center -- all driven by operational efficiencies.
  • Chicago - their FIRST container data center!
    • Entire first floor is full of containers ; each container houses 1,000 to 2,000 systems per container ; 150 - 220 containers on the first floor
    • Containers are parked at 45 degree angle ; due to trucks parking them
    • They build 'spines' of systems that connect power/cooling/bandwidth to their containers

8 comments:

Dave Ohara said...

Good summary of Mike's presentation. I referenced your talk on my own blog. http://www.greenm3.com/2008/04/attendees-comme.html

Anonymous said...

They monitor ~ 1 Million points but between all of their DC, not per DC.

The container system use intrigue me as I am not sure why would I build a facility of that size and then load it up with containers.
I still have to connect power and cooling but I then have to add another layer of fire protection and alarm independent of the building coverage and I wonder how will they handle disater recovery operations on the containers under cooling contingencies?.

Anonymous said...

Nothing new: SUN, IBM, Google all use the shipping container form factor. Google even owns a patent on that (why such a simple "invention" could be patented is another question, tough)

Anonymous said...

This is cool in one respect alone - it's mobile. The data center economies of the future will favour specific locations that may change at short notice. The ability to move data locations easily will trip up 'had-wired' data centers.

Anonymous said...

It seems data centers are a hot topic. Yes, pun was intended. There is tremendous pressure on data centers to add capacity, and justify the power they use. There are some basic things managers can do to get almost 2 orders of magnitude more capacity from the space and power they have. Upgrade, consolidate, and use every watt. I have posted a bunch of resources and information in the server room.

Anonymous said...

I am in agreement with David, I understand the use of a container for a small site, emergency data center, or mobile application, but I don't understand the data center RV park application. There are modular panel solutions that can be used for a quick build & expandable large scale data center that provide protection from theft, water, fire, acrid gases, EMI & RF interferences while not limiting the levels of capacity & redundancy we can achieve.

gre3nfly said...

I imagine the Container works well for the larger organisation that can standardise network and server specs across multiple business groups. It must also be a great way of identifying cost centres -container No.1 = xyz user etc. – but let’s be honest, it is a clever spin on a rather antiquated approach in itself – a heavy shipping box inherently inflexible outside its predefined parameters! The vision must be to create data centres that are not wasteful, inefficient silo products, but ones that responds to the needs of the process in hand - M&E responding to server needs load factor etc in an integrated fashion and M&E design playing second base to network topology - it is more energy efficient to have a network and software redundant approach than simply rely upon N+XYZ M&E configurations, I understand the Container approach of the larger organisations are looking at a maximum redundancy of N+1 for core apps, and N for equipment that either doesn’t need redundancy or can be replicated elsewhere. This is a far more sustainable design - using fewer natural resources and producing less waste energy and is the next step in the design evolution of the Data Centre.
Do Containers offer constraints in as much that you are cooling racks in series, hence you have to have the air cooled to the rack that is working hardest, irrespective of the actual load on the other racks? Parallel cooling within a data centre should offer greater opportunities for matching cooling to rack load – this is in its infancy at present however but must be the next step – will we revert to variable speed fan driven racks though take out the fans in the servers etc?
In tandem we need visionaries in the Planning and Fire Code industries worldwide to adopt to the needs for change in building design for data centres . Modern data centres are designed around codes that are not applicable to a mostly unmanned environment (It wouldn’t surprise me if containers are seen as mobile plant items and therefore overcome escape and code issues currently).
The Container is a good wake up call for the industry which has been lead for too long by M&E design rather than systems integration - Looking forward to the change.

Anonymous said...

搬家 搬家 搬家公司 搬家 搬家 票貼 借款 徵信社 徵信 補習班 多益 留學 在職進修 婚紗 補正下着 新娘秘書 汽車旅館 室內設計 室內設計 中古車買賣 植牙 坐月子 婚禮佈置 催眠 派報 漆彈 視訊會議 真空成型 Shade sail nike shoes MBA 在职研究生 在职博士 關鍵字廣告 租屋 花蓮民宿 花蓮民宿 花店 花店 花蓮民宿 租房子 xo醬 牛軋糖 牛嘎糖 代償 房屋貸款 信用貸款 失眠 減肥 糖尿病 日立家電 外遇 離婚 抓姦 外遇蒐證 外遇抓姦 侵權 仿冒 應收帳款 工商徵信 徵信 徵信社 外遇 徵信 徵信社 外遇 电动隔膜泵 自吸泵 化工泵 离心泵 磁力泵 螺杆泵 水泵 隔膜泵 气动隔膜泵 白蟻 花蓮民宿 美國留學 新娘秘書 汽車旅館 新娘秘書 催眠 花蓮民宿 搬家 搬家服務 搬家保障 搬家網 搬家估價 徵信 徵信的意義 徵信服務 徵信報導 徵信問答 徵信知識 婚禮佈置 婚禮佈置 http://www.life13.com 票貼 工作服 班服 團體服 糖尿病 勞工體檢 呼吸照護 資源回收 生日禮物 美國遊學 留學代辦 彩妝造型 新娘秘書 宜蘭民宿推薦 催眠 漆彈場 太陽能熱水器 減重 自助洗衣加盟 綠豆椪 床墊 創業加盟 資源回收 鋼戒 廢車回收 創業加盟 龍眼蜜 買房子 買房子 班服配件 團體服配件 團體服 班服 團體服 班服 團體服 眼鏡 創業加盟 室內設計公司 室內設計公司 室內設計公司 最好的婚禮佈置,婚禮佈置花園感官花園婚禮佈置,婚禮佈置最新消息,婚禮佈置專業多元化,為你量身定作完美婚禮佈置,完美的婚禮佈置請來感官花園。婚禮佈置時間雙休請來電預約,婚禮佈置周五正常營業。婚禮佈置地點遼寧街,婚禮佈置聯絡方式電話。婚禮佈置訂購方式,婚禮佈置請提前兩天通知,婚禮佈置請先完成付款及傳真或郵件傳送匯款單。婚禮佈置免費外送價格,婚禮佈置請參考列表。