NCSA HPC Data Center Workshop

I was fortunate enough to attend the first day of the NCSA's biennial workshop: Building the Data Center of the Future. It was a great event and absolutely worth every penny; it was completely free! My notes from the event are presented below.


It's 07:44; check-in begins in 16 minutes. Having scoped out the breakfast continental, I'm firmly set on acquiring one of the little wine-glass parfaits.

Break. The parfaits were for another group; I sincerely wish I'd attempted bribing one of the servers. The IBM speaker was a regrettably poor orator: his presentation and reasoning were sound, but his tone and manner were tedious and not the least bit engaging. However, I very much enjoyed the juxtaposition of IBM's presentation, which alleged that water cooling is the only way to proceed in high-end computing applications, with Microsoft's presentation, in which Shaun Harris, Microsoft's Principal Power Systems Architect, argued for highly modular, containerized, air cooled data centers with sparser computer nodes running hotter, rather than denser nodes running cooler. Indeed, according to Shaun, he was tasked by his manager to question everything in data center best practices, leaving no stone unturned, to produce their fourth generation data center solution. Really, the two talks made for a very nice gestalt.

Lunch. The last presentation, by Nova, was about the construction and design of the new National Petascale Computing Facility (NPCF), which will house the Blue Waters sustained petaflop supercomputer next year. It was an excellent presentation, and seeing the pictures of the facility made the idea of visiting it this afternoon all the more exciting.

A member of the National Human Genome Research center, two NCSA members, a professor at SIU, and a professor at UIUC sat at my table. Two interesting problems came up in conversation: the first was the difficulty of storing and moving enormous amounts of data, made particularly difficult by the memory practices of scientists, who wish to preserve data far beyond what seems useful to IT staff; the second was the difficulty of running code with tight synchronization requirements in inherently asynchronous environments, such as the "cloud," thus preventing the use of the web as a highly distributed computing platform for most scientific research. The first of these problems seemingly requires the application of brute-force: more bandwidth, more storage. However, this, as told by my fellow attendee, is simply not the case. The second of the two problems appears as though it could be solved by layered abstraction, a very fundamental idea in computing. However, again, I am told this approach is insufficient.

As I didn't wish to be a nuisance, I didn't press matters further. Still, that won't stop me from investigating the circumstances and background of both problems to form my own conclusions. For the moment, this has me terribly distracted.

19:04. The ominously scheduled "three-hour tour" of the Blue Waters facilities was awesome. The NPCF is enormous and fantastic; I've never seen a room so large as that of the nearly-empty data center floor. Of course, the mechanical and electrical facilities in the basement were positively epic, and the hum of the many 110 kV and 33 kV transformers was deafening; our tour guide was forced to yell over the noise. On top of all this, we also visited the Abbot Power Plant, capable of generating over 80 MW or powering nearly the entire campus, and the university's central water chilling facility, which provides air handling service to most of the campus, of which Blue Waters is merely a single, albeit large, recipient. Frankly, it was a lot to take in, but without seeing all three sites, it would be impossible to fathom the monumental effort that went into Blue Waters, let alone grasp the complexity of the system or the collective expertise required to maintain it.

At the end of the tour, we returned to the NPCF for the evening reception. Looking out the glass façade from the floating staircase of the NPCF, Merlot and oeur d'oeuvres in hand, I couldn't help but think this would be the first and last time I would ever be allowed to set foot in that beautiful facility. Of course, very few people ever will. Indeed, entry requires a magnetic card, which, in turn, activates an iris scan, but this only gets you into the anteroom. The rotating door into the facility, itself, contains a pressure sensitive plate in the floor which determines the weight of its occupants. In the event that multiple people should attempt to simultaneously enter the facility, their combined weight would exceed the weight of the authorized worker whose card and iris were scanned, and they would remain locked in the antechamber. Of course, there exist other scenarios in which this mechanism might prevent entry (a la Demolition Man), but I doubt the facility's personnel will want to give that much thought.

All in all, it was a fantastic workshop, and considering registration was free, I can't understand anyone not wanting to attend. Perhaps I'll be able to attend the next one in 2012. Who knows? I may again, one day, set foot inside the NPCF after all. And what a treat it would be with Blue Waters fully operational!