20 petaflop IBM supercomputer will rely on optical interconnects

Sequoia, a leading-edge supercomputer to be designed and built by IBM (Armonk, NY) and installed at Lawrence Livermore National Laboratory (Livermore, CA), will not only achieve 20 petaflop (quadrillion floating operations per second) speeds, but will rely on photonics--in the form of optical interconnects--to relay information back and forth between its 96 racks of electronics. It is conceivable that optical interconnects will even be used in some way to help link together the computer's 98,304 compute nodes--which together contain Sequoia's 1.6 million cores.

The computer is needed by the U.S. Department of Energy's National Nuclear Security Administration (NNSA; Washington, DC) to help ensure the safety and reliability of the nation's aging nuclear deterrent. It will also likely serve other purposes, such as computing for scientific research.

In fact, IBM will deliver two systems: Sequoia, a 20 petaflop system based on a future version of IBM's BlueGene technology, to be delivered starting in 2011 and deployed in 2012; and an initial delivery system called Dawn, a 500 teraflop (trillion floating operations per second) BlueGene/P system, scheduled for delivery in the first quarter of 2009. Dawn will lay the applications foundation for multi-petaflop computing on Sequoia.

With its 20 petaflop speed, Sequoia is expected to be the most powerful supercomputer in the world and will be more than 10 times faster than today's most powerful system.

Ron Favalli, a spokesperson for IBM, noted that the company will be using fiber optics to link together various parts of the computer "at all levels." This certainly means that rack-to-rack communications will be carried out optically. Although Favalli could not reveal any detail, it is interesting to speculate on a deeper level of optical interconnection--that between components in the same rack.

Fiber-optic interconnects were used for all rack-to-rack communications in the Roadrunner Petaflop supercomputer now running at Los Alamos National Laboratory (Los Alamos, NM), which was also built by IBM. The fiber-optic interconnects, which appear as bundles of slim orange cables, replace what would have been an almost unmanageable mass of electric cables. The Roadrunner, which can operate at a 1 petaflop speed, does not have intra-rack optical interconnection.

Sequoia and Dawn will serve NNSA's tri-lab Advanced Simulation and Computing (ASC) program, which unites the scientific computing resources and expertise of Los Alamos, Sandia and Lawrence Livermore National Laboratories. The Sequoia systems will be focused on strengthening the foundations of predictive simulation through running very large suites of complex simulations called uncertainty quantification studies. In addition, the machines will be used for weapons' science calculations necessary to build more accurate physical models. This work is a cornerstone of NNSA's Stockpile Stewardship program to ensure the safety, security, and reliability of the U.S. nuclear weapons stockpile today and into the future without underground testing.