Distributed Supercomputing Before the Cloud: The Legacy of SETI@Home
After over two decades of hunting for a radio signal from aliens, the SETI@Home project is hitting the “pause” button on a historic scientific mission that, for decades, relied on donated CPU cycles from millions of volunteers to scan the skies for possible signals from other life forms in space.
It’s been a remarkable journey for the crowdsourced calculations that spawned, not only a search for extraterrestrial aliens but dozens of additional “distributed” research projects. So, it’s a good time to look back at all the science we humans have been able to successfully perform. As this particular epoch comes to an end, it also offers an opportunity to consider how our technology has evolved. SETI@Home launched in a world before the cloud could handle our big data problems.
Will future historians look back at the effort and marvel at our ingenuity?
Taking a Break
A recent article in Wired called the SETI@Home project a “distributed supercomputer,” one harnessing the spare CPU cycles of participants’ home computers, in effect creating a virtual supercomputer dedicated to a humongous task, such as searching the data from a “state of the art multi beam receiver” radio telescope in Puerto Rico.
Yet despite all the advanced technology, the project needs to take a break, and “Unfortunately, it’s not Because They’ve Discovered Aliens,” joked astronomy site Universe Today.
“Scientifically, we’re at the point of diminishing returns,” explained an announcement on SETI’s page. “Basically, we’ve analyzed all the data we need for now.” And in addition, “It’s a lot of work for us to manage the distributed processing of data. We need to focus on completing the back-end analysis of the results we already have, and writing this up in a scientific journal paper.”
“However, SETI@home is not disappearing… We hope that other UC Berkeley astronomers will find uses for the huge computing capabilities of SETI@home for SETI or related areas like cosmology and pulsar research. If this happens, SETI@home will start distributing work again. We’ll keep you posted about this.”
On the project’s forum, SETI co-founder David Anderson called the hibernation “long overdue in my humble opinion,” saying it would free up some time “and hopefully let us finish this project before the sun goes red giant.” Wired notes the program only has four full-time employees.
“Now we move on to the last stage: deciding what scientific conclusions we want to make, and figuring out how to make them,” he wrote. “Finding E.T. would be a scientific conclusion. Failing that, we want to make (and quantitatively support) a statement about the ‘sensitivity’ of our search — a statement of the form ‘if there were a radio beacon in frequency range X, in part Y of the sky, with power at least Z, our search would have detected it with probability P…'”
SETI@Home was launched on May 17, 1999. But early on the project partnered with the legendary Planetary Society, co-founded by Carl Sagan to further space research — and the Society now calls it “the most successful public participation science project in history.” As of 2012 more than 6 million people across 226 countries have participated. Wired reports estimates of 25 trillion calculations per second, more than twice as powerful as the world’s best supercomputer at that time, as the project’s chief scientist Dan Werthimer acknowledges “We didn’t anticipate how fast it would grow.
“It grew exponentially and I think it’s because people are really excited about the question of whether we’re alone,” Werthimer asserted
And a pretty awesome closet it was! Let us know if you are in the area and we can show you around the new @brkthroughprize #Listen lab, though now with less purple plastic =) pic.twitter.com/jqY5DxwOQ2
— UC Berkeley SETI (@BerkeleySETI) March 7, 2020
The project, initially funded by the Planetary Society with a $50,000 grant, now relies on Berkeley Open Infrastructure for Network Computing (BOINC), an open-source platform that remotely manages the spare computing resources. Supported by the National Science Foundation, BOINC is now also being used by about 30 other projects, according to its home page, including MilkyWay@home (which is modeling our home galaxy) and Einstein@home (which searches for weak astronomical signals). Other projects are studying everything from pulsars to global warming.
Most recently, the Rosetta@home project is currently using donated computer cycles to more accurately model the important proteins of the coronavirus. In one case the group accurately predicted the atomic-scale structure of one protein several weeks before it could be measured in a lab. This work could eventually be used to guide the design of novel vaccines and antiviral drugs.”
The home page for SETI@Home now points visitors to other BOINC-based projects. For example, the University of Dayton and Wright State University “MindModeling@Home” project is trying to model human cognitive processes. And in Spain, San Jorge University is creating electrophysiological simulations of the heart.
Another option is Science United, which allows volunteering your computer for broad kinds of science rather than specific projects. Also being directed by Anderson, this program “has the advantage that new projects can get computing power without having to do their own publicity and volunteer recruitment,” according to its webpage, expanding the availability of virtual supercomputing “to thousands of scientists, rather than a few dozen as was previously the case.”
As the Planetary Society explains, “Thanks to SETI@home, researchers have begun to tap into the almost unlimited resource of public enthusiasm for science.”
Looking at a Legacy
SETI’s home page shares a link to a research paper from the University of Geneva about why people participated in the program. And on Twitter, volunteers shared fond memories.
“I was one of the first to load that software on all six of my computers running a BBS at the time!” remembered one participant. “It encouraged my love of science which I have passed onto both of my kids. Thank you for what I can only describe as an era-changing movement.”
Google Chrome Engineer Adrienne Porter Felt shared that she also had fond childhood memories of running the program on a home computer, adding “It made me feel like I could be part of science.”
And data scientist Mario Rugeles called it “The end of an era. Seti at Home was doing Big Data decades before people even knew the term.”
But “the world’s greatest extraterrestrial hunt is far from finished,” argues Wired, sharing an interesting idea from Eric Korpela, the director of SETI@home, who suggests after at least a year it might be possible to relaunch the program with data from other telescopes like the MeerKAT array in South Africa or China’s FAST telescope.
But Wired’s article also cites another project crunching radio signals from space: Breakthrough Listen (which in February shared nearly 2PB of its data online). Every day its telescopes generate another 100TB of data, explained Steve Croft, Breakthrough Listen’s project scientist at the Green Bank Telescope, adding “It’s just not feasible anymore to go over the internet to individual users.” Instead, it was time to “bring the computers to the data.” The new plan? Deliver the radio signals from space to the cloud, where it can all be turned over to machine-learning algorithms.
Croft calls cloud computing “a glorified version” of SETI’s original distributed supercomputer.
- Could humans colonize Saturn’s largest moon?
- The legendary PARC research center turns 50.
- Scaling a Dungeons & Dragons-playing AI to handle a million users.
- Cloudflare looks at the history of the URL.
- Right-to-repair comes to the farm.
Feature image: SETI.