A modern-day spaceflight software engineer paid his respects to the programmers who came before him more than half a century ago, and recreated the lost software that flew one of NASA’s moon-lander modules back in 1969.
By carefully and painstakingly resurrecting their assembly-language code, Mike Stewart has rescued a great piece of programming history. And by sharing it online, with a little help from his friends, he’s given us all a chance to remember and appreciate the challenges faced by the programmers of yesteryear — and just how far we’ve come in the decades since.
Stewart focused on a dramatic spaceflight that had happened early in 1969: Apollo 10. Its eight-day mission? Sending three astronauts into orbit around the moon — with two of them descending in a module to within 9 miles of the moon’s surface — in a “dress rehearsal” for the historic Apollo 11 moon landing that would take place two months later.
The software-controlled the lunar landing module — but where is that software now? For Apollo 11’s later historic moonwalk, those first astronauts to walk on the moon would abandon their moon lander altogether, leaving it behind in a slowly-decaying orbit, and it eventually crashed down onto the moon’s surface. But the previous Apollo 10 mission blasted its lunar lander up into space towards the sun. Less than a year ago astronomers finally spotted it again, now orbiting around the sun.
Unfortunately, it’s also carrying the only known copy of the software for its lunar lander.
Stewart’s friend Marc Verdiell once described that software as “lost to history,” calling its far-away capsule orbiting the sun “a difficult place to get it from.”
Not Lost in Space
But Stewart is uniquely qualified for the challenge since by day he’s an enthusiastic modern-day spaceflight software engineer at the private space-industry company Capella Space. He’s also held the same position at Google, and also performed a four-month internship working on simulations at NASA’s Ames Research Center.
And his friend Verdiell documented Stewart’s quest to recreate the software, in a 22-minute video for his YouTube show, “Curious Marc.” Verdiell also works on vintage computers — he’s one of the people working on the IBM 1401 restoration team at the Computer History Museum in Mountain View, California. (On his web page Verdiell describes himself as “a tech executive in Silicon Valley,” as well as a former Bell Labs researcher, an Intel Fellow and founder of several tech startups in Silicon Valley, all related to high-speed fiber optics communications.)
Fortunately, Stewart had an earlier version of the mission’s moon-lander software — “almost, but not quite, the version that flew on Apollo 10,” the video explains. But Stewart wanted the actual “flown version” of the software. The good news is he knew the same changes were made to a later version of the software. The bad news is he doesn’t have that version of the software either.
But a slightly later version (used two months later for the historic Apollo 11 moon landing) was derived from that revised version of the software and had presumably also included the changes. Somewhere…
Stewart had also found a memo describing the change — which had enabled the controller to more accurately calculate the moon’s gravity, incorporating something called “the Boeing R-2 Lunar potential model.” But the crucial clue ultimately came from Ron Burkey, who’s been interested in the flight controllers for nearly two decades. In 2003 Burkey had typed out every line of code for both the Apollo 8 and Apollo 13 missions from scanned images that had been uploaded by MIT, later using it to create a virtual Apollo Guidance Computer online. Burkey is now also an editor at the non-profit Apollo History Foundation, “dedicated to preserving the legacy of the Apollo Program by preserving the documentation of the program and making it permanently available for online public viewing.”
Burkey has been involved in the scanning of original NASA documents, many housed at the U.S. National Archives. In the video, Stewart explains how Burkey discovered a drawing which was a kind of Rosetta Stone, titled “the Computer Program Master Deck Tape.” For every version of the mission software run on the lunar modules’ guidance computers, it had a different six-digit number… A checksum.
I asked Curious Marc to explain why that’s useful — and first, he made sure I understood the terminology. The software had been broken up into small chunks of code called a “bank,” Burkey explained — “a section of 1,024 15-bit words in memory (about two KB in modern terms)” that could be swapped in as needed.
There were just 36 banks — about 72KB — of code, but that was still more than the CPU could access. “The erasable core memory (which we today we would call RAM), was divided in even smaller banks (about 512 bytes), which made programming really acrobatic!
“But it turned out to be a boon for Stewart’s recovery effort since he could isolate the parts of the code that had changed to specific banks.”
Now Stewart could make changes to the original code, “and if the checksums for the banks come out to be correct, then we probably did the right thing,” the video explains.
In an email later, Burkey reminded me that checksums are still used pervasively today. “You see it sometimes when you install downloaded software, and the computer first verifies the integrity of the download. All Ethernet traffic and disc data is checksummed.”
Checking the Checksums
Since Stewart had an earlier version of the software, he wrote a program that compares the “Rosetta Stone” numbers found by Burkey for the “flown version” of the software with the actual checksum numbers for the files he’d painstakingly re-created with a team of volunteers. The results? It found errors in just two banks. (Bank 11 and Bank 13). That meant the chunks of code in 34 of the 36 banks were already 100% correct.
But how could he correct the other two?
The checksum seemed to be off by more for Bank 11, so Stewart guessed that was where they’d added all those changes to the gravity model described in the memo. But for Bank 13, the difference in checksums suggested only a few bits of the data. Stewart looked at the “controlled constants” part of the code, “where they put all of the numbers related to physics or the mechanical bits of the lunar module, like geometries.”
And then on-camera, Stewart recreates what must’ve been a very dramatic moment. He knew that the later version (used two months later for the historic Apollo 11 moon landing) had included the change he was looking for. So he pulled up both versions of the code on his screen, side by side — and explains what he saw. “There is a group of constants here that are all related to the gravity model. And if we open the same thing… in the Apollo 11 code, all of these are the same — except for this line here.”
He cut-and-pasted that line of code into his recreation of the Apollo 10 software — and it made things worse. The difference in the checksums was greater. “So I was stuck there for a little bit…”
But there were hints in that memo describing the change, which indicated a new value needed for the variable TIMEDELT. Mike ran his test again using the new value — and the checksums matched. He’d solved the problem by adding one line of code, and updating the value of one constant.
“Now, we only have one bank wrong…!” he said triumphantly.
“So now we get to the hard part, which is the actual gravity model itself.”
But he caught a break, because it turns out that, at least for this section of the software, the gravity model code used for Apollo 11’s historic moon landing was identical to the code used in the “flown version” of the code for Apollo 10. “So we’re just going to copy-and-paste a whole bunch of code from Apollo 11.”
That code had been split into two different places in the Apollo 11 software — and in two different banks. But Stewart correctly guessed where it needed to appear in the Apollo 10 code — and proved it by running his program to compare the known checksum with the one for his newly-recreated program.
Success. Every checksum matched.
In the video, Curious Marc also shares his own take on a piece of history. The Apollo 10 mission didn’t carry enough fuel for a landing on the moon, which Marc jokes was because NASA knew that otherwise their Apollo 10 astronauts “would be way too tempted” to try landing themselves.
“But if they had the fuel — well, we now have the landing program, and we could prove that they could land.”
“It lands. We checked.”
- SpaceX’s software engineers answer questions on Reddit.
- What’s making Rust so popular? Stack Overflow asks its core team and Rust programmers.
- Top Linux kernel developer Greg Kroah-Hartman gives an enthusiastic new video interview.
- Source code released for two mid-’90s Command and Conquer videogames.
- The YouTube channel “When Phones Were Fun” remembers Samsung’s 2003 “Matrix Phone”.
- After 10 years, someone finally found that $1 million buried treasure chest hidden in the Rocky Mountains.
At this time, The New Stack does not allow comments directly on this website. We invite all readers who wish to discuss a story to visit us on Twitter or Facebook. We also welcome your news tips and feedback via email: firstname.lastname@example.org.