Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.

India’s Space Victory: Chandrayaan-3’s Successful Moon Landing

On Aug. 23, India became the first country to land a spacecraft near the moon's south pole.
Sep 3rd, 2023 3:00am by
Featued image for: India’s Space Victory: Chandrayaan-3’s Successful Moon Landing

More than 380,000 miles from Earth, near a patch of lunar soil where the sun never sets, India made history.

On Aug. 23, India became the first country to land a spacecraft near the moon’s south pole. The Associated Press described it as “a historic voyage to uncharted territory that scientists believe could hold vital reserves of frozen water,” as well as a “technological triumph for the world’s most populous nation.”

India now becomes the fourth country to “soft land” a craft on the moon — after the Soviet Union (1959), the United States (1962), the Soviet Union, and China (2013).

The landing “drew cheers at watching parties around the country,” reports the Guardian — And also in the mission’s control room and in Indian newsrooms. CNN reported that “Locals burst firecrackers, distributed sweets in the streets and danced enthusiastically to celebrate the achievement.”

There were exuberant street parties, the Guardian adds, while “Temples and mosques held special prayers for a safe landing. On the banks of the River Ganges in Varanasi, Hindu monks bestowed blessings on the mission and blew conch shells.” Reuters notes that local schools “organized live screenings for students.”

Sellers on Etsy are already hawking Chandrayaan-3 t-shirts. As the Guardian later report reported, “From Bollywood stars to cricketers, everyone wanted to express their delight.”

Even the next day, one newscast began by saying “As we speak the country is still bathing in the afterglow of the milestone that it reached yesterday

But beyond that exuberant moment, are there lessons to be learned — about perseverance, leadership, and maybe even project management? Landing on the moon ultimately required a process of high-stakes iterations, that real-world debugging that forms the essence of scientific progress.

So while their successful moon landing represents a chance to celebrate what’s been achieved, maybe it also offers an example of how challenges should be approached…

Learning to Fly

India’s space program is aiming for inclusivity. The Guardian cites estimates that already between 20 and 25% of the Indian Space Research Organisation’s (ISRO) 16,000 employees are women, writing that “approximately 54 female scientists and engineers have been involved in the Chandrayaan-3 mission.”

But its landing in August culminates more than 60 years of faith and perseverance. The lander was named after the Indian physicist/astronomer Vikram Sarabhai who first helped found the Indian Space Research Organisation in 1962. “They had no resources to speak of, a small pool of scientists, and barely any funding,” remembers the India Times. “In fact, for the first rocket they launched a year later, they were transporting the parts to be assembled by bicycle.”

Fast forward to this July, and the organization had crafted a massive apparatus weighing nearly 8,600 pounds (3,900 kilograms). That included a 4,735-pound propulsion module (2,148 kilograms) plus a 3,862-pound lander module (1,752 kilograms) — all to carry and deliver a moon rover weighing 57 pounds (or 26 kilograms).

Chandrayaan-3 launch vehicle taking off (via Wikipedia)

Yet the Guardian notes it was launched into space on a rocket less powerful than America’s Apollo moon missions of the 1960s, with Chandrayaan-3 first orbiting the earth several times just to build up extra speed and still taking over a month to reach the moon.

ISRO graphic of Chandrayaan-3 trajectory

ISRO graphic

That represents the second big surprise of India’s space organization. Reuters notes that the whole mission cost just $75 million (6.15 billion rupees). In fact, even this year India’s space program has a budget of just $1.93 billion, making it the eighth-largest in the world, according to figures from Statista. (The U.S. space program’s budget is the world’s largest at $61.97 billion, followed by China at $11.94 billion…)

Space program budgets around the world - from statista 2023

The New York Times suggests the streamlined organization may even have been an asset. “Analysts said that a key to ISRO’s success had been freeing itself more than other institutions from bureaucratic constraints and bloated staffing that comes with state funding.”

Due to the high cost of purchasing space technology, much of the equipment was developed internally by the ISRO, including the camera-based and laser-based systems, ISRO’s chairman said in an interview on August 24. (Adding that it will be shared with interested entrepreneurs for commercial space operations.) Editors at the Los Angeles Times hailed it as “an endearing example of the ‘jugaad‘ mindset of Indian engineers, which describes problem-solving with frugality and creativity…”

“Here’s to the country that transformed a shoestring budget into cosmic success, reminding us all that the universe favors the bold, but especially those who are both innovative and wise.”

Successes and Failures

Last month ISRO chairman S. Somanath explained what went wrong with their predecessor mission, Chandrayaan-2, on Sept. 6, 2019.

One Hindi-language YouTube channel explained this was mostly a software failure, summarizing Somnath’s explanation in a recent public appearance. Until the lander was within about 4.3 miles of the moon, it relied on its inertial navigation system for positioning (a series of gyroscopes and accelerometers operating without the benefit of an altimeter).

The next crucial operation involved calibrating its other navigational instruments (including altimeters and cameras), so the mission’s designers planned to keep the spacecraft steady — and programmed the software to avoid decreasing thrust. But unfortunately, the 2019 landing had required higher-than-expected thrust — so when the calibration phase ended, it wasn’t able to adjust fast enough. (An additional problem: the controls tracking time before landing was off by a few seconds…)

Chandrayaan-2 crashed 400 meters from its landing zone.

According to the Times of India, Somanath said the 2019 mission had a much smaller site targeted for landing — just 500 x 500 meters (about 546 yards). “This time we have loaded the lander with more fuel, provided a slew of safety measures and kept a bigger landing site to ensure a successful landing.” For Chandrayaan-3, the landing target was expanded to 2.6 miles by 1.5 miles (4.2 km by 2.5 km). And Somanath said later its final touchdown ultimately happened within 985 feet (300 meters) of the center of that site, according to ABC News.

But that wasn’t the only change. Chandrayaan-3 was upgraded to “instantaneous” thrust control for all phases of the landing. (NASA points out the descending lander had 12 different engines that could be throttled as needed.) Some instruments are new (like the Doppler velocimeter), while other just report positions more frequently, and the end result is a landing craft capable of adjusting its position much faster. The video credits “faster, better algorithms” for aiding the landing of Chandrayaan-3. But even the legs on this lander are stronger…

“We looked at sensor failure, engine failure, algorithm failure, calculation failure,” Somanath told the Times of India. “So, there are these different failure scenarios calculated and programmed inside.” Somanath described it as moving from “success-based design” — anticipating one single, optimistic outcome — with a much more flexible “failure-based design.”

One example? The ISRO added extra solar panels on different surfaces of the landing craft to make sure it can generate power no matter which side ends up facing the sun.

In the end, Somanath also said that along with high-resolution photos of the entire moon, that earlier mission had provided “very clear” images of the landing site, so “this time we know the site well in advance, as all the boulders and craters have already been mapped.” And the 2019 craft that delivered that earlier lander is still orbiting the moon to this day.

And in a kind of poetic justice, it was able to look down and photograph the successful landing of Chandrayaan-3.

‘We Should Do All of This

On Thursday, Aug. 24, ISRO released footage of their rover triumphantly rolling down its ramp to the moon’s surface.

ABC News noted that the rover “could help determine if there is oxygen and hydrogen on the moon.”

But that’s just the beginning. In September India plans to launch a mission to the sun, CNN reports. And later India will also attempt a manned lunar mission, Reuters reported (citing Somanath).

But the real lesson may be understanding the importance of passing along a legacy, since in an interview on Aug. 24, ISRO’s Somanath was already looking ahead to even more ambitious explorations in the future. “There are many more missions we should do — it’s not limited to the moon. We need to go to Mars, we need to go to Venus, we need to understand other planets, sometimes go out of the solar system… So all these are needed. We should do all of this.

“These are stepping stones of achieving that. For our generation, this is a step.

“Then for the next generation, there will be many bigger steps.”

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.