Julia Language Gaining in Enterprise Cred
The Julia programming language, originally designed for mathematical and scientific workloads, is gaining converts among a range of enterprise users, with rapid adoption taking place in China.
One of its most interesting use cases: The Celeste project at Berkeley Lab, which aims to catalog all of the telescope data for the stars and galaxies in the visible universe. Its Julia-based application performed an 8 billion-parameter Variational Bayesian Inference analysis of 188 million stars and galaxies, processing 178 terabytes of data in just 15 minutes.
In its most recent programming language rankings, Redmonk analyst Stephen O’Grady remarked on its steady growth in the past four quarters and dubbed it among the languages to watch.
Julia started as a research project at MIT in 2009, created by computer scientists Jeff Bezanson, Stefan Karpinski, Viral Shah and MIT professor Alan Edelman. It was spun off as a company and introduced to the public in 2012. Around 2015, commercial users, especially in financial services, began making inquiries about its applicability to their unique workloads, according to Keno Fischer, chief technology officer and one of the co-founders of the company Julia Computing.
“They came to us wanting to make sure it was robust,” he said.
Just this month, the open source project issued its release of Julia 1.0, with the years leading up to the release spent ensuring that robustness, stability and the ability to provide the usability guarantees enterprise users expect, Fischer said.
While traditionally, programming languages offered a hard trade-off between high performance and high productivity, its creators wanted it to go fast and also be easy to use, Fischer said.
Languages that go fast are designed by computer scientists, people close to the hardware, who know how to get the most out of it, he explained. Languages that are easy to use were designed by users, statisticians, scientists.
“The strength of the Julia team was having both kinds of people,” he said.
The result is described as delivering the speed of C++ and Java with the high-level productivity, simplicity and ease of use of Python and R.
Multiple Dispatch
In terms of ideas, it’s closest to Dylan, which was developed years ago at Apple, Fischer said. In terms of application, it’s similar to Matlab, R, SAS or Python. It’s focused on mathematics and linear algebra capabilities from the scientific world except it’s designed to be “non-scary” to program for those coming from a scientific background.
Its core principle is dynamic multiple dispatch, in which in which a function or method can be dynamically dispatched based on the runtime type or some other attribute.
Fischer explained it this way:
If I write A + B in my code, in a dynamic language, A and B may be many things — numbers, matrices, strings, database objects — a variety of complex things. But what plus means is that context depends on both A and B. If A and B are both numbers, you just add them together. But if A is a number and B is a matrix, it’s not so simple. Finding the optimal way to execute that is referred to as multiple dispatch.
“It’s a paradigm well-suited to expressing mathematical concepts,” Fischer said. “Then through a process of optimization, we extract static type information from these model dispatch declarations and use that to optimize the code much like a static compiler would.”
Julia relies on a sophisticated type system.
“Julia has a very rich notion of types that you can select at runtime… and most importantly these are first-class objects within the language that you can operate on,” he said. “You can pass them around, you can transform them, you can perform operations on them, you can annotate them. The types are one of the most carefully thought out and most critical innovations in Julia.”
Going back to intuitive and static path languages, he explained, “The reason you can’t take static languages and make them fast, apply them to traditional dynamic languages is that the dynamic languages don’t have enough information available statically to take advantage of those techniques. That’s one of the key things why that’s not true in Julia because of the type system. The type system allows us to statically reason what behavior the Julia program will be, give that information to the compiler and have it generate fast code.”
Big in China
More than 700 volunteers contributed to the 1.0 release. Among its features:
- A new built-in package manager
- canonical representation for missing values.
- The built-in String type can now safely hold arbitrary data.
- The ability to extend broadcasting to custom types and implement efficient optimized computations on GPUs and other vectorized hardware.
It was downloaded in more than 140 countries in two weeks following release with 34 percent of unique visitors to the Julia download page from China.
Julia is being used to power self-driving cars, robots, 3D printers and applications in precision medicine, augmented reality, genomics, energy trading, machine learning, financial risk management and space mission planning. It’s used in government agencies including the Federal Aviation Administration, the Federal Reserve and others; by financial services firms such as Blackrock, Aviva, Citibank, Bank of America; and by Silicon Valley players including Google, Amazon, Facebook, and Uber.
The company Julia Computing offers a supported version of Julia in Julia Pro as well as specific products aimed at financial services, databases and time-series workloads and more. It will be consolidating its offerings in the coming months, Fischer said.
