Modal Title
Data

The Power of Graph Algorithms

A better understanding of how to use graph algorithms can benefit data scientists, software engineers, and project managers.
Mar 11th, 2022 10:00am by
Featued image for: The Power of Graph Algorithms
Feature image via Pixabay.
Julian Shun
Julian Shun, lead instructor of MIT Professional Education’s Graph Algorithms and Machine Learning course, is an Associate Professor of Electrical Engineering and Computer Science at MIT and a lead investigator in MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). His research focuses on the theory and practice of parallel algorithms and programming, with particular emphasis on designing algorithms and frameworks for large-scale graph processing and spatial data analysis. He also studies parallel algorithms for text analytics, concurrent data structures, and methods for deterministic parallelism. Prior to joining MIT, he was a postdoctoral Miller Research Fellow at UC Berkeley. His honors include the NSF CAREER award, DOE Early Career Award, ACM Doctoral Dissertation Award, the CMU School of Computer Science Doctoral Dissertation Award, Google Faculty Research Award, Google Research Scholar Award, SoE Ruth and Joel Spira Award for Excellence in Teaching, Facebook Graduate Fellowship, and best paper awards at PLDI, SPAA, CGO, and DCC.

A graph is a data structure consisting of vertices and edges. A vertex is simply any entity represented by the data (for instance: a social media user), and an edge is a relationship between those various entities.

Graph algorithms have practical applications in a wide array of fields. They can help social media companies to better understand interactions and relationships between users, help to streamline transportation and logistics, and help financial institutions to identify instances of fraud more quickly and accurately. However, graph algorithms are often poorly understood, even among many data scientists — a problem that will become more pressing as the data gathered and managed by organizations continues to expand.

Here are three types of stakeholders that can benefit from knowledge about and experience with graph algorithms.

Data Scientists — It may seem obvious that data scientists should understand graph algorithms. But these analytics experts often have their hands full with different types of data sets, problems, and solutions, and many simply haven’t yet had an opportunity to get up to speed with graphs. For data scientists, the gap typically comes down to knowing what graph techniques exist and how to apply them to specific situations. By spending time learning about and applying a dozen or two dozen specific graph algorithms, data scientists can often very quickly become adept at using these tools to solve problems within their own organizations. It’s also important for these experts to know how to convert raw data into the proper format for graph algorithms, as well as to understand what sorts of software tools can help them to make use of graphs within their organizations.

Once data scientists understand some of the basics of graph algorithms, they are often able to unlock significant value. For instance, a data scientist working for a social media company might use graph algorithms to discover better ways to make recommendations to users, improve the quality of the service, and generate more revenue through ads. Meanwhile, a data scientist working in finance might use graphs to model the relationships between different asset types, or between various financial institutions. In sectors like manufacturing, data scientists can use graphs to optimize supply chains and shipping routes.

Along with knowing the basics about different graph algorithms, data scientists should understand how to use high-performance computing methods to process graph data, as well as visualization techniques to help stakeholders understand the information. These skills will become increasingly important as the volume of data collected and managed by organizations continues to expand.

Software Engineers — Different organizations will use graph algorithms in different ways, and it can be extremely helpful to have engineering talent in-house that understands how to make graph analysis faster and more efficient. Here again, high-performance computing techniques are crucial to putting graph algorithms into practice. When software engineers obtain skills around performance-oriented topics such as parallel computing, data compression, locality optimization, and algorithmic optimization, they can add significant value to their organizations’ graph analytics processes.

Looking again at the examples of social media and route optimization, we can see the impact of high-performance computing on the use of graph algorithms. Data scientists at large social networking companies like Facebook and Twitter are dealing with billions — or even trillions — of vertices and edges, and it’s simply not practical to process this much data in a timely manner without leveraging high-performance computing techniques. In travel and logistics, data can change rapidly with the ebb and flow of traffic, and it is important that analysis tools be able to respond efficiently to these changes in real time — rather than restarting a computation from scratch every time new data is added to a model.

Project Managers — The people overseeing projects related to graph algorithms may not need the same sort of in-depth knowledge that data scientists need, but they can still benefit greatly from a baseline understanding of what graph algorithms are, how they work, and what sorts of problems they can be used to solve. Without this knowledge, a project manager will not be able to make informed suggestions about different ideas for data science teams to try out, or about how to improve the performance of graph-related solutions. For project managers, a broad conceptual understanding of graph algorithms is more important than hands-on experience. Really, it’s similar to anything else: while a CEO doesn’t need to be an expert in IT or finance or human resources or marketing, it’s important that they have at least some comfort and familiarity with each of these areas. Likewise, it’s important for project managers overseeing solutions that utilize graph algorithms to understand the basics.

Organizations are now collecting and managing more data than ever before, and that volume is only going to keep increasing each year. Graph algorithms can help organizations position themselves to create value from that data, better serve their customers, and maintain a competitive edge.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.