Merico Devises Analytics to Assess Developer Code Contributions
How do you determine whether or not a developer on your team is up to par? Do you count the number of lines of code they contribute? Similarly, when you’re hiring developers, how do you determine whether or not they were contributing and productive members of their former teams?
These can all be difficult questions, and developers often face controversial coding interviews among other methods to determine their relative value. Merico, a “code repository analytics company” founded by a team from Microsoft Research, UC Berkeley, and Stanford, recently launched to solve precisely these issues, using a mixture of “advanced program analysis and machine learning to assess code repositories and deliver fair and reliable insight about individual developer contributions to a project,” according to the company website.
A primary goal of the company is to offer code assessment “without obfuscation by politics, personal bias, or other team dynamics,” according to a company statement, and it does so through analysis of code repositories.
“Code repositories are the most valuable asset of software companies, along with the people who contribute to them,” said Jinglei Ren, CEO of Merico, in a statement. “Merico Analytics is the first developer performance and talent management solution for enterprises that is data-driven and based on in-depth code analysis.”
Merico uses research-backed metrics and analyzes code repositories to determine the value of individual contributions, whether seemingly simple bug fixes or other features, and uses machine learning to determine the relative value of those contributions. Currently, the company focuses on a few primary areas for using this information — developer recruitment, HR and team management, job seekers, and open source project contributors — more on that last one in a moment.
In an interview with The New Stack, Hezheng Yin, chief technology officer at Merico, explained that companies often use misleading or gameable metrics to evaluate coder contributions.
“It’s very surprising what metrics they are using right now for evaluating developers contributions. Facebook, at least several years ago, was using lines of code. Studies have shown that the lines of code that team ships correlate positively with a long-term growth of the team, so [companies] use that metric to evaluate how much of a bonus they would give to a team leader,” said Yin. “That’s really surprising. It encourages people to write bloated code and it’s a pretty easy-to-game metric. But that’s what people are using – even some of the best companies in Silicon Valley. I believe we can do much better than that.”
Yin went on to explain that, beyond misguided metrics, companies often use other methods that account more for developer personality or internal politics than actual ability and impact.
“Right now, a lot of the evaluations are based on peer reviews. This heavily relies on your personality, whether you are introverted, extroverted, if you can keep a good relationship with your managers,” said Yin. “But what if your work could speak for itself? This is the philosophy that we had when we developed this algorithm. You have already done the work. Why don’t we just directly analyze from your code instead of purely relying on subjective peer reviews?”
Returning to the idea of open source contributors, Yin offered that Merico may help democratize the distribution of funding by determining a way to mete out financial contributions to more than a core team of maintainers.
“Raising funding for open source project is hard and we don’t claim that we can solve this problem. But no matter how you raise the money (either from big corporations or from individuals), you’ll face the second problem: how do you fairly distribute the rewards,” said Yin. “And that’s the part where we come in. I think the overall trend is that people’s awareness of open source sustainability is increasing and the funding status is getting better. This is partially supported by the recent emergence of various funding solutions for open source, including OpenCollective, Linux Foundation’s Community Bridge, Tidelift, Gitcoin, and etc.”
Feature image via Pixabay.
