Machine Learning Challenges and Trends for the Enterprise, 2020 and Beyond
As we look to 2020 and what it’s set to bring for machine learning (ML) in the enterprise, growth is a key observation. As ML applications steadily become more mainstream in small, medium and large companies, use cases continue to evolve and deployments increase.
A growing number of companies are in early-stage ML development, but with this growth comes challenges, in particular when it comes to extracting value from ML investments. There are a number of reasons for this, in particular deployment, scaling, and versioning.
1. Deployment
When it comes to deployment, recent research carried out by Algorithmia found that just over 50% of companies that have developed an ML model are spending between eight and 90 days deploying ML models. An additional 13% said they were spending between 90 days and a year deploying models, and 5% said they were taking more than a year.
Such long lead times are obviously less than desirable and lead to questions as to why it’s taking some firms such a long time to deploy their models. There isn’t necessarily a definitive answer, although the nature of research and developing ML applications may help to explain the misalignment between expectations and execution. Further, Gartner notes in its Top 10 Data and Analytics Trends blog that ML teams are using open-source tooling rather than dedicated commercial ML tooling applications for their ML development. The good news is the analyst firm expects that increased use of commercial AI and ML will tackle this by helping “to accelerate the deployment of models in production, which will drive business value from these investments.”
2. Scaling
According to the Algorithmia study, scaling was cited as the biggest challenge for firms when it comes to extracting value from ML, with 43% of respondents saying this. In companies of more than 10,000 employees, this number rises to 58%. The key difficulties are largely hardware, modularity, and data sourcing, and can be attributed to a number of factors, such as siloed data science teams. As 2020 progresses, the rise of centralized innovation hubs from firms like Ericsson, IBM and Pfizer, as well as emerging tech centers within larger firms, should help to start mitigating these challenges. Via these efforts, ML can evolve into a centralized effort, eliminating the risks of fragmented processes.
3. Versioning
When it comes to versioning, 41% of businesses said this was their greatest challenge. While robust version-control is critical to ML in terms of pipelining, retraining, and evaluating models for accuracy, speed, and drift, it can impact a model’s ability to mature and is different than traditional software versioning because of the multiple different files to keep track of, all of which may be written in different languages, while relying on multiple frameworks. This can impact the sophistication of ML models, which in turn hinders value extraction.
Growth
While firms will be working to manage these challenges, ML will also enjoy growth in 2020. In particular, data scientists are set to see their numbers grow as industries embrace ML across their business. This is particularly true of mid-size firms seeking to utilize ML in order to put themselves ahead of their peers. Indeed, according to Adobe’s 2019 CIO Perspectives Survey, “Nearly 80% of chief information officers at U.S. companies plan to increase the use of artificial intelligence and machine learning over the next 12 months.”
Algorithmia’s research finds that just over 50% of firms employ between one and 10 data scientists. While this is lower than Algorithmia’s 2018 research, which found that 58% of organizations said they employed between one and 10 data scientists, this year’s research also finds that 39% of firms employ 11 or more data scientists, which tells us that firms are building out their data science teams significantly.
However, organizations face a potential hurdle when it comes to growing their data science teams. As is always the case in the tech sector, the right talent can be hard to come by, making competition stiff. And in the world of ML this is only exacerbated by tech giants with large and ever-growing data science teams that enjoy better salaries and benefits than small to mid-sized firms can offer.
While there isn’t an obvious solution to this, we may start to see a migration of traditional software staff into ML roles to help fill the gap, potentially leading to an evolution in job titles to things like ML developer or AI Ops. We will also see firms adopting platforms to help bridge this gap and enable data scientists and ML teams to be more efficient.
2020 and Beyond
As 2020 progresses and ML efforts continue, firms will, of course, seek to overcome the challenges their ML efforts present them with, as well as continue to work out the best ways in which to extract value from their ML models. There will likely be increased activity in the ML-enabler space, with firms that provide services to address challenges becoming more prevalent.
Elsewhere, ML teams will grow and evolve. To manage the ever-growing demand for data scientists, along with migrating staff from more traditional development roles, firms may look to hire more junior members of staff.
Ultimately, growth will occur, in terms of both use cases and personnel, and this will bring with it an increased desire — and necessity — to overcome the key challenges extracting value from ML currently presents.
Feature image by Gerd Altmann from Pixabay.