TNS
VOXPOP
What news from AWS re:Invent last week will have the most impact on you?
Amazon Q, an AI chatbot for explaining how AWS works.
0%
Super-fast S3 Express storage.
0%
New Graviton 4 processor instances.
0%
Emily Freeman leaving AWS.
0%
I don't use AWS, so none of this will affect me.
0%
Cloud Services

Software Engineers Use Spreadsheets; Data Engineers Use the Cloud

Data engineers use cloud-based software and APIs as their primary tool to analyze data and are more likely to analyze data in the cloud.
Jan 14th, 2021 1:00pm by
Featued image for: Software Engineers Use Spreadsheets; Data Engineers Use the Cloud

Building or running data infrastructure is an important part of 55% of 372 data engineers’ jobs, according to the “2020 Kaggle Machine Learning & Data Science Survey.” These data engineers are supporting data science applications as well as other use cases. Data engineers are actually a bit more likely (58%) to be analyzing and understanding data in order to influence decisions as part of their job.

Data scientists focus on analysis, which is not as important for machine learning (ML) engineers. Still, there are many similarities between the 2,421 data scientists and 937 machine learning (ML) engineers in the Kaggle survey, with about the same percentage improving ML models, as well as building/running a ML service to improve a product or service.

At 18%, data engineers are more than twice as likely as data scientists to use cloud-based software and APIs as their primary tool to analyze data. They also exhibited a greater likelihood to analyze data in the cloud. Local development environments like Jupyter Notebooks are most likely to be used by all the job roles we reviewed. Basic statistical software, which is defined as spreadsheets, is very popular among software engineers. This is a reminder that just because they know Python doesn’t mean developers will use data tools for data science.

Feature image via Pixabay.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.