Alteryx, a company that improves businesses via data science and analytics, revealed that approximately 54 million data workers around the world face common challenges associated with data complexity, and the diversity and scale of their organizations’ data. In an increasingly data-driven world, the term “data worker” spans beyond the 54 million identified in this study, but the findings are indicative of the challenges specific to those engaging in significant data work in their day-to-day jobs. The Alteryx-commissioned IDC Infobrief, The State of Data Science and Analytics, uncovered inefficiencies, ineffectiveness and wasted time as many organizations turn to data as the lifeblood of their digital transformation.
Eighty percent of organizations now leverage data across multiple organizational processes, but despite increases in innovation, data workers still waste 44 percent of their time each week because they are unsuccessful in their activities. Data workers spend more than 40 percent of their time searching for and preparing data instead of gleaning insights and, on average, use four to seven different tools to perform data activities, adding to the complexity of the data and analytics process.
- On average, data workers leverage more than six data sources, 40 million rows of data and seven different outputs along their analytic journey.
- The top frustrations cited by data workers in the survey are indicative of root causes that are responsible for inefficiencies and ineffectiveness. For example, more than 30 percent of data workers say they spend too much time in data preparation, a task that can often be automated.
- Eighty-eight percent of data workers, approximately 47 million people worldwide, use spreadsheets in their data activities. Spreadsheet functions are often used as a proxy for data preparation, analytics and data application development tools but are error-prone and expose the organization to compliance and trust issues.
An Impetus for Change
The survey found that data workers are unsuccessful for a variety of reasons, including lack of collaboration, knowledge gaps and resistance to change. Participants reported the lack of creative and analytic thinking, analytic and statistical skills, and data preparation skills as the highest-ranked skills gaps responsible for productivity issues, indicative of the pervasive talent gap that exists between data scientists and data workers in the line of business.
To overcome these issues and more, many organizations are hiring Chief Data and/or Analytics Officers to streamline analytic processes, build a culture of analytics and encourage data literacy across the enterprise as part of their broader digital transformation strategy.