The benefits of big data analytics have been too good to resist for most Australian organizations, but now many are struggling with privacy implications and the ethical use of data, according to new research from technology analyst firm Telsyte.
Telsyte’s Big Data and Analytics Study 2018 found around 80% of organizations in Australia with more than 20 employees have a big data strategy, although many are in the embryonic stages with pilot programs and exploratory projects.
Despite this growing wave of investment in big data, only 61% of businesses were conscious of privacy and regulatory concerns around big data, even with Australia’s Notifiable Data Breaches (NDB) scheme coming into effect in February this year, and the European Union’s General Data Protection Regulation (GDPR) coming into force in May.
Furthermore, 38% are struggling with the ethical use of customer data and are unsure if they are being “careful not to be creepy”. Concerns include handling of sensitive data which customers might have shared without understanding what they had consented to, such as during the installation of mobile apps or signing up to free web-based services (e.g. social media sites and email services).
“There is a huge temptation to highly target and customize offers to individuals, but also predict behaviors which generate profits,” says Telsyte managing director Foad Fadaghi. “However, many have had to draw the line at how sensitive customer data, such as location movements, or passively scraped data is used to target customers.”
Despite self-imposed regulation, 65% of organizations claim they protect data like other assets (e.g. physical or IP assets) and around 53% are willing to share data if there is tangible benefit to their organization.
Overall, 84% of business and technology leaders saw an improvement to their organization by using big data analytics. Productivity and improved decision making were rated highly, as was the ability to improve customer intelligence, and control operating costs.
Top-down leadership matters
The CEO and CDO (chief data officer, sometime called chief data scientist) are becoming the main sponsors of big data strategies in Australian companies.
About 35% of organizations have the CEO leading big data strategies, as the decision-making role of the CEO is becoming less about ‘gut feel’, but rather about being data-driven and having an informed process that can be reviewed by the board.
Telsyte research shows that companies with CDOs or CEOs spearheading efforts are much more likely to have an organization-wide digital strategy, or the ability to break down silos of data that might sit across various business units.
This addresses the fastest growing challenge relating to the adoption and use of data – the lack of a standard, organization-wide approach. This is a challenge which is cited by 28% of companies –twice as many compared to 2016.
Other challenges cited by respondents include the cost of analytics software or services (31%) and lack of in-house skills (25%)
The main drivers for big data analytics adoption include digital transformation (58%), customer behavior measurement (44%) and e-commerce applications (36%).
Security and fraud detection is growing in popularity amongst organizations with a big data strategy and is now in use by over 40% of organizations.
MSPs are the big data winners
With a lack of skills and high cost of entry holding some organizations back, Telsyte research shows service providers are the big winners for delivering big data analytics capability.
Around a third of companies with a data strategy are using a big data managed service provider (or outsourcers) compared to around a quarter a year ago, which is ideal for companies struggling to acquire big data skills.
To plug the capability gaps, over one-third of businesses are investing in big data training for staff, which has seen a mini-boom during the last two years.