People, organizations and things will continue to generate ever-larger volumes of data in the months and years ahead, but expect to see some changes in how that information will be put to use, not just by humans but by artificial intelligence (AI) as well.
A recent survey by the analyst firm Gartner Inc., for instance, found that while big data technologies are seeing more overall investment, that spending will be done by fewer companies over the next couple years. And that trend is likely to lead to a growing divide between organizations that "know" and those that don't, according to IDC.
Big data users will also need to ask deeper questions about the kinds of information they are and aren't collecting and analyzing. Beyond that, they'll need to honestly consider their willingness to act on results that might conflict with old ways of doing business, according to Gartner.
Need for Unbiased Models, Openness to New Ideas
That last lesson is one that Gartner research director Robert Hetu pointed out shortly after the unexpected results of the U.S. presidential election in November.
"The failure to accurately predict the outcome of the elections has caused some backlash against big data and algorithms," Hetu wrote in a Nov. 10 blog post. "This is misguided. The real issue is failure to build unbiased models that will identify trends that do not fit neatly into our present understanding. This is one of the most urgent challenges for big data, advanced analytics and algorithms."
Identifying trends through big data analytics is one thing, but persuading people to accept the reality of those trends and act on them is something else entirely, Hetu noted. Before the election, for example, there were a few analysts who recognized the outcome the data was pointing to, but they were "repeatedly laughed at and disregarded," he said.
This is the foundation of the problem, according to Hetu. "Organizations must foster environments where new ideas are embraced and safely explored," he said. "This is how we will grow the convergence of things we know."
While people need to be open to new ideas, organizations will also have to develop better and smarter algorithms and AI systems to automate responses to insights revealed through big data analytics.
Â
