With Big Data, Asking Right Questions Is Key

With Big Data

William Terdoslavich has spent a career in and out of tech journalism, having written for InformationWeek, The Hubbs.com, Computer Reseller News, Computer Systems News and Mobile Computing and Communications. Technology will always change. Human nature remains the same, usually crazy.

In the realm of Big Data, many people are trained to use analytics to find the right answers. But do we have enough people who can ask the right questions?

Many articles have been written in the business and tech press about intuition, the gut feeling that allows a professional to cognitively leap to an insight. It is a skill needed to ask the right question. Yet no one knows how to create an “intuitive,” much less train one. And guidance on when to use intuition is limited to “use your judgment,” which is not exactly precise, or helpful.

So where does intuition come from?

You can cultivate intuition, perhaps by finding people who have it. Or you can ignore people entirely and focus on artificial intelligence and machine learning. Which one of these methods will serve you best?

Intuitive people do not just walk in off the street to join a team. Intuition must be kindled somewhere in the team to make analysis work.

“It’s really hard to quantify and give a recipe for creation,” said Ted Dunning, chief application architect at MapR, a California-based software firm that develops data platforms based on Apache Hadoop. “It is hard doing the unexpected by rote. By definition, it can’t happen that way.”

Instead, one can exercise the team to think intuitively. Such thought exercises encourage approximation as a first step in pursuing exactness.

One way is to ask team members questions that they must answer with a range, explained Ellen Freidman, a solutions consultant at MapR. For example, what is the weight of a Boeing 747? Or what is the diameter of the moon? If the factual answer falls within the range, then the approximate answer is correct. Narrowest range wins.

“Developing the ability to approximate is a huge skill that goes underdeveloped,” Freidman said. Approximation does not contradict precision, nor is it a wild guess. It does allow people to solve a problem within the boundaries of error, she noted: “People need to recognize the difference between what they know and what they don’t know.”

Another key step: formally setting some time aside to think, even if it just 30 minutes a week, Freidman continued. This allows you to get to the essential concept of the problem you are tying to solve. “It leaves the door open to making new connections.” she said. “No one takes time to do this.”

Also, “you need to make room for them (the team) to be wrong,” Dunning added. “If you have room to be wrong 90 percent of the time and catch errors quickly, then you have solid preparation for high creativity.”

In the end, the purpose is to arrive at questions quickly, test them, and if they fail, discard the query. And start over, quickly. Intuition can be a guide to a question, but it is analysis which tests the premise, pass or fail.

Gauging the limits of intuition may still come down to the team member’s personality or academic background, noted Scott Gnau, chief technology officer at Hortonworks.

“The world is full of really great data scientists,” Gnau said. But you can’t “create” a person with a solid work ethic or intuition: “Some people are more that way. Some people are less that way.

 

Share it:
Share it:

[Social9_Share class=”s9-widget-wrapper”]

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

You Might Be Interested In

5 Ways Artificial Intelligence Is Shaping the Future of Ecommerce

18 Nov, 2016

Few industries are as competitive as ecommerce. Not only are online retailers competing with other online stores and brick-and-mortar locations, …

Read more

How will machine learning change science?

3 Aug, 2021

Machine learning has burst onto the scene in the past two decades and will be a defining technology of the …

Read more

Digital transformation: 7 steps for communicating a strategy change

21 Jun, 2020

How do you rally your organization around changes after you’ve already sold them on your original digital transformation strategy? It starts …

Read more

Do You Want to Share Your Story?

Bring your insights on Data, Visualization, Innovation or Business Agility to our community. Let them learn from your experience.

Get the 3 STEPS

To Drive Analytics Adoption
And manage change

3-steps-to-drive-analytics-adoption

Get Access to Event Discounts

Switch your 7wData account from Subscriber to Event Discount Member by clicking the button below and get access to event discounts. Learn & Grow together with us in a more profitable way!

Get Access to Event Discounts

Create a 7wData account and get access to event discounts. Learn & Grow together with us in a more profitable way!

Don't miss Out!

Stay in touch and receive in depth articles, guides, news & commentary of all things data.