What is Data Discovery? 5 Need-to-Know Facts - Pyramid Analytics

One of the most common questions received during the BI Office Suite implementation process is, “How do we decide who needs which tool?” Since BI Office offers tiered licensing, customers can choose who has access to different functionalities or applications within the Suite. This leads to discussions about which users are requesting which capabilities, who has time for training, and enterprise data security concerns. Ultimately, the question becomes: “Who needs data discovery?”

Before answering that, let's discuss what data discovery is.

What is data discovery?

Data discovery is the process of extracting or finding information specific to an organizational problem or decision from a larger data set (raw or modeled). The content authors are subject matter experts on the business problem but not necessarily on the data platform and tool set. These capabilities are often referred to as Self-Service Business Intelligence.

What is the benefit of data discovery?

Data discovery, through the requisite tools, allows non-technical users to ask and answer their own questions and respond to their own data needs as they arise. The responsibility for determining what is informational and relevant falls to the subject matter experts closest to the business problem. This proximity of content creation to business requirements allows the business to leverage its data in very responsive and effective ways.

Additionally, this allows IT and BI departments with limited resources to focus on providing data discovery as a service. Being freed from continual data and reporting requests allows these resources with technical expertise to focus on better data stores, more intelligent ETL processes, and better platform training and delivery.

Why shouldn’t someone have data discovery?

With these benefits, why shouldn’t everyone inside of an organization have full data discovery capabilities?

There are several shortcomings with universal data discovery:

1. 1.The productivity trap – While doing data discovery looks like work and may seem like a productive use of time to the user, it is easy for casual users to spend time analyzing data without purpose.
2. 2.Multiple versions of the truth – If an enterprise has many subject matter experts across the organization performing their own data discovery about similar but parallel problems (e.g., sales management by territory), it is unlikely that they will determine the right analysis methods and content. A traditional BI approach may have provided an easier to manage and more uniform solution.
3. 3.Implementation and training costs – The total cost of ownership when deploying analytic capabilities to every user across an enterprise has a linear relationship to the capabilities provided. Training and software deployment obviously increase with complexity and are further complicated when the software is deployed directly to individual desktops.

With these trade-offs, true data discovery tools should be limited to those information workers that need to make or support multiple decisions in a complicated or uncertain environment. Traditional BI functionality is a more appropriate solution for most information workers that have well-defined decision support needs. This is usually referred to as a monitoring solution.

What should an enterprise solution deploy instead?

Data discovery is a critical capability for an enterprise. However, if it is not managed properly it can become expensive and disruptive. Often data discovery tools get deployed piecemeal—to reduce costs or to address spot needs. While this can be productive in the short term, it leads to data silos, duplicate efforts, and poor information transparency.

Instead, a more appropriate way to deploy data discovery capabilities is within the framework of a governed data discovery solution. Governed data discovery tools (like BI Office) provide the ability to easily share content created by data discovery users throughout the enterprise—under role-based security and asset management. This allows users the freedom to create the content they need and managers the ability to standardize content to match processes and best practices.

Well-designed governed data discovery tools allow administrators to assign the appropriate client and content and data access for the user role. By delivering the appropriate client to the appropriate user, administrators are better able to match capabilities to requirements and better manage the total cost of ownership for the solution.

What’s the bottom line?

Data discovery is a powerful and necessary function that gives business users the ability to be innovative with data. However, this is not something that everyone should have. It can also be counterproductive without a management framework, or if extended to everyone and anyone. Governed Data Discovery is an approach that gives organizations the best of both worlds.