Hybrid Data Connectivity via Pulse - Pyramid Analytics

With increased cloud choices and distributed servers, many organizations now have a BI server in the cloud, remote data sources, and end-users in different locations, potentially on different continents. This creates a dilemma: either move or copy the data for analysis to the cloud or deploy the BI solution on-premises.

Both options have shortcomings. Moving data to the cloud duplicates data and may sacrifice on-premises investments. Staying on-premises can limit scalability and flexibility offered by the cloud. Therefore, a “hybrid” strategy is needed—BI deployed in the cloud while data and infrastructure remain on-premises.

The challenge is to connect to remote data simply, without compromising speed and security or creating administrative complexity when managing various data sources in different hosting frameworks, especially with multi-tenant access issues.

To address this, Pyramid’s Pulse presents remote data sources to users as if they were local, offering speed and security. Configuring Pulse and connecting data are simple, and all data sources (remote or otherwise) can be centrally managed and governed from the main administrative tools, with full multi-tenant support.

The problem

Companies may implement BI in the cloud for easier maintenance, reduced capital expenses, and flexibility. However, if they don’t move the underlying data, they typically experience speed and security issues when running against remote data sources.

Connecting directly to remote data via VPN reroutes traffic across the network, usually making access slow. Transferring data from the data center to the cloud is often not viable due to overheads for data transfer and storage, and it duplicates the data, which undermines the central source of truth.

Some third-party BI vendors offer remote data access solutions, but with limitations. For example, Microsoft’s “Gateway” doesn’t connect to all data sources supported in Power BI, and SAP’s “Agent” can only connect to a limited set of data sources and is complicated to set up. Other tools may run queries inefficiently over the network, resulting in larger payloads and slower response times. These hurdles often lead users to export raw data from remote servers and reimport it into cloud-hosted data stacks, which breaches data governance and security protocols, duplicates analytic layers, decentralizes the “truth,” and reduces the value of remote data source investments.

Pyramid’s solution

Pulse is a stand-alone application designed to be deployed into disparate networks, separate from the main Pyramid deployment. It allows connectivity to remote data sources without using a VPN or other network security protocols, while still providing full data security. From within the main Pyramid application, remote data sources are accessible like local data sources, without the complexities of setting up reverse agents.

Pulse is quick and easy to set up (2-3 minutes), and its multi-tenancy capabilities allow different tenants to have their own deployments in a shared Pyramid instance. Multiple instances of Pulse in an environment offer a high-throughput solution. Pulse supports both direct querying of a data source and data streaming into data mash-ups and modeling applications—without loss of function, feature, or performance.

Business case

Michelle is the BI administrator for CPO Ventures, a multinational retailer with worldwide branches. Their SAP HANA data is stored on a server in Europe, and they run a Pyramid cloud installation on AWS on the East Coast of the U.S. Bruce, a BI analyst in Australia, builds dashboards and performs analysis via a web browser. Michelle set up access to the remote HANA instance using Pulse to provide all worldwide users with easy access and speed.

A typical user session involves:

1. 1.An end-user client interaction is sent to Pyramid hosted in the cloud.
2. 2.Pyramid launches a request to the Pulse server to locally run a direct query against the local data server.
3. 3.The Pulse server sends a response back to the cloud.
4. 4.The Pulse server transmits the result back to the end user.

To set up Pulse, Michelle ran the Pulse installation on a remote server in the same network as the HANA data (two minutes) and then supplied the Pyramid URL address and Pulse security key. She then added a new data source for the HANA server in the admin console as if it were local to Pyramid in the AWS cloud (two more minutes).

Bruce can now create a new data discovery by selecting the HANA server like any other data source hosted “locally.” The Pulse logo indicates the model is using Pulse to access the database and models—the only indication to Bruce that this is not a “local” data source.

Bruce then uses the Discover app to create a scatter plot to plot sales versus expenses and is able to query and retrieve 6684 records in 850ms—less than a one second round trip!

Summary

Multiple servers in different locations present the challenge of connecting speedily and securely without using VPN and network security. Remote data centers using networks provide very slow access as traffic is rerouted across the network. Third-party tools can access limited data sources with poor response times using their remote data implementations. As a result, most BI tools encourage users to export raw data out of remote data servers into their native data stacks, reducing the value of the remote data center investment.

In contrast, Pyramid’s Pulse application is quick and easy to set up and allows connectivity to remote data centers without using VPN or network security, providing access to remote data sources as if they were local. Both direct querying and data streaming are supported with no loss of security, function, feature, or performance—offering tremendous speed and security.