Connecting Cloud Analytics to On-Premises - Pyramid Analytics
The article discusses the challenges large enterprises face in fully migrating business analytics to the cloud due to complex, regulated, and secure IT environments, highlighting the drawbacks of data shipping such as data freshness and governance issues, and emphasizing the growing interest in hybrid cloud analytics solutions that avoid data duplication by securely connecting cloud analytics directly to on-premises or remote data sources, despite the infrastructural and logistical complexities involved.
As the cloud becomes a realistic hosting destination and technology solution for more and more organizations, it is becoming obvious that a black-and-white switch to the cloud is unlikely to happen in most real-world scenarios. This is especially true when an organization’s IT footprint is large, complex, regulated, and/or highly secured.
Data Shipping
Business analytics faces a similar prospect, with many large enterprise-sized BI deployments remaining on-premises, even with the advent of ‘cloud-first’ (or ‘cloud-only’) analytics offerings from companies like Microsoft, SAP, Looker, and Qlik, to name a few. In many cases, larger companies are exploring the cloud option for analytics. Still, they are daunted by the prospect of moving their entire data operation to the cloud—and the idea is often killed off soon.
In other cases, the cloud option is used for limited scenarios where the data can be comfortably hydrated from an on-premises location to the cloud. But copying the data to the cloud—‘data shipping’—comes with its own challenges. It can be counter-productive, offsetting cloud efficiencies while introducing a new chore: keeping the cloud data fresh, hydrated, and accurate. Any hope of achieving tight data governance is challenged by data shipping (duplicates?), while the timeliness of any analyses against the data suffers.
Private Networking
So, it stands to reason that organizations and CDOs are interested in the “hybrid” option—where they get to run cloud-based analytics without the ‘data shipping’ headache. To this, most companies have so far offered the dreaded private network solution (i.e., VPN) or some other method for securing a channel between the cloud and the on-premises data source. (‘On-premises’ should be expanded to ‘remote data’, covering any data source not local to the cloud implementation.)
Apart from the infrastructural mess required to get a single secure channel in place, it becomes a logistical nightmare to expand the footprint to multiple sources. And that does not even include the additional administrative effort required to monitor the framework or any related authentication mechanisms needed in the chain.
And we have yet to address performance! Running direct queries against a remote data source over VPN is horrifically slow because technologies like JDBC and ODBC were not designed for long-distance communications. And the bigger the load, the worse the outcome—which seems to contradict the high-scalability promise of the cloud.
No wonder no one is fully committed to the cloud.
Pulse to the Rescue
To solve the puzzle, Pyramid has released its Pulse Server application. Designed as a lightweight, on-premises server ‘spoke’ that works with a cloud-hosted Pyramid Application ‘hub’. It offers a variety of breakthroughs in the hybrid deployment of business analytics that make it a compelling choice for the job:
- 1.Secure connectivity made simple: Pulse does not need to operate with any special changes to the network and does not require any specialized VPN software. Instead, it simply requires the URL address of the main Pyramid ‘hub’ application, wherever it is installed. With this simple information, it can connect securely to the hub server. (Security is driven via SSL when deployed in the main application.)
- 2.8x-10x Performance: Pulse solves the issue of long-distance querying by providing a model for data requests and responses that’s better than JDBC or ODBC alone. Testing has shown the performance to be 8-10 times faster than JDBC via VPN, especially with larger data loads under more pressure (concurrency). If, for no other reason, the performance gap makes Pulse a requisite solution.
- 3.Centralized management: Pulse integrates tightly with Pyramid’s main application—providing a singular, tight governance and security model that is easily managed from inside Pyramid’s centralized administrative console.
It goes without saying that Pulse supports all the SQL-based direct querying data sources supported by the main Pyramid application out of the box—making it compatible with the top data stacks in the market today. And best of all, Pulse is free with Pyramid Enterprise.
Perhaps a hybrid cloud deployment for data analytics is a good idea, after all.
Other Resources
To get a better sense of how this all comes together, check out these links: