HTAP: Hybrid Transactional / Analytical Processing – a term in dictionary of Big Data
I recently came across a Gartner introduced term HTAP to only realise that this is what we have been doing all this while at BlobCity. This article is BlobCity’s understanding of HTAP and how we think it will affect data vendor selection for most CIO’s across the globe.
As CIO’s / business executives / data engineers we all have our experience with data. In a broad sense our data stores do transaction processing for operational data, and then we perform analytics on this data to increase our understanding of the business, identify trends and use the analysis for business gain.
We have always considered Online Transaction Processing (OLTP) to be different than Online Analytic Processing (OLAP) requirements. The operational database is different from the analytics data warehouse, where a data integration system is put in place that syncs the transactional data with the analytics data store.
Having the analytics data separate from the transactional system very quickly puts the analytics out of sync with the what is actually happening. Maintaining absolute consistency between separate data stores is practically impossible. But really how important is this consistency?
- Have you sent a bill payment reminder to your customer just few hours after the customer has paid the bill by means of online banking?
- Do you take a decision on your companies progress 24 hours late, as the reports you are looking at while sitting in a board meeting are yesterdays?
These are only a fraction of the scenarios that affect us on a daily basis, because the analytic systems had a delayed consistency with the live copy of the data.
Transaction processing and analytics have always been considered as two separate problems. This is not the case any more. To differentiate one from the competition, it is important for one to quickly make meaning from their data and thereby keep agile and fast changing business processes that can align the business quickly in the right direction by taking continuous, active and automated decisions based new and incoming data.
HTAP systems though very rarely adopted today, we can expect them to be the need of niches and small specialised businesses who are looking at faster data processing abilities as a strong differentiating factor against other large and established market players. These systems have the potential of opening several new doors for making use of data in a more active and involved manner than what has ever been done before.
This article is originally published in my LinkedIn Profile https://www.linkedin.com/pulse/htap-hybrid-transactional-analytical-processing-sanket-sarang