24 August 2016
Amir Halfon

Financial Technology Landscape

Amir Halfon - Product Strategy Advisory

9Posts 56,798Views 1Comments
Finextra community

Banking Architecture

A community for discussing the latest happenings in banking IT. Credit Crunch impacting Risk Systems overall, revamp of mortgage backed securities, payment transformations, include business, technology, data and systems architecture capturing IT trends, 'what to dos?' concerning design of systems.

ACID, BASE and NoSQL

09 May 2013  |  3692 views  |  0

My last post talked about Enterprise NoSQL and ACID vs. BASE in the context of handling data variety. In this one I'd like to delve deeper into transactional, Enterprise NoSQL. 

Let's start by focusing on the main question: How can one guarantee cross-record ACID transactions in a horizontally-scalable, schema-agnostic database?

The short answer is an architectural pattern called Multi Version Concurrency Control or MVCC.

The basic notion behind MVCC is that records are never modified, but instead a new version is created every time a record changes. The system eventually deletes these old versions after a configureable period of time, but within that time window it's simple to roll back a transaction. More over, it's also straight forward to roll back the entire database to an earlier point in time - A.K.A. point-in-time recovery - a key requirement of enterprise databases. 

Interestingly enough, the availability of Enterprise NoSQL - a schema-agnostic technology that satisfies these requirements - is now starting to blur the boundaries between the traditional Data Warehouse, Operational Data Store and DataMart, and converge them into a single store. The enabler for this is the notion of schema-on-read (vs. the traditional schema-on-write), which refers to the ability to enter data without requiring a pre-defined schema, while supporting multiple schemas when the data is read. This means that the categories mentioned above can be merged into a single platform that satisfies many data consumers without requiring intense modeling and transformation ahead of time.

In addition to schema-on-read, it is also the unification of data management and search that is key to handling data diversity. In fact it was the immense success of search engines that paved the way to this new data management paradigm. Search technologies have established the use of a rich set of indexes as a means for querying non-relational data. From there it was a small leap to apply this notion to a database, converging it with database indexing. But unlike traditional RDBMS, indexes in the NoSQL world do not have to be pre-defined, nor rebuilt as the data changes.

So we're witnessing some related convergence trends - the convergence of structured and unstructured data, that of database and search technologies, and of traditional data management tiers into a single platform.

My next post will tie these concepts back to the related industry use-cases that benefit from them.

 

 

TagsPost-trade & opsInnovation

Comments: (0)

Comment on this story (membership required)

Latest posts from Amir

The Case for Semantic Technology in Financial Services

14 April 2014  |  6324 views  |  0 comments | recomends Recommends 0 TagsTrade executionInnovationGroupData Management 101

NoSQL Use Cases

04 January 2014  |  3678 views  |  0 comments | recomends Recommends 0 TagsRisk & regulationInnovationGroupInnovation in Financial Services

ACID, BASE and NoSQL

09 May 2013  |  3692 views  |  0 comments | recomends Recommends 0 TagsPost-trade & opsInnovationGroupBanking Architecture

Enterprise Big Data: It's Not About Size

16 April 2013  |  3886 views  |  0 comments | recomends Recommends 0 TagsPost-trade & opsInnovationGroupInnovation in Financial Services

Big Data Use Cases

24 February 2012  |  14137 views  |  0 comments | recomends Recommends 0

Amir's profile

job title Principle
location New York
member since 2011
Summary profile See full profile »
Amir Halfon advises technology companies and investors on product strategy and positioning. He is a recognized expert on technology trends and their business implications.

Amir's expertise

What Amir reads
Amir writes about

Who's commenting on Amir's posts