Traditionally, core banking systems have been developed as large monolithic systems. There are a few issues with this approach.
- To scale such systems, the only way is to scale vertically, i.e. use larger and larger hardware, whether it be mainframes or UNIX systems. The cost of such vertical scaling is exponential.
- The more and more changes that are applied to such systems, the more and more difficult, costly and risky it becomes to change these systems. At some stage it becomes impossible to change and a replacement is required.
- Such systems are also typically built by quite hierarchical organisations. According to Conway's Law, the systems that are built by an organisation mimic that organisation's communication structure. An organisation where communication is difficult will
build a system that is typically monolithic.
About 10-15 years ago there was a push towards SOA. Very good ideas but generally poor implementations and very much driven by vendors who were pushing their enterprise service bus (ESB) and middleware solutions. Rather than using the middleware as a way
to allow systems to communicate with each other, people implemented business logic in this middle layer because the underlying systems were too difficult to change. This just tied the systems together and the result was still a big monolith, difficult to change.
As the saying goes, you can put lipstick on a pig, but it's still a pig.
Over the last 3 to 5 years there have been a few important changes in the way software systems are built.
- The practice of Continuous Deployment is becoming more and more popular. In CD, one considers that any change, however small, should be a possible release candidate. So rather than waiting and accumulating many changes for months and then doing for example
quarterly releases, every change can be a release. To enable this, we need to automate the whole pipeline from code change, via automated testing, to release.
- Micro-services, an evolution of SOA, has seen the light and pioneered at institutions such as Twitter and Linkedin. The idea is to break down a monolithic system in small parts according to business functionality provided. A micro-service is a service
like in SOA but the important thing is the 'micro' part, it does something small but it does it completely and it does only that. Another important aspect is the communication between micro-services. It uses lightweight protocols such as REST and publish subscribe
messaging systems such as Apache Kafka. No more big ESB's. A micro-services approach aims to build a system that is loosely coupled and highly cohesive which is what software architecture is all about. Changing such a system is much easier since changes will
typically only need to be made to certain micro-services without upsetting the whole system.
- Thanks to the openness of micro services and API based services, building a system is now more about assembling services rather than building things from scratch.
- Today, organisations have increasingly flatter structures which improves communication, something which we then see reflected in the systems that are built. Combined with the micro-services and continuous deployment approach, you see smaller teams taking
end to end responsibility for small parts of the system. In some organisations, such as Amazon, teams are responsible not only for developing something but also for running it in production. Contrast this with organisations where you have a team of developers
and a support team.
- New technologies such as Docker make it very easy to implement and deploy micro services as stand-alone entities. Micro-services can be easily distributed over different systems using technologies such as Docker containers.
- Cloud providers have evolved a lot, see for example how easy it is start up a system on Amazon Web Services.
All of this evolution has resulted in the fact that - with the right tools - we can now easily build distributed open systems and run them on cheap hardware and scale them horizontally (add more cheap boxes). So we no longer need the mainframe and large UNIX
systems from before.
Of course there are some drawbacks to this as well, but let's leave that for another time.