This is my first article in a series of articles aimed at helping financial services executives understand software concepts, design principles and how these contribute (and even shape) agile principles. As a result, I hope these articles illustrate just how effective agile can be, helping shape and cultivate different cultural behaviours across an institution. These articles are to be thought provoking, not just to executives responsible for IT, but for anyone who has a desire to help shape culture across the financial institution.

So, in this article I want to look at some software concepts of decoupling, distributed architecture, domains and how these in a way help form the basis of operating an effective agile structure…

I am sure many engineers and heads of engineering get very frustrated with superiors not understanding some basic principles of modern-day software architecture. If the executive can grasp and understand these, then by extension, understanding other engineering decisions becomes a little easier, it also empowers the executive to make better decisions about product roadmaps and in old school terms, understand resource allocation.

As a developer, right back in school (well in my day we learnt this in school) we started to understand concepts such as decoupling, cohesion and wider areas of object orientated design/development. Now I’m not going to run through these in this article, however, it is well worth reading up on these concepts if you don’t already have a pretty solid grasp of them. But in a nutshell, you want your software to be modularised, with each module being as self-contained as possible. When your modules need to communicate, less is better. At this macro level, it allows you to maintain and develop different areas of your system without breaking others, think of it like that….

What is interesting is by applying these basics to not just software, but to how we go about running engineering departments, we start to see strong linkages to agile methodologies and structure within an engineering department. As a result, as a CIO/COO or part of the executive, you will make better informed decisions regarding product evolution.

Decoupled

Think of decoupling as keeping everything you need to know within that module. Strong cohesion refers to the degree to which the elements inside a module belong together. The stronger your cohesion, the more decoupled your module can be, the more isolated it is, which has massive benefits. For example, an isolated module can be maintained, updated and deployed into production without impacting any other aspect of your software. Its therefore easier to maintain, update and get through strict risk based policies. You can think of it as also containing a “blast radius”, as in, if the module goes wrong, does it take your system down, or just that particular module.

Obviously modules need to communicate with each other, but less is better. The diagram below shows to basic examples of cohesion in action, version a) which is good as it has strong cohesion within apps 1 and 2 while the apps are very lightly coupled (connected). Version b) which is bad, as it shows apps being strongly coupled.

Now this concept of decoupling and cohesion can apply to your engineering teams, you want all the knowledge and skills they need right there in that singular team, as opposed to a team having to communicate and depend on other teams. Sure, there will be some dependencies and lines of communications, but, just like software, if you can keep them decoupled as much as possible, you don’t have bottlenecks or failures.

For me, this basic concept can be found in so many areas of agile and the engineering lifecycle.

Distributed

Taking the decoupling concept further, we therefore have a requirement for things like the same data to be distributed into software components. Now, most executive has an understanding of data storage. However, the typical view is that we have a singular database that stores all our data – and copying that data is bad, bad because which version of the data is the truth. We need to get away from this viewpoint, there is nothing wrong with multiple databases distributed across your system. Equally there is nothing wrong with data being replicated across those distributed databases.

Distributed systems and distributed data enable software to remain decoupled as much as possible from other components, that means we continue to see the benefits of strong cohesion and software remaining decoupled. If we look at the cohesion diagram above, the green node in model a) could be that of shared data and nothing more.

Again, if we take this concept back to our engineers, if teams have everything they need within them, including the “data” they need to build software, then they continue to remain self-sufficient and in control of their own timelines to deliver. “Data” in this case often is in the form of business knowledge in the head of a Subject Matter Expert (SME) or business analyst etc. Remember, your team members can be geographically distributed, there is nothing wrong with that and its easy to manage, providing you don’t have multiple teams dependent on each other, then such distribution becomes a communications challenge. (See Cohesion model B).

Domains

So, keeping in mind these two concepts of decoupling and distribution, we should apply these to business areas and function. I have always liked the term domains, as a business domain does not have to be linked on a 1 to 1 basis back to a department, rather it is a grouping that delivers specific functions, products, and features.

In recent years, domains have become even more important to me in terms of running an engineering effort. A domain after all allows me to focus engineering efforts on a set of real “product” deliverables and helps ensure the wider company understands the value of their knowledge, contribution and how to get that into the engineers themselves.

Domains are therefore a grouping of the necessary resources required to understand and build those grouping of products, ideally with those resources residing in individual teams that focus on individual product. Please keep in mind that your “end product” may actually be a culmination of many internal products brought together, as in a team may focus on product that is only ever consumed by other teams/areas of your end product that customers see. The key here is to keep remembering it is about product, never a project.

As a side note, let me explain why I hate projects. To be blunt, a project is something you do, complete, hand over and move away from. Software should never be a project unless you don’t want it to work in a few weeks’ time, or you don’t want to be able to re-use what you have built. Projects lead to bespoke software that never evolves, never moves forward and within just 2 years is always legacy. A product is something that is re-used not just for one customer but for many. A product is something you maintain, improve, evolve, and never becomes legacy. By simply thinking of product, you deliver better solutions in a timelier fashion.

Let’s provide an example…

Payment Systems

As a bank, a domain could be that of “Payment Systems”. That domain is ultimately responsible for the banks ability to make and receive payments in accordance with the various payment system rules. Now here in the UK we have various payment systems, so in our example let’s have a team for Faster Payments, you could have other teams focussing on other payment systems, such as CHAPS etc etc.

This domain is naturally self-isolated, it doesn’t need to talk to departments providing loans for example, but it does require data potentially on accounts and account balances. Typically, you would think this team has therefore a dependency on teams that handle accounts, but it should not. If we think of a distributed system, then we simply need to ensure our domain has its own database that has all the information it needs regarding accounts to be able to interact correctly with the payment system.

Our Faster Payments team then needs to be made up of engineers, a SME on the bank accounts and account balances (maybe someone from finance), a FPS specialist (someone who understands the system rules, operational processes etc), a Quality Assurance engineer (QA to ensure automated tests are written for our FPS connection product) and a DevOps engineer (someone who can make sure necessary infrastructure is created in code and someone who can build delivery pipelines to make sure our product can move through environments and into production). The team will have their own database of accounts so the product can confirm that a bank account is indeed one the bank holds for example. No dependency on the core banking system then.

The team can now build out the Faster Payments connection and product, ready to be consumed by the wider banks systems to enable customers to make and receive faster payments. The teams is able to build this pretty much in isolation, as part of a larger isolated payment systems domain.

Remember, the team is building a product, so they don’t walk away once they have a working solution and join another team working on something else. No, they will always be the ones that support that product, they will be the ones that evolve it, update it to meet new requirements and ultimately ensure it doesn’t become a legacy product.

Product roadmaps with domains

If you embrace these concepts and look to build products within domains, then you start to see specific roadmaps for specific products within specific domains. In our example, there will never be a case where the business cannot deliver a product enhancement to its Faster Payments capabilities because the high level roadmap priority has those resources working on something unrelated to FPS.

Building out “products” that span multiple domains therefore needs to create smaller products within those domains,  or add to the product roadmap of an existing product. At an executive level, you really need to understand this construct. As a CIO/COO/CTO the key to long term success is ensuring domains are aligned to deliver overarching products to the end customer, from the executive this requires leadership, direction, and discipline.

In the past I have used and implemented a “steering group” that brings together domain and product owners to ensure everyone is working towards the common goals. This really can work, as long as there is strong leadership ensuring that domain owners pull together to deliver the prioritised products, while not sacrificing the product model that you have built.

As a final point to this article, no matter how tempting it can be, you should never move away from “product”, as I said earlier, if you think in a “project” fashion you will break this model, you will pull resources from one team and add them to another, you will move teams across domains and you will do all this thinking you are moving faster towards the delivery of a specific product. But what you are doing is breaking our decoupling rule, the distributed nature of teams and independence of the domain model, you are now building a bespoke solution, something you cannot re-sell nor maintain, you are building legacy…