As a consequence of the global pandemic, there’s arguably never been a greater focus on the value of high quality, real-time data. As governments plot their roadmaps to recovery, it’s data that is informing the big decisions from vaccination strategies to the re-opening of shops, services and public transportation.

At KX, we’ve long understood the key role data plays in making critical business decisions. Magic really does happen when you combine historic and real-time data to deliver actionable insights for business decision making when they count the most.

Research commissioned at the start of the year validates this view, with 90% of firms reporting that they plan to increase investment in real-time analytics solutions over the next three to five years.  Moreover, nearly two-thirds (64%) of organisations believe that having access to real-time data is critical to making smarter business decisions, while over three-quarters (78%) say real-time data and insights are creating a competitive advantage for their business.

But the study also shows that businesses could be missing out on extracting the full value from their data by not thinking fast enough when it comes to real-time decision making. Research revealed that 69% of businesses consider real-time to mean over a second, with 45% of that group defining real-time as anything upwards from an hour.

Regardless of industry – finance, manufacturing, utilities, telecommunications and others – taking decision-making from minutes to microseconds can be a game changer for them. 

Based on our experience working with these firms, we believe there are five steps for taking real-time analytics to a sub-second level and enabling an operating model of continuous intelligence.

1. Assess and understand

While not all firms need to be operating at the sub-second level, all firms create data in real time, and an understanding of how and where faster analysis could lead to better operational and commercial performance is always useful. Businesses should ask themselves the following questions.

Is there a clear understanding of the value of the data that flows within a business and critically, the rate at which that value diminishes once created?

• Is there data-led culture and are the right tools and processes in place to ensure the people and applications that can extract full value from that data in place?
• What are the expectations and goals set against investments in real-time data analytics technologies and how will success be measured?

There is broad agreement on the value that an investment in these transformative technologies can bring to a business. Equally, access to the right technologies and having the right people with the right skills are common challenges. Having a clear understanding upfront of the data landscape, culture and goals is critical.

2. Get your data in shape

Put simply, the better a firm’s understanding of where data resides, its format and its history, the better placed they’ll be to shorten the decision-making window. Below are the most common types of datasets that businesses often seek to bring together. Each is a discrete, distinct set and the relationship between them can be complex.

• Datasets generated internally
• Datasets sourced externally
• Streaming data
• Data at rest
• Structured data
• Unstructured data

It is important to recognise that some data and data sources are more valuable than others. Time series data, for example, is one of the most valuable sources, particularly when generated in the IoT market. Highly-structured, machine-generated, and sent with timestamps between many thousands of devices at very high frequencies, it is relatively new to most organisations but prized. And you need a complete strategic solution for utilising it well.

3. Think faster

Once a robust, core real-time analytics system is in place, challenge the teams to think faster by testing and learning.

The best real-time analytics platforms enable sandbox environments where data scientists can build models and test outcomes rapidly without the worry of affecting critical systems running in parallel. This is the bedrock for unearthing news insights that allow for the iterative development of new capabilities, constantly adding value over time.

4. Anticipate challenges

The primary data challenge - and opportunity - that many organisations face is no longer that of volume, but of speed. Streaming analytics solutions need to work with existing and new datasets and therefore, must interact with many existing technologies. Subsequently, interoperability can be a challenge.

Additionally, IT teams can be small, stretched, or simply battling for the right talent in an increasingly competitive environment. Implementing a new technology can be a daunting task. But it also presents an opportunity to upskill the workforce in areas that will be vital to a business future success.

5. Find the right partner

Once the decision is made to increase investment in real-time analytics, the next step is to find a suitable partner. There are many questions to consider when doing this, including:

• Ask about a typical engagement. Streaming analytics goes beyond capturing data to report in quarterly meetings. Always look for a provider with clear and demonstrable experience in helping businesses to make those sub-second decisions.
• Ask about iteration and future flexibility. Since advanced, streaming analytics is a developing area, your provider should be able to clearly demonstrate how they plan to keep adding value over time. That will, of course, mean iterating their solution and offering regular updates and upgrades to match market demands.
• Look for a partner not a provider. Businesses benefit from technology when vendors think beyond the initial sale. And likewise, organisations should be looking for a collaborative, strategic partner over the years, rather than a one-time buy.

Analysing data is a process we cannot ignore, especially now as it informs important nation-wide decision making. The potential of streaming analytics is huge. But only if you can think quickly. A microsecond mindset is a must to equip your business with the tools needed for faster, smarter and more accurate decision making. Ultimately, leading to a competitive edge the company never had before.