In early 2021, the Reddit message board WallStreetBets upended Wall Street. In the process, it highlighted the power of democratic trading applications like Robinhood, and its Reddit-inspired investors showed how they could go toe-to-toe with Wall Street power players for market influence. Now, more and more Americans turn to sites like Reddit for trading advice rather than listening to more conventional sources like TV pundits, financial advisors, or equity research analysts. 

WallStreetBets is just one example of how Wall Street elites have been caught off guard by unforeseen events, shifts in consumer behavior, and changing market signals. Financial service institutions (FSI) must increasingly look to new forms of data to effectively prepare for and respond to rapidly changing market dynamics. Data is the fuel that powers dynamism in today’s leading FSIs. While the arms race for data escalates throughout the industry (particularly on the buy-side), many incumbent financial institutions have fallen behind. Financial organizations that successfully expand their data horizon (by using more data sources like alternative data) and expand their analytics capabilities (AI, ML, etc) can build more enduring moats around their businesses. 

The current data landscape within the financial sector

The financial services sector is one of the most data-intensive sectors in the global economy. In 2020 alone, there were 44 zettabytes of data created in the industry. However, while the volume and speed of new data are continuously increasing, some estimates suggest that businesses use only 0.5% of available data. Taking a systematic approach to productionizing data within financial services and fintech products is extremely difficult since it involves a seamless integration of processes, technology, people, and models. In addition, large incumbents rely on legacy IT and infrastructure systems that are rarely optimized for artificial intelligence and machine learning models, and do not offer the agility and flexibility for data analytics. A recent Deloitte study found that although the financial services industry is the frontrunner in data modernization, only 32% of respondents have fully implemented a modernized data approach. In other words, most FSIs today don’t even possess an ability to take advantage of the internal data they have, let alone external sources such as alternative data. 

The benefits of using different — and unique — data sources 

While advanced data analytics can help financial services organizations deliver better customer experiences, analytics and advanced models are nothing without the data that informs them. Organizations need to find new data sources to help predict and respond to events to prepare for any potential business impacts. 

Broadly, there are two categories of data financial services institutions can leverage: traditional data and alternative data. Traditional data involves internal data sets such as transactions, logs, and CRM, that financial services institutions have used for decades. Typically they involve structured/historical data. Alternative data involves external data sets usually generated outside of the company’s walls, such as satellite data, social media data, and web data scraping. As opposed to ‘traditional’ datasets, these datasets are often unstructured/real-time datasets. 

According to Krishna Nathan, the CIO of S&P Global, when analytics are applied to alternative data, they yield additional insights that complement the information received from traditional sources. Alternative data is not usually collected by credit reporting agencies or provided by customers while seeking credit. It can come from various sources that cover financial acts, such as cash flow analysis from bank accounts and credit card usage, to non-financial acts including rent, payday loans, payment history of internet bills, social media activity, education, and employment background. 

Hedge funds and investment banks have actually been using alternative data for years, and this trend is now spreading across the industry to inform customer segmentation, lending decisions, and ESG investing. Traders can use social media to track the first mentions of a new technology or collect insights on a given company's performance, which can then be analyzed to predict short-term investment trends. For example, with ESG investing, Databricks has asset managers that deploy advanced natural language processing analytics to quantify the intent surrounding sustainability disclosures by companies. It’s one thing to capture that the CEO of an energy company wants to reduce carbon emissions by 30%, but to generate sustainable alpha, asset managers need to quantify the intent and the specificity of the CEO’s words by deploying AI techniques. 

In other words, the real value of alternative data arises when it is combined with ‘traditional’ data and then coupled with scalable analytics power. However, up until recently, combining and coupling has been a complex and time-consuming process. 

Factors to keep in mind when working with alternative data

The use of alternative data has become more common — giving companies new insights into markets and industries. However, with the unstructured nature of alternative data coming from different sources such as video content, images, and Excel lists, it can be challenging to receive, integrate, and analyze. 

To remedy this, data technologies such as Spark or Hadoop can be leveraged to sort through alternative data and make it more searchable and accessible. When collecting alternative data, data scientists should only utilize quality data that is accurate, complete, and timely. To maintain a competitive edge, companies should test and incorporate new alternative data sets regularly. Financial services organizations looking to leverage more alternative data need to remember that AI and analytics are the easiest parts of the equation. The difficult parts are the steps around data ingestion, data cleaning, data curation, and data harmonization, which need to happen before the data can be used to find meaningful insights. These foundational steps to turn raw and messy data into reliable and actionable data require collaboration across data engineers, data scientists, and domain experts, as well as the use of a collaborative and scalable computing platform. Unless these steps are taken carefully, you can fall into the trap of bad data, making bad decisions with falsely high confidence.

While many financial institutions and fintech companies have vast amounts of information at their disposal, there are many opportunities to expand and enhance their data strategies with new or previously untapped types of data. Today, there are dozens of data models to work with to help companies predict and prepare for upcoming events. By tying their business and data strategy together and understanding their data requirements, infrastructure capabilities, and tech needs, businesses can create better data strategies to develop faster insights and better prepare for the unexpected. With the right data tools, businesses today can empower themselves to be able to use the right data to make the right decisions with truly high confidence.