The report, "Big Data Cases in Banking and Securities: A Report from the Front Lines," is available to the public at www.STACresearch.com/news/bdwp. The authors presented the key study findings at the STAC Summit in New York today.

Funded by Intel, the research surveyed 16 projects at 10 top-tier global investment and retail banks on the STAC Benchmark Council. The study found that "big data" technologies were deployed in business functions ranging from card transactions and customer prospecting to credit-risk management and trading. The business drivers for these projects reflected industry-wide imperatives: responding to legal or regulatory requirements, reducing costs, and more adroitly responding to changing market opportunities.

Despite media buzz about the so-called "unstructured" nature of big data, the study found that most of the content in these banking cases adhered to highly structured formats. Nevertheless, "banks are struggling with variety at the semantic level," according to Peter Lankford, STAC Director and co-author of the report. "Despite clear delineation of data fields, it is often difficult for analytic applications to know what those fields mean when dealing with dozens or hundreds of source systems." The use cases showed a wide range of analytic complexity, from simple searches to machine learning and graph mining.

While the scope of the study was not specific to any technology, it revealed a clear tendency among banks to use Hadoop in their solutions. Hadoop is an open-source software ecosystem capable of spreading data processing over multiple inexpensive machines. The research also uncovered a trend toward reliance on centralized, Hadoop-based analytic platforms-as-a-service to support multiple business applications. Jennifer Costley, co-author of the report explains "these centralized services go beyond traditional enterprise data warehouses by providing new analytics across a broader range of data while radically shifting the IT cost curve."

Despite the benefits banks associated with big data technologies, the report also details shortcomings that the banks said were holding back wider adoption. These include insufficient controls over which users have access to which data, as well as difficulty managing multiple applications competing for computational resources. While most of the banks viewed the relative immaturity of big data technologies and vendors as an acceptable risk, some pointed to the high rate of change in open source products as a poor fit for highly regulated cases that prioritize predictability.

The research was part of an initiative underway at the STAC Benchmark Council to develop technology benchmark standards based on big data workloads. Details are at www.STACresearch.com/bigsig.