The race to deploy AI in financial services has never been more intense. Over the past two to three years, banks of all sizes have moved with urgency to extract competitive value from their data, and the results have been uneven, revealing just how complex the challenge really is.
The early wins came quickly. AI-powered tools for unstructured data, including document summarization, meeting transcription, and conversational copilots for investment banking analysts, proved relatively straightforward to deploy and delivered near-immediate value. But as financial institutions pushed deeper, a harder problem emerged around structured data. The vast repositories of structured transactional data sit in siloed systems across the enterprise. Every credit card swipe, every loan application, every customer interaction generates rows of high-value structured data, and most banks are still struggling to make that data accessible to AI at scale.
The problem comes when you try to merge the value of structured and unstructured data productively. In a sense, this signals a new phase for the industry. Leading banks are moving beyond point solutions and building toward agentic AI architectures. These intelligent systems are capable of routing queries across both structured and unstructured data sources, seamlessly and in real time. Getting there requires more than cutting-edge models. It demands a modern data foundation built to access data wherever it lives.
This article traces that journey, from the low-hanging fruit of unstructured AI applications, through the development efforts surrounding structured data modernization, to the emerging frontier of unified agentic financial intelligence, and what it takes to advance along that maturity curve.
The rush to deploy conversational AI
Financial institutions have moved quickly to implement AI-powered tools focused on unstructured data processing. Large banks have rushed to deploy various flavors of copilot technologies, including note-taking systems and document summarization tools. These applications have proven particularly valuable for investment banking analysts who need immediate access to market research summaries and other critical information.
The appeal of these conversational AI tools lies in their ability to provide instant feedback and insights. When an analyst needs a quick summary of market research reports, they can access that information through natural language queries rather than manually searching through documents. This immediate accessibility has made unstructured data AI applications the low-hanging fruit in banking’s AI adoption journey.
The structured data challenge
While banks have found success with unstructured data applications, they’ve struggled significantly when attempting to leverage their institutional knowledge stored in transactional data. Every credit card swipe generates a row of data, creating enormous volumes of structured information that exists across multiple data silos. This transactional data represents a goldmine of customer insights and behavioral patterns, but accessing and analyzing it remains an ongoing challenge for most financial institutions.
The volume and complexity of this structured data present unique obstacles. Unlike unstructured documents that can be processed with existing AI tools, transactional data requires sophisticated data product solutions and robust analytics infrastructure to deliver meaningful insights.
The evolution toward agentic AI
Banks are now evolving their AI approach to focus more on agentic layers that can intelligently route queries to appropriate data sources. The goal is to create services that enable analysts to either run agents independently or, in the near term, simply ask questions that get automatically directed to the right place.
This agentic approach requires the flexibility to bring together both structured and unstructured data in a meaningful way.
For example, an analyst might want to perform a three-year lookback analysis on retail loan portfolios to understand how customer behavior has changed. Such queries require seamless integration between different data types and sources.
Building a data foundation predicated on context layers
Most banks are addressing this challenge by focusing first on the structured data side, building robust context layers that can support more sophisticated AI applications. This foundational work involves creating a financial services data analytics infrastructure that can handle the scale and complexity of banking data.
The context layer serves as the bridge between raw data and AI applications, providing the necessary governance and structure to make data accessible to AI systems. This approach aligns with broader trends in building data applications that require reliable, well-governed data foundations.
Moving from unstructured to unified data
Banks are at different stages of AI maturity, but the general progression follows a predictable pattern. Organizations typically start with unstructured data applications, then move to structured data initiatives, and finally work toward bringing both together in unified systems.
This progression reflects the technical and organizational challenges involved in each phase. Unstructured data applications can often be implemented with existing tools and minimal infrastructure changes. Structured data initiatives require more significant investments in data migration solutions and analytics platforms. The final phase of unification demands sophisticated orchestration capabilities and comprehensive data governance.
Infrastructure requirements for AI success
What does this look like in practice? First and foremost, successfully implementing AI for both structured and unstructured data requires a modern data architecture that can handle diverse workloads. Banks need platforms that support open data lakehouse architectures, enabling them to store and process different data types without being locked into proprietary systems.
It doesn’t stop there.
To function, data infrastructure must also support ELT data processing workflows that can efficiently transform and prepare data for AI applications. This is particularly important when dealing with the massive volumes of transactional data that banks generate daily. All of this infrastructure works together to provide the context and semantic layer needed for a strong data foundation.
Overcoming the challenge of data silos
What are the typical challenges in this AI rollout? One of the biggest obstacles banks face is the existence of data across multiple data silos. Credit card transactions, loan applications, customer service interactions, and market data often reside in separate systems with different formats and access controls. Breaking down these data silos is essential for creating comprehensive AI applications that can provide holistic customer insights.
Modern analytics platforms help address this challenge by providing federated data access capabilities that can access data across different systems while limiting how much data needs to be copied between different infrastructure environments. This approach allows banks to maintain their existing systems while still enabling cross-system analytics and AI applications.
The path towards structured and unstructured data for AI
As banks continue to mature their AI capabilities and as more AI initiatives reach production, the focus is shifting toward creating more sophisticated systems that can seamlessly blend structured and unstructured data analysis. The organizations that succeed will be those that invest in building robust data foundations while maintaining the flexibility to adapt to new AI technologies and use cases.
Agentic AI proves the need for a strong data foundation
The current trend toward agentic AI represents the next evolution in this journey, where intelligent systems can automatically determine the best data sources and analytical approaches for specific queries. This capability will be crucial for banks looking to scale their AI initiatives beyond simple document processing to comprehensive business intelligence.
Banks that have started with unstructured data applications are now working to extend those capabilities to their vast repositories of structured transactional data. Success in this endeavor requires not just advanced AI technologies, but also the underlying data infrastructure and governance frameworks to support enterprise-scale AI and analytics solutions.
Transformation relies on data infrastructure and federated access
The financial services industry’s AI transformation is still in its early stages, but the pattern is clear. Organizations that can successfully integrate both structured and unstructured data will have significant competitive advantages in understanding customer behavior, managing risk, and delivering personalized services.
For more information, check out the video below.
