Scaling AI in financial services starts with governance and architecture

The real challenge in scaling AI in financial services lies in the data foundation underneath. Organizations struggle to unify fragmented data, enforce rigorous governance, and maintain observability across legacy systems that have operated for decades.

Thomas Mathew, senior director of financial services industry cloud at Microsoft, puts it plainly: "Most organizations are failing in the AI space not because they lack data, but because they can't adequately trust and interpret the data they already have."

Poor data quality doesn't just produce inaccurate outputs. It destroys user trust, invites regulatory scrutiny, and makes the entire AI investment difficult to justify. For CIOs, CTOs, and CDOs navigating a highly regulated environment, that's an unacceptable risk.

The early wave of AI adoption in financial services focused heavily on customer-facing applications: chatbots, personalization, and digital assistants. Organizations quickly discovered that deploying these tools without a solid backend architecture leads to hallucinations, compliance violations, and escalating costs.

The focus has since shifted. IDC data confirms it: Companies are now prioritizing infrastructure, data, and governance over front-office innovation. Customer experience, which ranked last in budget immunity just a year ago, has moved up but only after organizations recognized they needed to fix the back end first.

Jerry Silva, vice president of financial insights at IDC, frames this as a strategic imperative: "Treat AI as an enterprise capability, not as a technology. Make sure all of the governance is in place, and then look for those experts that can help you with the actual business value being driven out of AI."

This is the pivot that separates organizations making real progress from those still running isolated experiments.

Financial services companies have vast amounts of information, but it’s often siloed, unstructured, or outdated. To solve this, a unified platform must ingest and organize information from across the enterprise.

When organizations implement robust search capabilities like semantic search, their models can draw from accurate, relevant, and current sources. This transforms raw logs and documents into actionable intelligence.

You can't treat governance as an afterthought in regulated environments. Your architecture must automatically enforce rules for data sovereignty, access controls, and privacy. Effective AI governance needs role-based access controls and complete audit trails. This control protects the organization and allows for secure innovation.

Fragmentation is the enemy of scale. When different teams use separate tools, blind spots emerge, and investigating fraud or system failures becomes a slow, manual process. Unifying metrics, logs, and traces into a single platform helps teams develop health forecasting and anomaly detection models.

For example, one of the largest property and casualty insurers in the US partnered with Kyndryl, Elastic, and Microsoft to implement this unified approach. The results were impressive: They reduced incidents by about 5,000 annually and found that 90% of their past outages could've been prevented.

Financial services AI strategy is shifting from generative to agentic systems. These autonomous agents don't just answer questions; they observe, reason, and execute complex workflows like automating claims or investigating security threats.

However, autonomy introduces new risks. Agentic systems need real-time context, strict guardrails, and human-in-the-loop escalation for high-stakes decisions. Few agentic implementations in financial services operate without human oversight, especially where sensitive client data, potential for financial loss, large credit decisions, or regulatory explainability are involved.

Tim Brophy, principal solutions architect at Elastic, advises a pragmatic starting point: "Think small. Let's start with a small project and a small use case and iterate until it becomes big ... because the use case is only as strong as the context that is provided."

A highly observable AI architecture — one that tracks how agents make decisions and what data they access — is essential for safe deployment at scale.

Elastic's Search AI Lake brings together data from across the enterprise. It uses machine learning to speed up root cause analysis and find patterns that humans might miss. When all telemetry is in one place, AI can spot anomalies before they cause major outages or security incidents.

This unified approach also supports different use cases. As Brophy explains, once the data is consolidated for observability, the same foundation can support security analytics, fraud detection, and AI-assisted search. This eliminates the need for a complete architectural rebuild.

Scaling AI in financial services isn't just an IT initiative; it demands collaboration between business, data, security, and compliance teams. Isolated projects often fail because they don't consider the needs of the wider organization.

Niloy Sengupta, VP financial services modernization leader at Kyndryl, puts it this way: "If one side of the house tries to do something, the chances of adoption across the enterprise are lower than if everybody comes to do it."

Successful companies build environments where teams create solutions together. Using shared platforms, they break down silos and ensure AI projects align with business goals and regulatory requirements.

The complexity of modern financial architectures means that very few organizations can build everything internally. The pace of change, from new agent-to-agent protocols to evolving regulatory frameworks, requires specialized expertise that's difficult and unnecessarily costly to maintain in-house.

Ecosystem collaboration across software providers, cloud vendors, and system integrators is essential. Platforms like Elastic running on Microsoft Azure and managed by Kyndryl deliver prebuilt integrations, proven reference architectures, and enterprise-grade support. These partnerships reduce implementation risk and accelerate time to value.

Financial services companies that succeed in the next era of AI will be the ones that prioritize their data foundations now. By investing in unified search, comprehensive observability, and rigorous financial AI governance, they build the resilient architecture required for autonomous systems. This approach reduces risk, improves operational efficiency, and drives measurable business outcomes.

The journey from experimentation to execution demands a clear strategy and cross-functional alignment. It requires treating AI as a core enterprise capability, supported by the right technology and the right partners.

The organizations making real progress aren't waiting for a perfect plan. They're starting small, governing carefully, and building from a foundation they can trust.