With AI taking the centre stage, data streaming platform (DSP) is no longer just a good feature; it is also a core enabler for companies aiming to scale meaningfully.
Today, customers expect product recommendations, service interactions, and support responses to be tailored, impactful, and delivered in real time—and they hold AI to the same standard.
AI’s effectiveness is directly tied to the quality of data it relies on. This requires steady, trustworthy, real-time data flowing across the organisation. This is where data streaming becomes critical for powering AI capabilities and helping businesses to remain competitive in today’s market.
According to data streaming pioneer, Confluent’s 2025 Data Streaming Report, which surveyed 4,175 IT leaders across 12 countries, including 650 respondents from India, about 95% IT leaders see DSPs easing AI adoption by tackling data access, quality, and governance challenges head-on.
The report further highlights that 91% of organisations are increasing their use of DSPs to deliver real-time, contextual, and reliable data for AI systems.
For 94% of leaders, it’s now a top strategic priority. The investment is proving valuable, with 86% reporting an ROI of two to five times—an increase from 80% reported in 2024. Moreover, 81% of IT leaders have adopted a ‘shift-left’ approach to data processing and governance, resulting in reduced costs and risks across development and operations.
Talking to AIM, Rohit Vyas, country presales leader at Confluent, highlighted that when enterprises, SMEs, or government bodies look to adopt generative AI—especially at scale—it becomes clear that a robust data streaming platform is essential.
“Data has time value to it. The more time elapses after data gets created, the lower is the value of that data proportionately.” That is why it is critical to invest in infrastructure that can deliver data at the pace it is generated.
This points to a fascinating trend unfolding. Notably, almost every company today has a dedicated budget set aside for AI adoption.
Yet, despite this momentum, the vast majority still struggle to turn their AI vision into tangible results. Vyas pointed out that “AI evangelism just remains largely on the PowerPoint decks”, with actual implementation proving elusive.
Why is This Happening?
According to Vyas, while there are multiple reasons, a key challenge is that many organisations don’t know how to “actionise this on the ground” beyond the basics—setting up an LLM, installing some infrastructure racks and then throwing in some GPUs.
These are merely the bare bones. What’s often overlooked is the data serving layer, which, in many cases, is still constrained by legacy systems. “If the foundations are not built right, the building, of course, will not be beautiful.”
Furthermore, Vyas added that there is a clear contrast emerging among enterprise respondents, reflecting the real-world diversity in digital maturity. The company also acknowledged in the report that “a significant percentage of respondents” still operate on legacy data platforms, though the intent is not to eliminate these systems but to “modernise” them.
Vyas observed that “a very high mature adoption of data streaming platform capabilities” is seen particularly among digital-native businesses, especially leading e-retailers, food delivery platforms, and fashion retailers. These organisations represent one end of the maturity spectrum.
At the other end lie sectors like BFSI and public sector entities. While they are still early in their streaming adoption journey, he noted that there is “tremendous room for growth”.
In this regard, Vyas explained that Confluent offers the flexibility for enterprises to either modernise their existing legacy setup or completely leapfrog multiple stages of maturity and directly land from fully batched to fully modern streaming pipeline.
Why is DSP Important?
“If Confluent is down, Swiggy is down,” Vyas explained. That’s how critical Confluent’s real-time data streaming platform is to Swiggy’s operations, as he has previously mentioned to AIM. Handling billions of orders across over 700 cities, Swiggy relies on Confluent’s managed Apache Kafka service to streamline deliveries, optimise demand surges, and enhance customer experience.
Initially, Swiggy relied on Apache Kafka, an open-source platform, for its real-time data streaming needs. However, as order volumes surged, maintaining Kafka internally became resource-intensive. Swiggy’s engineering teams had to dedicate significant efforts to managing, scaling, and troubleshooting Kafka clusters, impacting overall efficiency.
Confluent’s Data Streaming and LLMs
Real-time decision logic, combined with processing and execution, forms the backbone of autonomous operations. The rise of autonomous systems such as drones exemplifies this architecture. Take, for instance, the decision: “Can you hit the drone inside your boundary or outside it” is one the system must make in milliseconds. However, the ability to do so depends on whether the underlying data infrastructure can deliver the necessary speed and reliability.
Confluent forms the core “data fabric”, enabling this real-time loop.
Vyas also highlighted the platform’s ability to support LLM portability. In this use case, a prompt is sent to one LLM, and its response is passed to a second LLM for scoring. Then, both outputs are sent to a third LLM for further evaluation. Vyas called it a “very intelligent way” of handling prompt engineering and model benchmarking.
Confluent powers this with its data pipeline infrastructure and processing engines like Flink, enabling advanced use cases such as Retrieval-Augmented Generation (RAG).
The platform allows saving of intermediate responses, state management, and enrichment through ingestion from external sources—all while maintaining a governed data layer.
According to Vyas, this means that “LLMs are confident that they get the data served over a governed layer and RAG infused together as well.”
For users, the burden is reduced to simply monitoring how their LLMs perform, because data governance and data injection processing layers are all sorted.
The implications go far beyond user-facing AI tools. The same architecture powers pattern identification, matching, and operational feedback loops. Vyas said that the “shift-left” approach brings ingestion, governance, and processing of data as close to the production time” as possible.
Confluent’s unified platform enables this loop by pulling data not only from operational sources but also from analytical platforms like Snowflake, traditionally used for reporting and insights.
What are Others Doing?
Meanwhile, Striim is an end-to-end streaming platform designed for enterprises that demand high data service-level agreements regarding uptime, freshness, and quality.
It offers a comprehensive suite including change data capture, high-speed streaming SQL, support for both historical and real-time data, and streaming visualisations—enabling faster implementation and more reliable data delivery.
Azure Event Hubs, Microsoft’s native streaming solution, is optimised for big data and stream analytics within the Azure ecosystem. It delivers a cost-effective alternative to Confluent, particularly for organisations already invested in Microsoft’s cloud stack.
Furthermore, Google Cloud Pub/Sub stands out for its ability to handle large-scale real-time streams with minimal complexity. Its robust scalability and native support for fan-in/fan-out architectures across geo-regions make it particularly valuable for companies with a significant footprint in Google Cloud.
The post Streaming is the New Cloud appeared first on Analytics India Magazine.
