CASE STUDY
Real-time migration to reporting
How Confluent works
The confluent data streaming platform reimagines your data architecture by building upon the heritage of Apache Kafka and Apache Flink to enable enterprise-wide adoption of data products.
With Confluent, your data architecture is no longer a complicated, expensive, and risky mess. The platform untangles data problems, breaking down data barriers and silos across your enterprise. It delivers universal data products that connect teams, systems, and applications, ensuring a consistent view of the most up-to-date data.
With the pillars describe below, Confluent’ s Data Streaming Platform transforms data mess into reusable, high quality data products. These data products remove the need for brittle point-to-point integrations, and can be leveraged by every team, in any organization, for operational, analytical, and never-before possible use cases.
Stream
Confluent reimagines data streaming through the award winning Kora engine, which removes the need to manage Apache Kafka. With Kora, you have the scale, elasticity, resiliency, availability, and security required for mission-critical workloads across hybrid and multi-cloud environments. Kora improves ROI by reducing TCO through pay-as-you-go storage.
Connect
Data streaming requires connections to be built, deployed, and managed for each data source. Confluent provides 120 + pre-built connectors that allow your teams to instantly connect data. Your teams are free from writing generic integration code and managing connectors.
Process
Confluent lets you join, enrich, and curate data at the source while removing the operational complexity and burden of running Apache Flink. Regardless of whether the data comes from the operational or analytical estate, it is presented in a consistent format that seamlessly works across both.
Govern
Confluent provides a suite of governance capabilities that enable your teams to maintain data contracts, classify and organize data into a neat catalogue, track data lineage and securely find and consume trustworthy data products through a self-service portal, while ensuring observability, compliance and confidentiality of data that is on the move.
Moniepoint also had to seamlessly migrate its transaction data to a reporting data store for analytical purposes. Its legacy infrastructure could not support real-time transfers of data without adding costs. With Confluent Cloud, Moniepoint found a managed solution to copy data from the master server to a reporting server.
Unlike the complexities of building custom solutions to copy data from master databases to reporting servers, Confluent’ s managed platform ensures efficient and real-time data replication, eliminating the need for additional infrastructure costs.
Decoupling services, enhancing scalability
Before the company’ s use of Confluent, certain products relied on database-level transactions that triggered important decisions for the product. The activation of these products was only possible with the development of a specialised layer.
The solution was to log the transactions on Confluent’ s managed streaming layer. After which, Felix said his scaling needs were resolved. The implementation of Confluent Cloud was so efficient that you had near real time data in the read replicas.
Moniepoint leveraged topics to expose events, empowering each financial solution to consume the data immediately and independently for decisionmaking, making reporting more efficient.
Breaking department monoliths
While Confluent helped Moniepoint scale, without the discipline of an intermediary service, the microservices evolved to become monoliths. The tightly coupled architecture could not separate the company’ s four financial products into microservices, without affecting each other’ s database.
For example, the payment and settlement services could not be scaled because they were single components. A reactive solution would be to break up the service databases.
The Moniepoint team discovered Confluent as an intermediary solution, by plugging in Confluent between the two services, which solved the problem elegantly. Once a transaction happens and the payment is successful, you log the event and then you move on. provided a scalable, reliable foundation for handling growing transaction volumes.
Redesigning the system architecture with Confluent Cloud at its core, Moniepoint revolutionised its
52 INTELLIGENTCIO AFRICA www. intelligentcio. com