Company Background
SHOPLINE is a commerce enablement platform that helps merchants launch, operate, and grow retail businesses across the region. From checkout to fulfillment and post-purchase workflows, SHOPLINE sits on the critical path of daily transactions—where faster, more consistent analytics translate directly into better operational decisions and better merchant experience.
For merchants, timely sales and order insights directly influence operational decisions—campaign adjustments, fulfillment prioritization, and inventory planning. SHOPLINE needed a reliable way to deliver real-time sales insights—such as order volume, GMV, items sold, and breakdowns by channel and payment method—while keeping the system maintainable as data volume and reporting needs grow.
Challenges: Real-Time Analytics Without the Overhead
SHOPLINE's analytics pipeline had to reconcile three realities.
1. From Lambda to Kappa: One Continuous Transformation Layer
SHOPLINE's real-time analytics stack had evolved into a classic Lambda architecture:
- A batch pipeline produced historical and reconciled results for reporting.
- A separate "speed layer" delivered near-real-time metrics by computing closer to the serving layer.
Over time, this created duplicated modeling logic—two sets of transformations to maintain and validate—and increased operational overhead whenever definitions changed or new metrics were introduced. SHOPLINE wanted to move toward a Kappa-style approach—one continuous transformation layer—without sacrificing correctness or freshness.
2. Analytics API Latency Became a User-Facing Issue
As merchants increasingly relied on real-time dashboards and analytics APIs, response-time consistency became critical.
In practice, the speed layer was doing heavy analytical work directly on the application database—joins, rollups, and repeated aggregations. As data volume and concurrency grew, these workloads started to contend with operational queries, driving up API response times and making latency less predictable, directly impacting the merchant experience.
3. Serving Cost Increased to Sustain Real-Time Computation
To support real-time analytics at scale, the application database often had to absorb heavy analytical workloads, driving up infrastructure cost and operational pressure. SHOPLINE needed a way to offload transformation and pre-compute metrics without sacrificing freshness.
As a result, SHOPLINE needed a unified transformation layer to continuously pre-compute business-ready metrics, improve API responsiveness, and control serving-layer costs.
The Solution: A Single SQL Transformation Layer Powered by RisingWave
SHOPLINE adopted RisingWave as a unified transformation layer that ingests from both Kafka and Object Storage and continuously maintains query-ready results using SQL and materialized views.
The implementation follows a layered model:
- Bronze: Raw data ingested from Kafka, plus historical and backfill data from Object Storage
- Silver: Reusable "clean + wide" tables for lookup joins
- Gold: Serving-ready aggregates designed for low-latency downstream queries
- Serving: Continuously publishes curated Gold tables to:
- Application databases for real-time access
- The data warehouse as intermediate tables for batch processing
Kafka streams + Object Storage backfill feed RisingWave Bronze/Silver/Gold MVs, then curated Gold tables are continuously sunk into the data warehouse and application database for customer-facing analytics.
This architecture consolidates transformation logic into a single place, so the same definitions apply consistently whether the data arrives via streaming topics or historical backfill.
Voice of the Customer
"By adopting RisingWave, we moved to a cost-effective Kappa architecture. It provides the low-latency real-time analytics we need and delivers a much smoother experience for our merchants."
Henry Chi, Manager, Data Team, SHOPLINE
Key Results and Business Impact
SHOPLINE's unified approach produced improvements across three areas.
Operational Wins
- Cost-Effective Scalability: Transitioned to a Kappa architecture that delivers low-latency analytics without the compounding infrastructure costs of maintaining legacy dual-stack Lambda systems.
- Infrastructure Simplicity: Eliminated the need to manage separate batch and streaming pipelines, significantly reducing the number of moving parts in the data ecosystem.
Engineering Velocity
- Declarative Development: New analytics are defined via DDL and materialized view changes instead of bespoke coding, enabling modular development through reusable Silver MVs.
- Scalable Backfills & Migrations: For heavy migrations, a hybrid bootstrap approach joins real-time streams with pre-computed historical tables—avoiding the massive resource spike of full-stream replays while ensuring data consistency.
Serving Performance
- Optimized Data Delivery: Gold tables are continuously synced to application databases and data warehouses, minimizing query-time joins and maximizing responsiveness.
- Proven Impact: API response times decreased by 76.7%, providing a seamless experience for merchants.
Interested in how RisingWave can power your real-time analytics? Get started for free or talk to our team.
