Project Challenge
Objective
While most URL shorteners prioritize simplicity, our challenge was to build a system that could operate at internet scale — without sacrificing speed, reliability, or insight.
We engineered a low-latency architecture powered by a high-performance key-value store and CDN-first routing to achieve sub-millisecond redirects. The platform supports millions of requests per second with consistent throughput and zero performance degradation under load.
On top of that, we integrated a real-time analytics layer capable of capturing clickstream metadata — including referrer, device, location, and time-to-click — and surfaced it via a developer-friendly API. Every redirect isn’t just fast — it’s measured, observable, and built for business impact.
The Research
Technical Architecture
Our research began with comprehensive benchmarking of in-memory datastores (Redis, Aerospike), persistent KV stores (RocksDB), and distributed caches to identify the optimal storage layer. Redis Cluster emerged as our primary choice after achieving 850K QPS with p99 latency under 0.8ms across our distributed test harness.
We implemented a custom base62 encoding algorithm with collision detection that generates compact, human-readable URLs while maintaining uniqueness guarantees. The encoding layer incorporates a two-tier Bloom filter (8-bit and 16-bit variants) that reduces unnecessary database lookups by 97.3%, significantly decreasing average response time.
For edge deployment, we developed a novel consistent hashing strategy that distributes shortened URLs across 38 global PoPs while maintaining cache coherency. Our CDN configuration uses a custom Varnish VCL implementation with strategic TTLs based on URL access patterns, achieving a global cache hit ratio of 94.2%.
The analytics pipeline implements a streaming architecture using Kafka (400K events/sec) with custom retention policies, compressing telemetry data by 78% through Protobuf serialization before persisting to ClickHouse for real-time querying at millisecond-level granularity across billions of redirects.