Lambda Architecture
· One min read
Why lambda architecture?
To solve three problems introduced by big data
- Accuracy (好)
- Latency (快)
- Throughput (多)
e.g. problems with scaling a pageview service in a traditional way
- You start with a traditional relational database.
- Then adding a pub-sub queue.
- Then scaling by horizontal partitioning or sharding
- Fault-tolerance issues begin
- Data corruption happens
The key point is that ==X-axis dimension alone of the AKF scale cube is not good enough. We should introduce Y-axis / functional decomposition as well. Lambda architecture tells us how to do it for a data system.==
What is lambda architecture?
If we define a data system as
Query = function(all data)
Then a lambda architecture is
batch view = function(all data at the batching job's execution time)
realtime view = function(realtime view, new data)
query = function(batch view. realtime view)
==Lambda architecture = CQRS (batch layer + serving layer) + speed layer==