Cold Standby: Use heartbeat or metrics/alerts to track failure. Provision new standby nodes when a failure occurs. Only suitable for stateless services.

Hot Standby: Keep two active systems undertaking the same role. Data is mirrored in near real time, and both systems will have identical data.

Warm Standby: Keep two active systems but the secondary one does not take traffic unless the failure occurs.

Checkpointing (or like Redis snapshot): Use write-ahead log (WAL) to record requests before processing. Standby node recovers from the log during the failover.

• cons
  • time-consuming for large logs
  • lose data since the last checkpoint
• usercase: Storm, WhillWheel, Samza

Active-active (or all active): Keep two active systems behind a load balancer. Both of them take in parallel. Data replication is bi-directional.

Design a system to take user-provided URLs and transform them to a shortened URLs that redirect back to the original. Describe how the system works. How would you allocate the shorthand URLs? How would you store the shorthand to original URL mapping? How would you implement the redirect servers? How would you store the click stats?

Assumptions: I generally don't include these assumptions in the initial problem presentation. Good candidates will ask about scale when coming up with a design.

• Total number of unique domains registering redirect URLs is on the order of 10s of thousands
• New URL registrations are on the order of 10,000,000/day (100/sec)
• Redirect requests are on the order of 10B/day (100,000/sec)
• Remind candidates that those are average numbers - during peak traffic (either driven by time, such as 'as people come home from work' or by outside events, such as 'during the Superbowl') they may be much higher.
• Recent stats (within the current day) should be aggregated and available with a 5 minute lag time
• Long look-back stats can be computed daily
• Targeted use cases: The shortened URLs are to be copied-and-pasted only.
  • The URLs won't be typed in via a keyboard. Therefore, don't worry about distinguishing between 0 and o, l and 1, etc.
  • The URLs won't be spelled out verbally. Therefore, there's no need to make the shortened URLs phonetic.

Assumptions​

1B new URLs per day, 100B entries in total the shorter, the better show statics (real-time and daily/monthly/yearly)

Encode Url​

http://blog.codinghorror.com/url-shortening-hashes-in-practice/

Choice 1. md5(128 bit, 16 hex numbers, collision, birthday paradox, 2^(n/2) = 2^64) truncate? (64bit, 8 hex number, collision 2^32), Base64.

• Pros: hashing is simple and horizontally scalable.
• Cons: too long, how to purify expired URLs?

Choice 2. Distributed Seq Id Generator. (Base62: az, AZ, 0~9, 62 chars, 62^7), sharding: each node maintains a section of ids.

• Pros: easy to outdate expired entries, shorter
• Cons: coordination (zookeeper)

KV store​

MySQL(10k qps, slow, no relation), KV (100k qps, Redis, Memcached)

A great candidate will ask about the lifespan of the aliases and design a system that purges aliases past their expiration.

Followup​

Q: How will shortened URLs be generated?

• A poor candidate will propose a solution that uses a single id generator (single point of failure) or a solution that requires coordination among id generator servers on every request. For example, a single database server using an auto-increment primary key.
• An acceptable candidate will propose a solution using an md5 of the URL, or some form of UUID generator that can be done independently on any node. While this allows distributed generation of non- colliding IDs, it yields large "shortened" URLs
• A good candidate will design a solution that utilizes a cluster of id generators that reserve chunks of the id space from a central coordinator (e.g. ZooKeeper) and independently allocate IDs from their chunk, refreshing as necessary.

Q: How to store the mappings?

• A poor candidate will suggest a monolithic database. There are no relational aspects to this store. It is a pure key-value store.
• A good candidate will propose using any light-weight, distributed store. MongoDB/HBase/Voldemort/etc.
• A great candidate will ask about the lifespan of the aliases and design a system that ==purges aliases past their expiration==

Q: How to implement the redirect servers?

• A poor candidate will start designing something from scratch to solve an already solved problem
• A good candidate will propose using an off-the-shelf HTTP server with a plug-in that parses the shortened URL key, looks the alias up in the DB, updates click stats and returns a 303 back to the original URL. Apache/Jetty/Netty/tomcat/etc. are all fine.

Q: How are click stats stored?

• A poor candidate will suggest write-back to a data store on every click
• A good candidate will suggest some form of ==aggregation tier that accepts clickstream data, aggregates it, and writes back a persistent data store periodically==

Q: How will the aggregation tier be partitioned?

• A great candidate will suggest a low-latency messaging system to buffer the click data and transfer it to the aggregation tier.
• A candidate may ask how often the stats need to be updated. If daily, storing in HDFS and running map/reduce jobs to compute stats is a reasonable approach If near real-time, the aggregation logic should compute stats

Q: How to prevent visiting restricted sites?

• A good candidate can answer with maintaining a blacklist of hostnames in a KV store.
• A great candidate may propose some advanced scaling techniques like bloom filter.

Why lambda architecture?​

To solve three problems introduced by big data

1. Accuracy (好)
2. Latency (快)
3. Throughput (多)

e.g. problems with scaling a pageview service in a traditional way

1. You start with a traditional relational database.
2. Then adding a pub-sub queue.
3. Then scaling by horizontal partitioning or sharding
4. Fault-tolerance issues begin
5. Data corruption happens

The key point is that ==X-axis dimension alone of the AKF scale cube is not good enough. We should introduce Y-axis / functional decomposition as well. Lambda architecture tells us how to do it for a data system.==

What is lambda architecture?​

If we define a data system as

Query = function(all data)

Then a lambda architecture is

batch view = function(all data at the batching job's execution time)
realtime view = function(realtime view, new data)

query = function(batch view. realtime view)

==Lambda architecture = CQRS (batch layer + serving layer) + speed layer==

Why Use Lambda Architecture?​

To address the three issues brought by big data:

1. Accuracy (good)
2. Latency (fast)
3. Throughput (high)

For example: The problems of scaling web browsing data records in a traditional way:

1. First, use a traditional relational database.
2. Then, add a "publish/subscribe" model queue.
3. Next, scale through horizontal partitioning or sharding.
4. Fault tolerance issues begin to arise.
5. Data corruption phenomena start to appear.

The key issue is that in the AKF Scaling Cube, ==having only the X-axis for horizontal partitioning of one dimension is not enough; we also need to introduce the Y-axis for functional decomposition. The lambda architecture can guide us on how to scale a data system==.

What is Lambda Architecture?​

If we define a data system in the following form:

Query=function(all data)

Then a lambda architecture is:

batch view = function(all data at the batching job's execution time)
realtime view = function(realtime view, new data)

query = function(batch view, realtime view)

==Lambda architecture = Read/Write separation (Batch Processing Layer + Service Layer) + Real-time Processing Layer==

Why Practice Nonviolent Communication?​

Improve communication quality by ==valuing everyone's needs==. ==Doubt and violence are manifestations of unmet needs==.

What NVC Is Not​

• It is not about appearing friendly.
• It is not about getting others to do what we want; it concerns mutual understanding between people.

Ways to Strengthen Connections and Understanding Between People​

1. Express our feelings and needs vulnerably
   • Recognize ongoing feelings and needs
   • Expose the vulnerability of feelings and needs
2. Actively listen to the feelings and needs of others
   • The core of empathetic listening: presence, focus, space, and care, ==along with verbal expression of feelings and needs==
   • Do not give advice, make judgments, comfort, tell stories, sympathize, analyze, or explain, ...
   • Regardless of what is said, the key is to listen to the other person's feelings, needs, opinions, and requests

For example: ==Because you need... so you feel...?==

Engaging in These Behaviors Can Lead to Distance Between Us​

• Evaluating others, making judgments, labeling, analyzing, criticizing, comparing, etc.
• Worth considering (i.e., certain behaviors deserve punishment or reward)
  • Demands (not accepting others' choices; wanting to punish those who do not act according to one's own ideas)
  • Refusing to choose or take responsibility (keywords: have to, should have, guess they will, they made me do it, etc.)

KYC (Know Your Customer) is Not Easy​

==How often do you get the chance to hear your customers describe the problems they face?== If you are an employee in a large company outside the marketing department, the answer is likely never.

Of course, if you do get the chance, it’s important to know that there are two very crucial parts to the ==core concept of buyer persona==:

1. Asking exploratory questions
2. Listening

==KYC (Know Your Customer)== is not easy. For example, in 2008, the sales of the iPhone 3G in Japan were poor. Japanese consumers were accustomed to recording videos with their phones and paying with debit cards or train passes.

Just Having a Buyer Profile is Not Enough​

A typical buyer profile cannot accurately inform marketers about the buyers' decisions to purchase. Marketers often make guesses based on demographics (such as age, income, marital status, education) or psychographics (personality, values, lifestyle, opinions).

Nonetheless, buyer profiles can provide some obvious answers. For instance, reaching out to a Chief Financial Officer (CFO) via email is quite challenging. Moreover, emphasizing the spaciousness of a car's cargo area, even if it can fit a large dog, is pointless for a woman who only has time to care for a goldfish.

The most effective way to build buyer persona models is not through guessing, but by researching those who have weighed their options, considered or rejected proposed solutions, and made decisions similar to the ones you want to influence.

Buyer persona = buyer profile (who will buy) + buyer insights (when/how/why they buy)

Relational database is the default choice for most storage use cases, by reason of ACID (atomicity, consistency, isolation, and durability). One tricky thing is "consistency" -- it means that any transaction will bring database from one valid state to another, which is different from Consistency in CAP theorem.

Schema Design and 3rd Normal Form (3NF)​

To reduce redundancy and improve consistency, people follow 3NF when designing database schemas:

• 1NF: tabular, each row-column intersection contains only one value
• 2NF: only the primary key determines all the attributes
• 3NF: only the candidate keys determine all the attributes (and non-prime attributes do not depend on each other)

Db Proxy​

What if we want to eliminate single point of failure? What if the dataset is too large for one single machine to hold? For MySQL, the answer is to use a DB proxy to distribute data, either by clustering or by sharding

Clustering is a decentralized solution. Everything is automatic. Data is distributed, moved, rebalanced automatically. Nodes gossip with each other, (though it may cause group isolation).

Sharding is a centralized solution. If we get rid of properties of clustering that we don't like, sharding is what we get. Data is distributed manually and does not move. Nodes are not aware of each other.

When customers evaluate a product or service, they weigh ==perceived value== against ==actual price==.

Here are 30 "elements of value."

1. Functional Value

   • Time savings
   • Simplified processes
   • Revenue generation
   • Risk reduction
   • Organization
   • Integration
   • Connectivity
   • Reduced effort
   • Trouble avoidance
   • Cost reduction
   • Quality
   • Variety
   • Sensory appeal (e.g., food and beverages)
   • Alerts

2. Emotional Value

   • Anxiety reduction
   • Reciprocity
   • Nostalgia
   • Design/Aesthetics
   • Badge value
   • Health
   • Therapeutic value
   • Pleasure/Entertainment
   • Attractiveness
   • Providing methods

3. Life-Changing

   • Providing hope
   • Self-actualization
   • Motivation
   • Heirloom
   • Connection/Affiliation

4. Social Impact

   • Self-transcendence

In a regular Internet service, the read:write ratio is about 100:1 to 1000:1. However, when reading from a hard disk, a database join operation is time-consuming, and 99% of the time is spent on disk seek. Not to mention a distributed join operation across networks.

To optimize the read performance, denormalization is introduced by adding redundant data or by grouping data. These four categories of NoSQL are here to help.

Key-value Store​

The abstraction of a KV store is a giant hashtable/hashmap/dictionary.

The main reason we want to use a key-value cache is to reduce latency for accessing active data. Achieve an O(1) read/write performance on a fast and expensive media (like memory or SSD), instead of a traditional O(logn) read/write on a slow and cheap media (typically hard drive).

There are three major factors to consider when we design the cache.

1. Pattern: How to cache? is it read-through/write-through/write-around/write-back/cache-aside?
2. Placement: Where to place the cache? client-side/distinct layer/server side?
3. Replacement: When to expire/replace the data? LRU/LFU/ARC?

Out-of-box choices: Redis/Memcache? Redis supports data persistence while Memcache does not. Riak, Berkeley DB, HamsterDB, Amazon Dynamo, Project Voldemort, etc.

Document Store​

The abstraction of a document store is like a KV store, but documents, like XML, JSON, BSON, and so on, are stored in the value part of the pair.

The main reason we want to use a document store is for flexibility and performance. Flexibility is achieved by the schemaless document, and performance is improved by breaking 3NF. Startup's business requirements are changing from time to time. Flexible schema empowers them to move fast.

Out-of-box choices: MongoDB, CouchDB, Terrastore, OrientDB, RavenDB, etc.

Column-oriented Store​

The abstraction of a column-oriented store is like a giant nested map: ColumnFamily<RowKey, Columns<Name, Value, Timestamp>>.

The main reason we want to use a column-oriented store is that it is distributed, highly-available, and optimized for write.

Out-of-box choices: Cassandra, HBase, Hypertable, Amazon SimpleDB, etc.

Graph Database​

As the name indicates, this database's abstraction is a graph. It allows us to store entities and the relationships between them.

If we use a relational database to store the graph, adding/removing relationships may involve schema changes and data movement, which is not the case when using a graph database. On the other hand, when we create tables in a relational database for the graph, we model based on the traversal we want; if the traversal changes, the data will have to change.

Out-of-box choices: Neo4J, Infinitegraph, OrientDB, FlockDB, etc.

KYC is not easy​

==How often do you have an opportunity to listen to your customers describe their problems?== The answer is probably NEVER if you are an employee positioned in a non-marketing department of a large company.

If you do have the chance, two things lie at the core of the ==buyer persona concept,==

1. asking probing questions
2. listening

==KYC (knowing your customer)== is not easy. e.g. iPhone 3G was not selling well in Japan in 2008. Japanese customers were accustomed to using phones to shoot videos / pay with debit card chips / train pass chips.

A buyer's profile is good but not enough​

A generic buyer profile cannot make marketer understand exactly what determine’s the buyer’s buying decision. Marketers are just guessing based on demographics (age, income, marital status, education) or psychographics (personality, values, lifestyles, opinions).

The buyer profile can still give some obvious answers though. e.g. reaching CFO via an email campaign is so difficult. Emphasizing the spaciousness of the car’s cargo for a large dog is not useful for a busy woman that only raises goldfishes.

Rather than guessing, the most effective way to build buyer personas is to interview buyers who have previously weighed their options, considered or rejected solutions and made a decision similar to the one you want to influence.

Buyer persona = buyer profile (who will buy) + buyer insights (when/how/why to buy)