Is Innovation Disruptive?​

Disruptive Innovation vs. Continuous Innovation

• Does it ==change our current behavior patterns== or ==alter the other products and services we rely on==?
• There exists a range of demands for behavioral change between continuous and discontinuous innovation.

High-tech companies frequently introduce disruptive innovations. During these disruptive innovations, people become users of these products. This transition typically occurs in a normal distribution, resulting in an S-curve for user growth.

When Do People Buy High-Tech Products?​

In the ==technology adoption lifecycle==, people become users of disruptive innovation products at different stages. They are:

1. Innovators
2. Early Adopters
3. Early Majority (Pragmatists)
4. Late Majority (Conservatives)
5. Laggards

What is the High-Tech Market Model?​

This cycle provides guidance for the ==high-tech market model==, ==showing how to develop a high-tech market by smoothly transitioning this cycle from left to right, breaking through one user group at a time.== Leveraging the momentum of the left-side user group makes it easier to market products to the right-side user group.

User momentum is crucial because it can:

1. Save costs
2. Accelerate the cycle if you don't want to miss opportunities in the next disruptive innovation or against competitors.

Where is the Chasm?​

Observing the technology adoption lifecycle, we can see

• Two chasms

  1. There is a chasm between Innovators and Early Adopters, which is the appropriate application scenario. For example, Esperanto, VRML, Second Life, 3D printing. To cross this chasm, we need a flagship product.
  2. There is another chasm between the Early Majority and the Late Majority, which is competitive products. For example, home automation, scanning, and project management software. To cross this chasm, we need to make the product more acceptable.

• And a gap

  1. The gap from Early Adopters to the Early Majority. Their needs are different:

     1. Early Adopters want a change -- they expect to achieve a leap in a competitive market. Minor issues with the product are acceptable.
     2. The majority of Early Majority users, who are pragmatists, want increased productivity. They want technology to ==enhance, rather than disrupt their fundamental ways of working==.

  2. Incompatibility between the first two stages and the last two stages

     1. The behavior of Early Adopters is not a good reference for the majority of Early Majority users.
     2. Furthermore, the majority of Early Majority users do not want to disrupt their existing organizational structures. A good reference is crucial for their decision on whether to use this product.

  3. Who really encountered this gap in 2014? For example, holograms, pen-based tablets, fuel cells, QR codes (in the U.S.), most online courses, Segways, Motorola Iridium.

Motivation​

How to keep users' viewing data in scale (billions of events per day)?

Here, viewing data means...

1. viewing history. What titles have I watched?
2. viewing progress. Where did I leave off in a given title?
3. on-going viewers. What else is being watched on my account right now?

Architecture​

The viewing service has two tiers:

1. stateful tier = active views stored in memory

   • Why? to support the highest volume read/write
   • How to scale out?
     • partitioned into N stateful nodes by account_id mod N
       • One problem is that load is not evenly distributed and hence the system is subject to hot spots
     • CP over AP in CAP theorem, and there is no replica of active states.
       • One failed node will impact 1/nth of the members. So they use stale data to degrade gracefully.

2. stateless tier = data persistence = Cassandra + Memcached

   • Use Cassandra for very high volume, low latency writes.
     • Data is evenly distributed. No hot spots because of consistent hashing with virtual nodes to partition the data.
   • Use Memcached for very high volume, low latency reads.
     • How to update the cache?
       • after writing to Cassandra, write the updated data back to Memcached
       • eventually consistent to handling multiple writers with a short cache entry TTL and a periodic cache refresh.
     • in the future, prefer Redis' appending operation to a time-ordered list over "read-modify-writes" in Memcached.

How could APIs be un-robust and un-predictable?

1. Networks are unreliable.
2. Servers are more reliable but may still fail.

How to solve the problem? 3 Principles:

1. Client retries to ensure consistency.

2. Retry with idempotency and idempotency keys to allow clients to pass a unique value.

   1. In RESTful APIs, the PUT and DELETE verbs are idempotent.
   2. However, POST may cause ==“double-charge” problem in payment==. So we use a ==idempotency key== to identify the request.
      1. If the failure happens before the server, then there is a retry, and the server will see it for the first time, and process it normally.
      2. If the failure happens in the server, then ACID database will guarantee the transaction by the idempotency key.
      3. If the failure happens after the server’s reply, then client retries, and the server simply replies with a cached result of the successful operation.

3. Retry with ==exponential backoff and random jitter==. Be considerate of the ==thundering herd problem== that servers that may be stuck in a degraded state and a burst of retries may further hurt the system.

For example, Stripe’s client retry calculates the delay like this...

def self.sleep_time(retry_count)
  # Apply exponential backoff with initial_network_retry_delay on the
  # number of attempts so far as inputs. Do not allow the number to exceed
  # max_network_retry_delay.
  sleep_seconds = [Stripe.initial_network_retry_delay * (2 ** (retry_count - 1)), Stripe.max_network_retry_delay].min

  # Apply some jitter by randomizing the value in the range of (sleep_seconds
  # / 2) to (sleep_seconds).
  sleep_seconds = sleep_seconds * (0.5 * (1 + rand()))

  # But never sleep less than the base sleep seconds.
  sleep_seconds = [Stripe.initial_network_retry_delay, sleep_seconds].max

  sleep_seconds
end

Why are APIs unreliable?

1. Networks can fail.
2. Servers can fail.

How can we solve this problem? Three principles:

1. The client uses "retry" to ensure state consistency.

2. The retry requests must include an ==idempotent unique ID==.

   1. In RESTful API design, the semantics of PUT and DELETE are inherently idempotent.
   2. However, POST in online payment scenarios may lead to the ==“duplicate payment” issue==, so we use an "idempotent unique ID" to identify whether a request has been sent multiple times.
      1. If the error occurs before reaching the server, after retrying, the server sees it for the first time and processes it normally.
      2. If the error occurs on the server, based on this "unique ID," an ACID-compliant database ensures that this transaction occurs only once.
      3. If the error occurs after the server returns a result, after retrying, the server only needs to return the cached successful result.

3. Retries must be responsible, such as following the ==exponential backoff algorithm==, because we do not want a large number of clients to retry simultaneously.

For example, Stripe's client calculates the wait time for retries like this:

def self.sleep_time(retry_count)
  # Apply exponential backoff with initial_network_retry_delay on the
  # number of attempts so far as inputs. Do not allow the number to exceed
  # max_network_retry_delay.
  sleep_seconds = [Stripe.initial_network_retry_delay * (2 ** (retry_count - 1)), Stripe.max_network_retry_delay].min

  # Apply some jitter by randomizing the value in the range of (sleep_seconds
  # / 2) to (sleep_seconds).
  sleep_seconds = sleep_seconds * (0.5 * (1 + rand()))

  # But never sleep less than the base sleep seconds.
  sleep_seconds = [Stripe.initial_network_retry_delay, sleep_seconds].max

  sleep_seconds
end

AKF scale cube visualizes the scaling process into three dimensions…

1. ==Horizontal Duplication== and Cloning (X-Axis). Having a farm of identical and preferably stateless instances behind a load balancer or reverse proxy. Therefore, every request can be served by any of those hosts and there will be no single point of failure.
2. ==Functional Decomposition== and Segmentation - Microservices (Y-Axis). e.g. auth service, user profile service, photo service, etc
3. ==Horizontal Data Partitioning== - Shards (Z-Axis). Replicate the whole stack to different “pods”. Each pod can target a specific large group of users. For example, Uber had China and US data centers. Each datacenter might have different “pods” for different regions.

Want an example? Go to see how Facebook scale its social graph data store.

==One Word: Split==

==The AKF Scale Cube== tells us the three dimensions of "splitting":

1. ==Horizontal Scaling== Place many stateless servers behind a load balancer or reverse proxy, so that each request can be handled by any of those servers, eliminating single points of failure.
2. ==Business Splitting== Typical microservices divided by function, such as auth service, user profile service, photo service, etc.
3. ==Data Partitioning== Separate the entire technology stack and data storage specifically for a large group of users, for example, Uber has data centers in China and the United States, with different Pods for different cities or regions within each data center.

Research found that

1. Successful Caucasian men receive more career guidance than women and multicultural professionals.
2. More women have mentors than men, but fewer women win promotions.

Why? ==Mentors cannot help with promotions but sponsors can==.

What is a sponsor?​

A sponsor is

• ==Investor and Beneficiary==. A sponsor is a senior level staff member invested in a protégé’s career success.
• ==Direct influencer of your promotion==. Sponsors promote protégés directly, using their influence and networks to connect them to high-profile assignments, people, pay increases and promotions.
• ==Resource-sharer==. Sponsors give protégés their active network connections and make new connections for them.
• ==Amplifier==. Sponsors champion their protégés visibility, often using their own platforms and reputation as a medium for exposure.

Who can be the sponsors?​

1. who makes pay, promotion or project assignment decisions that affect you/your employee?
2. which senior leaders could benefit from you/your employee’s career advancement?
3. which senior leader has a network or platform most equipped to help you/your employee advance in their career?

Research Findings

1. Successful white men receive more career guidance than women and professionals from other cultural backgrounds.
2. Women have more career mentors than men, but they get promoted less than men.

Why? ==Because career mentors cannot help you get promoted, but career sponsors can==.

What is a Career Sponsor?​

A sponsor is...

• ==Your career investor and beneficiary==. They are a higher-level employee who can invest in your career development and benefit from it.
• ==A direct influencer of your promotions==. They can directly promote you or use their influence to help you get promoted.
• ==A resource sharer==. They can share their network resources to assist you.
• ==An amplifier==. They can leverage their platform and reputation to enhance your visibility in the company.

Who is Your Career Sponsor?​

1. Who controls the budget? Who decides promotions? Who allocates projects?
2. Which leader can benefit from your career achievements?
3. Which leader's network and platform can help you or your subordinates get promoted?

Motivation​

Video service over Http Live Streaming for mobile devices, which...

1. ==has limited memory/storage==
2. suffers from the unstable network connection and variable bandwidth, and needs ==midstream quality adjustments.==

Solution​

1. Server-side: In a typical configuration, a hardware encoder takes audio-video input, encodes it as H.264 video and AAC audio, and outputs it in an MPEG-2 Transport Stream

   1. the stream is then broken into a series of short media files (.ts possibly 10s) by a software stream segmenter.
   2. The segmenter also creates and maintains an index(.m3u8) file containing a list of the media files.
   3. Both the media fils and the index files are published on the web server.

2. Client-side: client reads the index, then requests the listed media files in order and displays them without any pauses or gaps between segments.

Architecture​

The essence of KV Cache is to reduce data access latency. For example, it transforms the O(logN) read/write and complex queries on a database that is expensive and slow into O(1) read/writes on a medium that is fast but also costly. There are many strategies for cache design, with common ones being read-through/write-through (or write-back) and cache aside.

The typical read/write ratio for internet services ranges from 100:1 to 1000:1, and we often optimize for reads.

In distributed systems, these patterns represent trade-offs between consistency, availability, and partition tolerance, and the specific choice should be based on your business needs.

General Strategies​

• Read
  • Read-through: A cache layer is added between clients and databases, so clients do not access the database directly but instead access it indirectly through the cache. If the cache is empty, it updates from the database and returns the data; if not, it returns the data directly.
• Write
  • Write-through: Clients first write data to the cache, which then updates the database. The operation is considered complete only when the database is updated.
  • Write-behind/Write-back: Clients first write data to the cache and receive a response immediately. The cache is then asynchronously updated to the database. Generally, write-back is the fastest.
  • Write-around: Clients write directly to the database, bypassing the cache.

Cache Aside Pattern​

Use the Cache Aside pattern when the cache does not support read-through and write-through/write-behind.

Reading data? If the cache hits, read from the cache; if it misses, read from the database and store in the cache. Modifying data? First modify the database, then delete the cache entry.

Why not update the cache after writing to the database? The main concern is that two concurrent database write operations could lead to two concurrent cache updates, resulting in dirty data.

Does using Cache Aside eliminate concurrency issues? There is still a low probability of dirty data occurring, especially when reading from the database and updating the cache while simultaneously updating the database and deleting the cache entry.

Where to Place the Cache?​

• Client side,
• Distinct layer,
• Server side.

What to Do If the Cache Size Is Insufficient? Cache Eviction Strategies​

• LRU - Least Recently Used: Keeps track of time and retains the most recently used items, evicting those that have not been used recently.
• LFU - Least Frequently Used: Tracks usage frequency, retaining the most frequently used items and evicting the least frequently used ones.
• ARC: Performs better than LRU by maintaining both recently used (RU) and frequently used (FU) items, while also recording the history of recently evicted items.

Which Cache Solution Is the Best?​