The Two Forces Influencing Supply Chains: Technology and Politics​

There are two types of globalization: one is the vast dispersion brought about by the Industrial Revolution, and the other is the significant convergence resulting from the Information Revolution, where developed countries provide capital and high technology while deindustrializing, and developing countries offer cheap labor and industrialization. The latter form of globalization has created a golden age for supply chains, but its impact on the world has been more sudden and difficult to control.

Currently, we find ourselves at the end of this latter globalization, with slowing growth. The Economist has coined the term slowbalisation to describe this phenomenon, exemplified by events like Brexit and the U.S.-China trade war.

On the technological front, advancements in AI, robotics, 3D printing, autonomous vehicles, and 5G are leading to unprecedented transformations in supply chains, making globalization more manageable. However, these changes are hindered by a technological cold war.

Changes in Industry​

More than half of companies believe they should implement major changes in their supply chains, with one in ten businesses advocating for a complete overhaul. There are two primary considerations:

One is the risk associated with reducing supply chain costs — you are merely part of a node in the supply chain network, making it difficult to control adjacent nodes. The typhoon in Japan in 2011 caused a semiconductor giant to spend over a year with 100 executives trying to understand the specific companies involved in the network created by their supply chain.

Another consideration is that global trade has not only brought more products but also more services. Services account for one-third of trade value, and their growth is 60% faster than that of products, with telecommunications and information industries seeing growth rates 2 to 3 times faster.

Examining three specific industries:

1. Apparel: Some production has shifted from China to Southeast Asia, Vietnam, Bangladesh, and Ethiopia. Despite rising costs, China's skilled labor remains highly competitive.

2. Automotive: Regionalization and hub formation are evident, with Mexico serving North America, Eastern Europe and Morocco serving Western Europe, and Southeast Asia and China serving Asia. A major reason for regionalization is differing consumer habits; for example, pickup trucks are only popular in the U.S. Other factors include trade wars and the electrification of vehicles (which require significantly fewer components).

3. Computing: Some companies are relocating from China to Vietnam, Cambodia, and Mexico, and even back to the U.S. Many companies find it challenging to move out of the U.S. Shenzhen remains a powerhouse due to its automation capabilities and high added value.

Amazon and Alibaba Remain Benchmarks for the Next Transformation​

Amazon uses data, algorithms, and robotics to achieve logistics that are one-third faster than most competitors.

Digitalization: How New Technologies Are Transforming Old Industries​

1. Predictability
   1. Companies traditionally use historical records to forecast sales and adjust manufacturing and inventory accordingly. AI can leverage social media data to more accurately fine-tune parameters at each node in the supply chain.
   2. Research shows that executives believe the most important technologies to adopt are cognitive analytics and AI, while blockchain and drones have seen a decline in priority.
   3. JDA's Blue Yonder deep learning algorithms have reduced stockouts by 30% and shortened excess inventory time by several days.
   4. ORSAY uses JDA's automated pricing system to reduce inventory.
   5. Intel saved $58 million using predictive models.
2. Transparency
   1. For multinational corporations, "Where are the goods?" remains a significant unresolved issue. Why? Because their control over products is not direct; they do not manufacture, transport, store, or sell the products.
      1. For example, in 2017, a strike in Germany's freight industry left IBM's expensive mainframes exposed to the elements on the tarmac for a month, while the company mistakenly believed the goods were safely stored in an airport warehouse.
   2. The Internet of Things (IoT) can address this issue. Sensors can report not only location but also direction, temperature, humidity, and other parameters.
   3. Singapore is automating port transport with autonomous vehicles.
   4. IBM and Maersk are using blockchain to make transportation paperless and transparent.
3. Speed
   1. In Silicon Valley, Tom Linton's Pulse command center resembles a Pentagon command center, allowing real-time visibility of 92 variables in the supply chain, sharing data and decisions with customers via mobile and computer, reducing inventory by 11 days and freeing up $580 million in cash flow.
   2. Li & Fung has reduced the time from product design to shelf from 40 weeks to half that.
   3. 3D printing unicorn Carbon is helping Ford and Adidas develop and produce products.
   4. Uber or Airbnb for warehousing.

Security Challenges​

Supply chain management is transitioning from intuition and experience-driven practices to data-driven approaches, facing three major security challenges:

1. Huawei. Replacing Huawei products will increase costs, and the adoption of incompatible technology standards for 5G will force different countries to take sides, while 5G technology is crucial for IoT.
2. Cyberattacks. New IoT devices are rushed to market without adequate security considerations; however, "no one wants to drive a tank to work" because it's just too slow. A better response strategy is to be reactive and quick.
3. Trade wars. Policies can change daily, but factories cannot be moved at will.

Requirements​

• realtime / low-latency typeahead and autocomplete service for social networks, like Linkedin or Facebook
• search social profiles with prefixes
• newly added account appear instantly in the scope of the search
• not for “query autocomplete” (like the Google search-box dropdown), but for displaying actual search results, including
  • generic typeahead: network-agnostic results from a global ranking scheme like popularity.
  • network typeahead: results from user’s 1st and 2nd-degree network connections, and People You May Know scores.

Architecture​

Multi-layer architecture

• browser cache
• web tier
• result aggregator
• various typeahead backend

Result Aggregator​

The abstraction of this problem is to find documents by prefixes and terms in a very large number of elements. The solution leverages these four major data structures:

1. InvertedIndex<prefixes or terms, documents>: given any prefix, find all the document ids that contain the prefix.
2. for each document, prepare a BloomFilter<prefixes or terms>: with user typing more, we can quickly filter out documents that do not contain the latest prefixes or terms, by check with their bloom filters.
3. ForwardIndex<documents, prefixes or terms>: previous bloom filter may return false positives, and now we query the actual documents to reject them.
4. scorer(document):relevance: Each partition return all of its true hits and scores. And then we aggregate and rank.

Performance​

• generic typeahead: latency < = 1 ms within a cluster
• network typeahead (very-large dataset over 1st and 2nd degree network): latency < = 15 ms
• aggregator: latency < = 25 ms

Acquisition Efficiency Problem：How to achieve a better ROI in advertising?​

In details, Lyft's advertisements should meet requirements as below:

1. being able to manage region-specific ad campaigns
2. guided by data-driven growth: The growth must be scalable, measurable, and predictable
3. supporting Lyft's unique growth model as shown below

However, the biggest challenge is to manage all the processes of cross-region marketing at scale, which include choosing bids, budgets, creatives, incentives, and audiences, running A/B tests, and so on. You can see what occupies a day in the life of a digital marketer:

We can find out that execution occupies most of the time while analysis, thought as more important, takes much less time. A scaling strategy will enable marketers to concentrate on analysis and decision-making process instead of operational activities.

Solution: Automation​

To reduce costs and improve experimental efficiency, we need to

1. predict the likelihood of a new user to be interested in our product
2. evaluate effectively and allocate marketing budgets across channels
3. manage thousands of ad campaigns handily

The marketing performance data flows into the reinforcement-learning system of Lyft: Amundsen

The problems that need to be automated include:

1. updating bids across search keywords
2. turning off poor-performing creatives
3. changing referrals values by market
4. identifying high-value user segments
5. sharing strategies across campaigns

Architecture​

The tech stack includes - Apache Hive, Presto, ML platform, Airflow, 3rd-party APIs, UI.

Main components​

Lifetime Value(LTV) forecaster​

The lifetime value of a user is an important criterion to measure the efficiency of acquisition channels. The budget is determined together by LTV and the price we are willing to pay in that region.

Our knowledge of a new user is limited. The historical data can help us to predict more accurately as the user interacts with our services.

Initial eigenvalue:

The forecast improves as the historical data of interactivity accumulates:

Budget allocator​

After LTV is predicted, the next is to estimate budgets based on the price. A curve of the form LTV = a * (spend)^b is fit to the data. A degree of randomness will be injected into the cost-curve creation process in order to converge a global optimum.

Bidders​

Bidders are made up of two parts - the tuners and actors. The tuners decide exact channel-specific parameters based on the price. The actors communicate the actual bid to different channels.

Some popular bidding strategies, applied in different channels, are listed as below:

Conclusion​

We have to value human experiences in the automation process; otherwise, the quality of the models may be "garbage in, garbage out". Once saved from laboring tasks, marketers can focus more on understanding users, channels, and the messages they want to convey to audiences, and thus obtain better ad impacts. That's how Lyft can achieve a higher ROI with less time and efforts.

Requirements​

• for guests
  • search rooms by locations, dates, number of rooms, and number of guests
  • get room details (like picture, name, review, address, etc.) and prices
  • pay and book room from inventory by date and room id
    • checkout as a guest
    • user is logged in already
  • notification via Email and mobile push notification
• for hotel or rental administrators (suppliers/hosts)
  • administrators (receptionist/manager/rental owner): manage room inventory and help the guest to check-in and check out
  • housekeeper: clean up rooms routinely

Architecture​

Components​

Inventory <> Bookings <> Users (guests and hosts)​

Suppliers provide their room details in the inventory. And users can search, get, and reserve rooms accordingly. After reserving the room, the user's payment will change the status of the reserved_room as well. You could check the data model in this post.

How to find available rooms?​

• by location: geo-search with spatial indexing, e.g. geo-hash or quad-tree.
• by room metadata: apply filters or search conditions when querying the database.
• by date-in and date-out and availability. Two options:
  • option 1: for a given room_id, check all occupied_room today or later, transform the data structure to an array of occupation by days, and finally find available slots in the array. This process might be time-consuming, so we can build the availability index.
  • option 2: for a given room_id, always create an entry for an occupied day. Then it will be easier to query unavailable slots by dates.

For hotels, syncing data​

If it is a hotel booking system, then it will probably publish to Booking Channels like GDS, Aggregators, and Wholesalers.

To sync data across those places. We can

1. retry with idempotency to improve the success rate of the external calls and ensure no duplicate orders.
2. provide webhook callback APIs to external vendors to update status in the internal system.

Payment & Bookkeeping​

Data model: double-entry bookkeeping

To execute the payment, since we are calling the external payment gateway, like bank or Stripe, Braintree, etc. It is crucial to keep data in-sync across different places. We need to sync data across the transaction table and external banks and vendors.

Notifier for reminders / alerts​

The notification system is essentially a delayer scheduler (priority queue + subscriber) plus API integrations.

For example, a daily cronjob will query the database for notifications to be sent out today and put them into the priority queue by date. The subscriber will get the earliest ones from the priority queue and send out if reaching the expected timestamp. Otherwise, put the task back to the queue and sleep to make the CPU idle for other work, which can be interrupted if there are new alerts added for today.

Requirements​

1. High-performance, distributed key-value store

• Why distributed?
  • Answer: to hold a larger size of data

2. For in-memory storage of small data objects
3. Simple server (pushing complexity to the client) and hence reliable and easy to deploy

Architecture​

Big Picture: Client-server

• client
• given a list of Memcached servers
• chooses a server based on the key
• server
• store KVs into the internal hash table
• LRU eviction

The Key-value server consists of a fixed-size hash table + single-threaded handler + coarse locking

How to handle collisions? Mostly three ways to resolve:

1. Separate chaining: the collided bucket chains a list of entries with the same index, and you can always append the newly collided key-value pair to the list.
2. open addressing: if there is a collision, go to the next index until finding an available bucket.
3. dynamic resizing: resize the hash table and allocate more spaces; hence, collisions will happen less frequently.

How does the client determine which server to query?​

See Data Partition and Routing

How to use cache?​

See Key value cache

How to further optimize?​

See How Facebook Scale its Social Graph Store? TAO

Customer Acquisition Efficiency Issue: How can advertising campaigns achieve higher returns with less money and fewer people?​

Specifically, Lyft's advertising campaigns need to address the following characteristics:

1. Manage location-based campaigns
2. Data-driven growth: growth must be scalable, measurable, and predictable
3. Support Lyft's unique growth model, as shown below:

The main challenge is the difficulty of scaling management across various aspects of regional marketing, including ad bidding, budgeting, creative assets, incentives, audience selection, testing, and more. The following image depicts a day in the life of a marketer:

We can see that "execution" takes up most of the time, while less time is spent on the more important tasks of "analysis and decision-making." Scaling means reducing complex operations and allowing marketers to focus on analysis and decision-making.

Solution: Automation​

To reduce costs and improve the efficiency of experimentation, it is necessary to:

1. Predict whether new users are interested in the product
2. Optimize across multiple channels and effectively evaluate and allocate budgets
3. Conveniently manage thousands of campaigns

Data is enhanced through Lyft's Amundsen system using reinforcement learning.

The automation components include:

1. Updating bid keywords
2. Disabling underperforming creative assets
3. Adjusting referral values based on market changes
4. Identifying high-value user segments
5. Sharing strategies across multiple campaigns

Architecture​

Technology stack: Apache Hive, Presto, ML platform, Airflow, 3rd-party APIs, UI.

Specific Component Modules​

LTV Prediction Module​

The lifetime value (LTV) of users is an important metric for evaluating channels, and the budget is determined by both LTV and the price we are willing to pay for customer acquisition in that region.

Our understanding of new users is limited, but as interactions increase, the historical data provided will more accurately predict outcomes.

Initial feature values:

As historical interaction records accumulate, the predictions become more accurate:

Budget Allocation Module​

Once LTV is established, the next step is to set the budget based on pricing. A curve of the form LTV = a * (spend)^b is fitted, along with similar parameter curves in the surrounding range. Achieving a global optimum requires some randomness.

Delivery Module​

This module is divided into two parts: the parameter tuner and the executor. The tuner sets specific parameters based on pricing for each channel, while the executor applies these parameters to the respective channels.

There are many popular delivery strategies that are common across various channels:

Conclusion​

It is essential to recognize the importance of human experience within the system; otherwise, it results in garbage in, garbage out. When people are liberated from tedious delivery tasks and can focus on understanding users, channels, and the messages they need to convey to their audience, they can achieve better campaign results—spending less time to achieve higher ROI.

The design lead at Spotify started various side projects after dropping out of school at 15. With decades of experience, he believes that successful side projects must be "stupid," allowing for bolder and more enjoyable exploration without burdens. All major endeavors start as small ones; 50% of Google's new projects originate from their famous 20% side projects. Complex issues like scaling, funding, and competitive analysis can hinder your progress. Taking action is more important than learning because others won't be smarter than you, and it's hard to empathize with others' experiences. Many successful startups are not planned. Projects attract like-minded individuals.

Another psychological benefit of side projects is that they are "fun." You are your own boss, and you can decide what to do without anyone stopping you. :)

1. empathy / perspective-taking is the most important.

   1. realize that code is written for human to read first and then for machines to execute.
   2. software is so "soft" and there are many ways to achieve one thing. It's all about making the proper trade-offs to fulfill the requirements.
   3. Invent and Simplify: Apple Pay RFID vs. Wechat Scan QR Code.

2. choose a sustainable architecture to reduce human resources costs per feature.

3. adopt patterns and best practices.

4. avoid anti-patterns

   • missing error-handling
   • callback hell = spaghetti code + unpredictable error handling
   • over-long inheritance chain
   • circular dependency
   • over-complicated code
     • nested tertiary operation
     • comment out unused code
   • missing i18n, especially RTL issues
   • don't repeat yourself
     • simple copy-and-paste
     • unreasonable comments

5. effective refactoring

   • semantic version
   • never introduce breaking change to non major versions
     • two legged change

Metrics Turn Vague Ideas into Great Ideas​

• A rough estimate is sufficient
• Focus on how to evaluate a great product
• Note, do not worry about the overall market size
  • Simply asking "Will it ultimately be large?" is enough

What Should We Measure About the Product?​

• Are users receiving value?
  • Active users
  • Profit
  • Transaction volume
  • Retention

Retention is Important​

The chart below shows the retention rate over time for each annual cohort.

If the cohort retention rate keeps declining without a lower limit, your growth will be like a fire ring in a field that cannot sustain itself because

• Old users will be burned out
• New users will become fewer

You should keep the retention rate above a certain line.

Conversely, if the cohort retention rate increases over time, it indicates that your product is excellent. For example:

1. Whatsapp
2. Uber
3. Facebook

Tom Tunguz: High Retention Means High Valuation

Growth​

If you have high expectations and want to achieve exponential growth, then assess your weekly growth rate. Strive to keep this weekly growth rate steady, and you will experience exponential growth.

Iteration Speed​

• Smaller companies have an advantage in iteration speed
• Metrics are key to guiding the direction of iterations
• Measure the speed of the iterations themselves

How Much Should You Value Metrics?​

1. Not enough emphasis
   • Easy to deceive oneself and hit a wall
2. Very high emphasis
   • Your decisions will be cautious
3. Overemphasis
   • You increase numbers but do not add value

Reality Can Be Painful​

• Metrics can shatter illusions, and sometimes it can be a bit painful
• Most leaders attract subordinates through illusions
• You should
  • Learn from the past
  • Clarify the present
  • Smile at the future

Paul Graham and Robert Morris's first startup around the age of 30, which involved putting art exhibitions online, ultimately turned out to be a bad idea, let alone for those in their 20s.

Why do smart people come up with bad business ideas?

1. Bad ideas are products of intuition.
   1. People often think of what to do based on what they see; however, if you plan to work on something for several years, you should weigh several different ideas beforehand.
   2. The most tragic part is the subsequent IKEA effect: despite being a bad idea, because you invested effort into it, you will grow fond of the idea.
   3. What to do? Getting started quickly is good, but you must recognize that ==spending time does not necessarily improve the situation.== Keep asking yourself: Would anyone be willing to pay for it?
2. Bad ideas prioritize being impressive over making money.
   1. If you want to make money, don’t shy away from dirty work: the dirtier and more laborious the work, the fewer people are willing to do it, leading to a tighter supply-demand relationship; if you take it on, your bargaining power increases.
   2. Since making money is hard, your primary goal should be to make money; otherwise, you won't make any.
3. Bad ideas yield to fear of competition.
   1. Competition may not be as fierce as the media portrays; they often lack technical knowledge and coding skills.
   2. Why do we fear? Understanding programming but not business? In reality, the term "business" is too broad and abstract, obscuring the concrete issues below: selling products, promotions, understanding what customers want, pricing, customer support, billing, etc. Once you tackle these details, you have resolved "business."
4. Bad ideas are too superficial.
   1. Most ideas are hybrids of blogs, calendars, dating sites, and social networks. Meanwhile, there are better, unsolved problems in the distance with high demand, yet no one is addressing them.
   2. Why is it difficult for people to research what customers truly want? The existing education system teaches people how to solve problems but fails to teach them how to ask questions.

Understanding what users want is challenging, as it requires recognizing that you need to invest time and effort into this task. Unsure how to start? The answer lies in Carnegie's classic "How to Win Friends and Influence People" — the answer is empathy, or putting yourself in others' shoes.

Fortunately, the ability to conduct user research can be learned; "smart creators" and "user research" are a perfect match that can unleash endless potential.

In a nutshell: Create products and services that the public loves and enjoys.