Target Audience​

Individuals with some or little experience, or those who have not held any leadership or design positions in their careers (whether formal or informal).

Problem Description​

Describe a project experience from your past that you found particularly interesting or memorable. Follow-up questions include:

• Why did you find it interesting?
• What was the most challenging part of the project, and how did you address those challenges?
• What did you learn from this project? What would you have liked to know before starting the project?
• Did you consider other design or implementation methods? Why did you choose the approach you took? If you were to choose again for the same project, what would you do differently?

Interviewer Tips​

Since the goal here is to assess a person's technical communication skills and level of interest, and they may have participated in boot camps, you should be prepared to ask them more questions (whether for more details or about other aspects of the project). If they are recent graduates who have just completed their thesis, the thesis is often a good starting point. While this question is similar in many ways to resume questions in phone interviews, the content is about four times that of a phone interview and should be proportionately more detailed in asking what they did. Therefore, while the scoring criteria are similar, they should be evaluated with higher expectations and more data.

Scoring​

Excellent candidates can:

• Discuss project experiences thoroughly, with interactions in the interview being a dialogue rather than a directive.
• Have a good understanding of the entire project, not just their area of focus, and be able to clearly articulate the design and intent of the project.
• Show passion for any project and clearly describe the project elements that sparked that passion.
• Clearly explain what alternatives were considered and why they chose the implementation strategy they took.
• Reflect on their experiences and learn from them.

Good candidates may:

• Encounter some questions in the interview but can resolve them with the interviewer's help.
• Lack a broader understanding of the project but still have a strong grasp of the parts they interacted with directly and specific areas.
• Appear passionate but may struggle to accurately explain where that passion comes from.
• Discuss alternatives they considered but may not have thought deeply about them.
• Reflect on their past experiences and draw lessons from them.

Poor candidates exhibit:

• Struggle during the interview, making the interviewer feel like the candidate is interrogating them rather than having a conversation.
• Lack detailed knowledge of the project even in their field of work. They may not understand why their product was designed that way or how it interacts with other systems.
• When asked about the most interesting projects they have worked on, they should show interest in the product, but in fact, they may appear disinterested.
• Be unfamiliar with potential alternative implementation methods.
• Seem not to have reflected on or learned from their past project experiences. A key indicator of this situation is that answers to "What did you learn?" and "What would you do differently?" are very brief or nearly identical.

What is an interview?​

An interview is a process for workers to find future co-workers, during which they are finding signals to answer the following three key questions:

1. capability question - can you do the job?
2. willingness question - will you do the job?
3. culture-fit question - will you fit in?

Why is soft skill so important?​

None of the critical questions above can be answered without good communication. Your job will be taken away by people who can talk better than you.

Generic answers (stories) to prepare​

1. hard-won triumph. How do you deal with failure? Briefly talk about the harshest moment, and then focus on how you battled back, and then salute the people who helped you. It demonstrates that you have the grit, team-building skills, and relevance to the job.
2. influence. Can you guide people to yes? leaders = visionaries who inspire self-sacrifice. A leader does not exist without the ability to persuade.
3. technical skills. Do you have a story proving your great technical skills?
4. culture-fit. The FBI used to ask prospective agents what books they read until an underground network of tipsters figured out the ideal answer: “Tom Clancy spy novels.”
5. fascination. What's fascinating about you? What makes you different from other people?

Apache Kafka is a distributed streaming platform.

Why use Apache Kafka?​

Its abstraction is a ==queue== and it features

• a distributed pub-sub messaging system that resolves N^2 relationships to N. Publishers and subscribers can operate at their own rates.
• super fast with zero-copy technology
• support fault-tolerant data persistence

It can be applied to

• logging by topics
• messaging system
• geo-replication
• stream processing

Why is Kafka so fast?​

Kafka is using zero copy in which that CPU does not perform the task of copying data from one memory area to another.

Without zero copy:

With zero copy:

Architecture​

Looking from outside, producers write to brokers, and consumers read from brokers.

Data is stored in topics and split into partitions which are replicated.

1. Producer publishes messages to a specific topic.
   • Write to in-memory buffer first and flush to disk.
   • append-only sequence write for fast write.
   • Available to read after write to disks.
2. Consumer pulls messages from a specific topic.
   • use an "offset pointer" (offset as seqId) to track/control its only read progress.
3. A topic consists of partitions, load balance, partition (= ordered + immutable seq of msg that is continually appended to)
   • Partitions determine max consumer (group) parallelism. One consumer can read from only one partition at the same time.

How to serialize data? Avro

What is its network protocol? TCP

What is a partition's storage layout? O(1) disk read

How to tolerate fault?​

==In-sync replica (ISR) protocol==. It tolerates (numReplicas - 1) dead brokers. Every partition has one leader and one or more followers.

Total Replicas = ISRs + out-of-sync replicas

1. ISR is the set of replicas that are alive and have fully caught up with the leader (note that the leader is always in ISR).
2. When a new message is published, the leader waits until it reaches all replicas in the ISR before committing the message.
3. ==If a follower replica fails, it will be dropped out of the ISR and the leader then continues to commit new messages with fewer replicas in the ISR. Notice that now, the system is running in an under replicated mode.== If a leader fails, an ISR is picked to be a new leader.
4. Out-of-sync replica keeps pulling message from the leader. Once catches up with the leader, it will be added back to the ISR.

Is Kafka an AP or CP system in CAP theorem?​

Jun Rao says it is CA, because "Our goal was to support replication in a Kafka cluster within a single datacenter, where network partitioning is rare, so our design focuses on maintaining highly available and strongly consistent replicas."

However, it actually depends on the configuration.

1. Out of the box with default config (min.insync.replicas=1, default.replication.factor=1) you are getting AP system (at-most-once).

2. If you want to achieve CP, you may set min.insync.replicas=2 and topic replication factor of 3 - then producing a message with acks=all will guarantee CP setup (at-least-once), but (as expected) will block in cases when not enough replicas (<2) are available for particular topic/partition pair.

Apache Kafka is a distributed streaming platform.

Why use Apache Kafka?​

Its abstraction is a ==queue==, and its features include:

• A distributed publish-subscribe (pub-sub) messaging system that simplifies N ^ 2 relationships into N. Publishers and subscribers can operate at their own rates.
• Ultra-fast zero-copy technology.
• Support for fault-tolerant data persistence.

It can be applied to:

• Logging by topic.
• Messaging systems.
• Off-site backups.
• Stream processing.

Why is Kafka so fast?​

Kafka uses zero-copy technology, where the CPU does not perform the task of copying data across storage area replicas.

Without zero-copy technology:

With zero-copy technology:

Architecture​

From the outside, producers write to the Kafka cluster, while users read from the Kafka cluster.

Data is stored by topic and divided into partitions of replicable replicas.

1. Producers publish messages to specific topics.
   • First, they are written to an in-memory buffer and then updated to disk.
   • To achieve fast writes, an append-only sequential write is used.
   • Messages can be read only after being written.
2. Consumers fetch messages from specific topics.
   • They use an "offset pointer" (offset is the SEQ ID) to track/control their unique reading progress.
3. A topic includes partitions and load balancing, where each partition is an ordered, immutable sequence of records.
   • Partitions determine the parallelism of users (groups). At any given time, a user can read from only one partition.

How to serialize data? Avro

What is its network protocol? TCP

What is the storage layout within a partition? O(1) disk reads.

How does fault tolerance work?​

==In-Sync Replicas (ISR) protocol==. It allows (numReplicas - 1) nodes to fail. Each partition has one leader and one or more followers.

Total replicas = In-sync replicas + Out-of-sync replicas

1. ISR is a set of live replicas that are in sync with the leader (note that the leader is always in the ISR).
2. When publishing new messages, the leader waits to commit the message until it has been received by all replicas in the ISR.
3. ==If a follower fails to stay in sync, it will exit the ISR, and then the leader will continue to commit new messages with fewer replicas in the ISR. Note that at this point, the system is operating in a low-replica state.== If a leader fails, another ISR will be elected as a new leader.
4. Out-of-sync replicas continuously pull messages from the leader. Once they catch up to the leader, they will be added back to the ISR.

Is Kafka an AP or CP system in the CAP theorem?​

Jun Rao believes it is CA because "our goal is to support replication within a Kafka cluster in a single data center, where network partitions are rare, so our design focuses on maintaining high availability and strong consistency of replicas."

However, it actually depends on the configuration.

1. If using the initial configuration (min.insync.replicas=1, default.replication.factor=1), you will have an AP system (at most once).
2. If you want to achieve CP, you can set min.insync.replicas=2, topic replication factor to 3, and then generate acks=all messages to guarantee CP settings (at least once). However, if there are not enough replicas (replica count < 2) for a specific topic/partition, writing will not succeed.

Why fall sleep fast?​

In the battle ground, if you don’t sleep, you’ll burn out pretty quickly. You’ll make bad decisions. You’ll let people down and become a liability.

How to fall asleep in 120 seconds?​

Principle: Relax the whole body one part by another and don’t think.

1. Stretch out
2. ==Relax face muscles and slow down everything==
3. Relax upper body
   1. Let shoulders drop as low as they can
   2. Breathe in deeply. Then exhale slowly, blowing out all of the tension
   3. Relax arms. If it’s not relaxing, tense it first, then let it go loose
   4. Relax fore arm
   5. Relax hand
4. Relax legs
5. Not think about anything
   1. keep your mind still. You can do this by holding a static image in your head
   2. If that doesn’t work, say the words “don’t think… don’t think… don’t think” over and over for at least 10 second

Why Fall Asleep Quickly?​

On the battlefield, if you don't sleep, you will quickly feel exhausted, make wrong decisions, feel burdened by your mission, and become a liability.

How to Do It?​

Principle: Gradually relax your entire body, avoiding thoughts.

1. Stretch your body
2. Relax your facial muscles, slow everything down
3. Relax your upper body
   1. Lower your shoulders as much as possible
   2. Take a deep breath, then exhale slowly, releasing all tension
   3. Relax your arms. If they are not relaxed enough, tense them first, then let them relax
   4. Relax your forearms
   5. Relax your hands
4. Relax your legs
5. Think of nothing at all
   1. Keep your mind calm. You can do this by holding a still image in your mind
   2. If that doesn't work, keep saying "don't think... don't think... don't think" for at least 10 seconds

Answer: Morality, Benevolence, Righteousness, Propriety

• Dao: Truth is the law of the world and human development. It embodies Mao Zedong's principle of "seeking truth from facts" and Ray Dalio's "Embrace Reality And Deal With It." How can one grasp the truth? By observing more, learning more, asking questions, and solving problems. The truth that is held by a few and can benefit others is called insight, while the ability to predict future outcomes based on truth is called vision. When the truth you discover helps others, they will help you in return, which is the principle of "gaining support through righteousness."

• Virtue: This means "give people what they want," serving the people, and efficiently creating products and services that the public enjoys. In companies and organizations, virtue is about aligning the goals of subordinates, managers, and the company to create synergy.

• Benevolence: Empathy, the ability to see things from others' perspectives and understand what they are thinking. It also involves genuinely wishing for others to succeed.

• Righteousness: Fairness and justice; achieving success requires rewarding good deeds and punishing wrongdoings.

• Propriety: Adhering to rules, which means predictable professionalism, where actions meet the community's expectations and are subject to peer review. For example, as a software engineer, my goal is not only for users to enjoy my product but also for my peers to recognize my achievements, which is the true success in my profession. In East Asian rice civilization, diligence, waking early and sleeping late, and being earnest align with Eastern propriety. In the Western world, mercantilism emphasizes mutual benefit in business dealings, creating a pleasant atmosphere, which aligns with Western propriety. Eastern people should not expect Westerners to admire their diligence, nor should Westerners expect Easterners to praise their flattering words.

With these characteristics, one can achieve harmony among people. Coupled with seizing the right timing, great accomplishments can be achieved. If one is born at the wrong time, they may simply remain unknown in the world. ¯\_(ツ)_/¯

Original Text​

The First Chapter of the Original Text

The five elements of Dao, Virtue, Benevolence, Righteousness, and Propriety are one body.

Dao is what people follow, allowing all things to be unaware of their origins. Virtue is what people obtain, enabling all things to achieve their desires. Benevolence is what people are close to, possessing compassion and empathy to sustain their existence. Righteousness is what people ought to pursue, rewarding good and punishing evil to establish achievements. Propriety is what people practice, rising early and retiring late to maintain the order of human relationships.

To be the foundation of humanity, one cannot lack any of these.

The wise and virtuous understand the principles of prosperity and decline, comprehend the numbers of success and failure, discern the trends of governance and chaos, and grasp the principles of advancement and retreat. When the time is right, they can reach the heights of ministerial positions; when opportunities arise, they can achieve extraordinary accomplishments. ==If they are not fortunate, they may simply remain unknown==. Therefore, their principles are lofty, and their names are esteemed by future generations.

Source: China Academy of Information and Communications Technology Trusted Blockchain Promotion Plan

What is Blockchain​

Blockchain is a technology maintained collaboratively by multiple parties, using cryptography to ensure secure transmission and access, enabling consistent data storage, tamper resistance, and non-repudiation. It is also referred to as Distributed Ledger Technology (DLT). A typical blockchain stores data in a block-chain structure. As a new computing paradigm and collaborative model for establishing trust at low cost in an untrusted competitive environment, blockchain is changing the application scenarios and operational rules across various industries with its unique trust-building mechanisms, making it an indispensable technology for the future development of the digital economy and the establishment of a new trust system.

Blockchains can be classified based on whether they have a node admission mechanism:

• Permissioned chains, where only authorized entities can join. They can be further divided based on whether the controlling entities are centralized:
  • Consortium chains
  • Private chains
• Permissionless/public chains

Features:

1. Evolution from double-entry bookkeeping to distributed bookkeeping
2. Transition from "add, delete, modify, query" to only "add, query"
3. Shift from single-party maintenance to multi-party maintenance
4. Development from external contracts to built-in contracts

Applicable Scenarios: New types of databases, multiple business entities, mutual distrust, and strong business interrelations.

Architecture​

1. Infrastructure = Hardware resources
2. Core components = Network discovery + Data transmission and reception + Cryptographic library + Data storage + Message notification
3. Ledger = Blockchain +== (Asset-based data model OR Account-based data model) ==
4. Consensus = Write first, then reach consensus (PoW, PoS, DPoS) OR Reach consensus first, then write (PBFT)
5. Smart contracts = Turing complete OR Non-Turing complete
6. System management = CA certification OR PKI certification OR Third-party identity verification
7. Interface = RPC + SDK
8. Applications = Value transfer types + Evidence storage types + Authorization management types (e.g., data sharing)
9. Operations and maintenance

Development Trends:

1. In terms of architecture, the integration of public chains and consortium chains continues to evolve.
2. In terms of deployment, blockchain as a service accelerates application implementation.
3. In terms of performance, the demand for cross-chain and high-performance solutions is becoming increasingly prominent.
   1. Cross-chain: == Current mainstream cross-chain technologies include: Notary schemes, Sidechains/relays, and Hash-locking ==
   2. High performance
      1. Blockchain to DAG
      2. Changing consensus strategies
      3. Improving overall system throughput by enhancing horizontal scalability, represented by technologies such as sharding, sub-chains, and multi-channels.
4. In terms of consensus, consensus mechanisms are evolving from singular to hybrid approaches.
5. In terms of contracts, pluggability, usability, and security are becoming key development focuses.

Industry Status​

1. Countries are competing to establish a leading position in the blockchain industry.
2. The integration of blockchain with the real economy has become the main theme.
3. Blockchain technology innovation is becoming increasingly active.
4. == The construction of a blockchain standard system is accelerating ==

Opportunities and Challenges​

Opportunities

Challenges:

1. Potential risks in technology maturity.
2. Unclear application scenario models.
3. Relative scarcity of industry professionals.
4. Relevant laws and regulations need improvement.

Development Measures and Recommendations​

1. Guide society to have an objective and rational understanding.
2. Strengthen research on core key technologies.
3. Promote deep integration with the real economy.
4. Improve the policy environment for blockchain development.

Disclaimer: I highly doubt those guys who succeed only by giving advice on how to succeed. ==A bird's flight does not need the guidance of ornithologists.== Anyhow, food for thought that these are what American women think helpful to their promotions.

1. claim your achievements often
2. realize that others will not notice and reward your contributions unless you say it
3. expertise is not the only criteria
4. leverage relationships
5. enlist allies from day one
6. put your career before your job
7. don't be a perfectionist
8. kill the disease to please
9. be relaxed and stretched out
10. be less emotional, speak less, and disclose less
11. move on effectively from failures
12. be undistracted

What are the challenges?​

Before TAO, use cache-aside pattern

Social graph data is stored in MySQL and cached in Memcached

3 problems:

1. list update operation in Memcached is inefficient. cannot append but update the whole list.
2. clients have to manage cache
3. Hard to offer ==read-after-write consistency==

To solve those problems, we have 3 goals:

• online data graph service that is efficiency at scale
• optimize for read (its read-to-write ratio is 500:1)
  • low read latency
  • high read availability (eventual consistency)
• timeliness of writes (read-after-write)

Data Model​

• Objects (e.g. user, location, comment) with unique IDs
• Associations (e.g. tagged, like, author) between two IDs
• Both have key-value data as well as a time field

Solutions: TAO​

1. Efficiency at scale and reduce read latency

   • graph-specific caching
   • a standalone cache layer between the stateless service layer and the DB layer (aka Functional Decomposition)
   • subdivide data centers (aka Horizontal Data Partitioning)

2. Write timeliness

   • write-through cache
   • follower/leader cache to solve thundering herd problem
   • async replication

3. Read availability

   • Read Failover to alternate data sources

TAO's Architecture​

• MySQL databases → durability
• Leader cache → coordinates writes to each object
• Follower caches → serve reads but not writes. forward all writes to leader.

Read failover