39 posts tagged with "performance"

Most engineers building on LLMs treat latency as a single dial. They tune something — a batch size, a quantization level, an instance type — observe whether "it got faster," and call it done. This works until you hit production and discover that your p50 TTFT looks fine while your p99 is over 3 seconds, or that the optimization that doubled your throughput somehow made individual users feel the system got slower.

TTFT and throughput are not two ends of the same slider. They are caused by fundamentally different physics, degraded by different bottlenecks, and fixed by different techniques. Treating them as interchangeable is the root cause of most LLM inference incidents I've seen in production.

Most LLM latency advice falls into one of two failure modes: it focuses on the wrong metric, or it recommends optimizations that are too hardware-specific to apply unless you're running your own inference cluster. If you're building on top of a hosted API or a managed inference provider, a lot of that advice is noise.

This post focuses on what actually moves the needle — techniques that apply whether you control the stack or not, grounded in production data rather than benchmark lab conditions.

Teamwork is crucial. Even geniuses like Turing need help from others to crack the Enigma. So, what are the key factors that enable a team to succeed? People naturally believe it is the abilities and levels of individual members, but the reality might surprise you.

At the beginning of The Culture Code, the author describes an interesting competition among kindergarten children, business school students, and lawyers: participants had to build the tallest structure possible using raw spaghetti, tape, string, and marshmallows. The competition ended with the kindergarten children winning. Why did the seemingly least capable group manage to defeat the others? Upon reviewing the competition, we found that business school students typically analyzed the problem first, discussed the right strategy, and then quietly established a hierarchy; whereas the kindergarten children simply started building and experimenting with different approaches.

A strong team culture emphasizes communication among team members rather than individual skills. Such a culture maximizes overall performance. To foster a positive team culture that enhances collective performance, there are three key skills.

1. Create a Safe Work Environment​

People are more likely to unleash their full potential in a familiar environment, so creating a safe space is crucial. The sense of safety within a team comes from familiarity and connection among its members. If you want to cultivate a safe work environment, it is essential to learn to listen and let others know they are heard. When people know that what they say is being listened to and valued, they feel secure. You can provide appropriate feedback while listening, which enhances interaction and makes people feel needed.

2. Be Vulnerable to Build Trust​

Although it may seem counterintuitive, showing vulnerability can indeed enhance team performance. We often observe the behaviors of those around us and learn by imitation. Admitting your weaknesses and mistakes to team members shows that they can do the same. This helps to strengthen internal trust among the team.

At the same time, displaying your shortcomings expresses an expectation for collaboration. When you show that you rely on others for help, they can also comfortably acknowledge their need for assistance. Over time, everyone understands that they shouldn’t bear everything alone, naturally fostering a sense of unity within the team.

3. Establish Common Goals and Reinforce Them​

A steadfast pursuit of common goals is key to good team performance. A team's common goal refers to the beliefs and values that motivate the actions of its members. This common goal clarifies the team's self-identity and communicates it to the outside world. Psychologist Gabriele Oettingen has demonstrated through several studies that communicating common goals helps unite members and makes achieving those goals easier.

To deepen members' impressions, repetition is essential. To express things more clearly, repeating them ten or even a hundred times is worthwhile. You can repeatedly convey the company's mission in meetings or turn the goals into catchy slogans.