3 posts tagged with "vision"

Your vision model benchmarks 90%+ on your eval suite. Then real users upload photos of physical documents, screenshots from low-DPI monitors, and scanned PDFs that have been round-tripped through three fax machines. Accuracy craters. The model "works" — it returns coherent responses — but the responses are wrong in ways that are hard to catch without knowing the ground truth. You file it under "model limitations" and move on.

The model probably isn't the problem. The input pipeline is.

Most teams building with vision LLMs spend enormous effort on prompt engineering and model selection, and nearly zero effort on the preprocessing that happens before the image ever reaches the model. That asymmetry is where production quality goes to die. The preprocessing decisions nobody documents are also the ones responsible for the biggest silent accuracy drops in production multimodal systems.

Adding vision to an LLM application looks deceptively simple. You swap a text model for a multimodal one, pass in an image alongside your prompt, and the demo works brilliantly. Then you push to production and discover that half your invoices get the total wrong, tables in PDFs lose their structure, and low-quality scans produce confident hallucinations. The debugging is harder than anything you faced with text-only systems, because the failures are visual and the LLM will not tell you it cannot see clearly.

This post covers what actually goes wrong when you move multimodal LLM inputs from prototype to production, and the architectural decisions that prevent those failures.

Why do some people want to make money by becoming entrepreneurs? Because they want to beat the market—achieving returns that exceed the market at a cost lower than the market—meaning they want to obtain a profit margin higher than the market. The price exceeding the market comes from the scarcity/uniqueness of a product or service; to achieve uniqueness, one must innovate. Therefore, to become an entrepreneur, one must at least be an innovator.

Most companies succeed because they know how to continuously draw inspiration from the right things and consistently generate new ideas. How can one identify the most suitable sources of innovation to outperform competitors and stand out in the industry?

Seven Sources of Innovation​

• Internal

  • Unexpected occurrences: For example, when there was a sudden surge in the purchase of home appliances, Macy's limited sales while Bloomingdale's seized the opportunity to expand its appliance department, thereby increasing profits.

  • Changes in the market and industry: For instance, when the automotive market globalized, Volvo also followed suit, performing better than Citroën, which did not globalize quickly.

  • Weak links in processes: Pharmaceutical sales representative William Connor noticed a troublesome aspect of eye surgery: hemorrhage of the eye ligament. He suggested using enzymes to dissolve the ligament instead of cutting it, significantly reducing surgical risks, and this innovation was widely accepted in the field of ophthalmology. This innovation addressing a shortcoming brought his company substantial profits.

  • The gap between reality and perception (Is TK also a disciple of Drucker?): For example, early on, ferry freight mistakenly believed that the key to reducing time was to increase sailing speed, but in reality, this would lead to skyrocketing costs; the key issue was actually to reduce the time the ship was idle in port.

• External: For example, politics, academia, science

  • Changes in social concepts: The growing enthusiasm for environmental protection and high technology has made the electric vehicle market thrive.
  • Changes in demographic structure: For instance, the increase in digital natives in China and the demand for online communities gave rise to Bilibili.
  • Hybridization of new knowledge: For example, computers are a hybrid product of mathematics, electronics, and programming technology developed over hundreds of years.

Both Small and Large Companies Need Innovation​

A newly established company needs specific goals and plans, as detailed in The Five Stages of Company Building.

In the early stages of entrepreneurship, entrepreneurs should try different fields to find the right market. It is very likely that you will ultimately succeed in a field you never considered. The second step is to establish the correct financial focus. Ensuring that the company has sufficient funds to address issues when they arise is extremely important. The final step is to build a trustworthy management team for the company. This team should be established before the company’s team grows.

Not only small businesses need reform and innovation, but large industries also need fresh blood. In the initial stages, they should standardize the rules for innovating and phasing out the old within the company. Secondly, the newly innovated projects should be managed by new leaders. Lastly, companies should establish reward mechanisms to help improve employee performance and effectively review the impact of innovations.

Four Innovation Strategies​

All In (Fustest with the mostest)​

A wise entrepreneur should aim to become a pioneer in their industry, putting everything on the line to lead the way. Hoffmann-La Roche had a small chemical company, but he cleverly identified the business opportunity in the vitamin industry. Therefore, to produce and sell vitamins, he invested a large sum of money and hired many experts. Although it sounded very risky, this "gamble" ultimately paid off, and he remained a leader in the vitamin industry for 60 years.

Hit Them Where They Ain’t​

Identifying vulnerabilities that competitors overlook is not easy, but there are two ways to achieve this. The first is to imitate competitors' ideas using newer and more appealing methods. For example, IBM imitated the ideas of competitor ENIAC and added more innovative concepts, ultimately profiting from it. Additionally, some companies can win by targeting their opponents' weaknesses, which is especially effective against complacent large companies.

Ecological Niches​

This originally is a biological concept: Ecological niche refers to the environment a species inhabits and its lifestyle habits. Each species has its unique ecological niche, distinguishing it from other species.

A company that specializes in an irreplaceable field is more likely to succeed. A good example is the enzymes developed by William Connors. These enzymes later became a crucial step in cataract surgery. However, it is worth noting that this company could also lose its absolute advantage in the industry if competitors develop substitute drugs.

Changing Values and Characteristics​

To increase demand for your product, you do not necessarily need to change the product itself. Instead, finding a method that better aligns with consumer interests may be more important. Entrepreneurs should understand what makes consumers willing to pay. For example, Gillette's strategy of offering razors for free while charging for blades was based on the company's realization that consumers were unwilling to pay more for blades than the razor itself.