Key Trends in Generative AI​

• Machine learning advancements redefine computational capabilities
• Evolving computation and hardware requirements
• Scaling (compute, data, model size) improves results

Progress in AI Capabilities​

• Image Recognition
  • Example: “Leopard” classification, 90.88% accuracy (ImageNet)
  • AlexNet initial performance: 63.3%
• Speech Recognition
  • Improved performance on LibriSpeech test-other dataset

Transformers and Foundation Models​

• Key techniques
  • Autoregressive training
  • Pre-training with trillions of tokens
  • Example: "The cat sat on the mat"
• Optimization
  • Supervised Fine-Tuning (SFT)
  • Reinforcement Learning from Human Feedback (RLHF)

Gemini Models​

• Project started February 2023
• Gemini 1.0 release: December 2023
• Gemini 1.5 release: February 2024
• Features
  • Multimodal reasoning across text, image, and video
  • Long context capabilities (up to 10M tokens)
  • Reduced hallucination rates

Enterprise AI Trends​

1. Accelerating AI development as data requirements decrease
2. Transition from single modality to multimodal systems
3. Shift from dense to sparse model architectures
4. Importance of scalable and flexible platforms
5. Declining API costs
6. Integration of LLMs and search

Customization and Efficiency​

• Techniques
  • Fine-tuning and parameter-efficient tuning (e.g., LoRA)
  • Distillation for performance and latency optimization
• Challenges
  • Balancing cost, latency, and performance in deployment
• Function Calling
  • Integrates APIs, databases, and external systems
  • Applications: data retrieval, workflows, customer support

Addressing Limitations​

• Issues
  • Frozen training data causing outdated knowledge
  • High hallucination rates
  • Inconsistent structured outputs
• Solutions
  • Retrieval-Augment-Generation (RAG) frameworks
  • Grounding in private, fresh, and authoritative data
  • Structured outputs with citations

Future of Generative AI​

• Enhanced multimodal reasoning and extended context capabilities
• Optimization to reduce costs and improve scalability
• Improved grounding and factual accuracy in outputs

Reasoning is pivotal for advancing LLM capabilities

Introduction​

• Expectations for AI: Solving complex math problems, discovering scientific theories, achieving AGI.
• Baseline Expectation: AI should emulate human-like learning with few examples.

Key Concepts​

• What is Missing in ML?
  • Reasoning: The ability to logically derive answers from minimal examples.

Toy Problem: Last Letter Concatenation​

• Problem

  : Extract the last letters of words and concatenate them.

  • Example: "Elon Musk" → "nk".

• Traditional ML: Requires significant labeled data.

• LLMs: Achieve 100% accuracy with one demonstration using reasoning.

Importance of Intermediate Steps​

• Humans solve problems through reasoning and intermediate steps.
• Example:
  • Input: "Elon Musk"
  • Reasoning: Last letter of "Elon" = "n", of "Musk" = "k".
  • Output: "nk".

Advancements in Reasoning Approaches​

1. Chain-of-Thought (CoT) Prompting
   • Breaking problems into logical steps.
   • Examples from math word problems demonstrate enhanced problem-solving accuracy.
2. Least-to-Most Prompting
   • Decomposing problems into easier sub-questions for gradual generalization.
3. Analogical Reasoning
   • Adapting solutions from related problems.
   • Example: Finding the area of a square by recalling distance formula logic.
4. Zero-Shot and Few-Shot CoT
   • Triggering reasoning without explicit examples.
5. Self-Consistency in Decoding
   • Sampling multiple responses to improve step-by-step reasoning accuracy.

Limitations​

• Distraction by Irrelevant Context
  • Adding irrelevant details significantly lowers performance.
  • Solution: Explicitly instructing the model to ignore distractions.
• Challenges in Self-Correction
  • LLMs can fail to self-correct errors, sometimes worsening correct answers.
  • Oracle feedback is essential for effective corrections.
• Premise Order Matters
  • Performance drops with re-ordered problem premises, emphasizing logical progression.

Practical Implications​

• Intermediate reasoning steps are crucial for solving serial problems.
• Techniques like self-debugging with unit tests are promising for future improvements.

Future Directions​

1. Defining the right problem is critical for progress.
2. Solving reasoning limitations by developing models that autonomously address these issues.

Anthropic’s History​

• Founded: 2021 as a Public Benefit Corporation (PBC).
• Milestones:
  • 2022: Claude 1 completed.
  • 2023: Claude 1 released, Claude 2 launched.
  • 2024: Claude 3 launched.
  • 2025: Advances in interpretability and AI safety:
    • Mathematical framework for constitutional AI.
    • Sleeper agents and toy models of superposition.

Responsible Scaling Policy (RSP)​

• Definition: A framework to ensure safe scaling of AI capabilities.
• Goals:
  • Provide structure for safety decisions.
  • Ensure public accountability.
  • Iterate on safe decisions.
  • Serve as a template for policymakers.
• AI Safety Levels (ASL): Modeled after biosafety levels (BSL) for handling dangerous biological materials, aligning safety, security, and operational standards with a model’s catastrophic risk potential.
  • ASL-1: Smaller Models: No meaningful catastrophic risk (e.g., 2018 LLMs, chess-playing AIs).
  • ASL-2: Present Large Models: Early signs of dangerous capabilities (e.g., instructions for bioweapons with limited reliability).
  • ASL-3: Higher Risk Models: Models with significant catastrophic misuse potential or low-level autonomy.
  • ASL-4 and higher: Speculative Models: Future systems involving qualitative escalations in catastrophic risk or autonomy.
• Implementation:
  • Safety challenges and methods.
  • Case study: computer use.

Measuring Capabilities​

• Challenges: Benchmarks become obsolete.
• Examples:
  • Task completion time relative to humans: Claude 3.5 completes tasks in seconds compared to human developers’ 30 minutes.
  • Benchmarks:
    • SWE-bench: Assesses real-world software engineering tasks.
    • Aider’s benchmarks: Code editing and refactoring.
• Results:
  • Claude 3.5 Sonnet outperforms OpenAI o1 across key benchmarks.
  • Faster and cheaper: $3/Mtok input vs. OpenAI o1 at $15/Mtok input.

Claude 3.5 Sonnet Highlights​

• Agentic Coding and Game Development: Designed for efficiency and accuracy in real-world scenarios.
• Computer Use Demos:
  • Coding: Demonstrated advanced code generation and integration.
  • Operations: Showcased operational tasks with safety considerations.

AI Safety Measures​

• Focus Areas:
  • Scaling governance.
  • Capability measurement.
  • Collaboration with academia.
• Practical Safety:
  • ASL standard implementation.
  • Deployment safeguards.
  • Lessons learned in year one.

Future Directions​

• Scaling and governance improvements.
• Enhanced benchmarks and academic partnerships.
• Addressing interpretability and sleeper agent risks.

Key Trends​

• Skyrocketing Capabilities: Rapid advancements in LLMs since 2018.
• Declining Access: Shift from open paper, code, and weights to API-only models, limiting experimentation and research.

Why Access Matters​

• Access drives innovation:
  • 1990s: Digital text enabled statistical NLP.
  • 2010s: GPUs and crowdsourcing fueled deep learning and large datasets.
• Levels of access define research opportunities:
  • API: Like a cognitive scientist, measure behavior (prompt-response systems).
  • Open-Weight: Like a neuroscientist, probe internal activations for interpretability and fine-tuning.
  • Open-Source: Like a computer scientist, control and question every part of the system.

Levels of Access for Foundation Models​

1. API Access

   • Acts as a universal function (e.g., summarize, verify, generate).
   • Enables problem-solving agents (e.g., cybersecurity tools, social simulations).
   • Challenges: Deprecation and limited reproducibility.

2. Open-Weight Access

   • Enables interpretability, distillation, fine-tuning, and reproducibility.
   • Prominent models: Llama, Mistral.
   • Challenges:
     • Testing model independence and functional changes from weight modifications.
     • Blueprint constraints of pre-existing models.

3. Open-Source Access

   • Embodies creativity, transparency, and collaboration.
   • Examples: GPT-J, GPT-NeoX, StarCoder.
   • Performance gap persists compared to closed models due to compute and data limitations.

Key Challenges and Opportunities​

• Open-Source Barriers:
  • Legal restrictions on releasing web-derived training data.
  • Significant compute requirements for retraining.
• Scaling Compute:
  • Pooling idle GPUs.
  • Crowdsourced efforts like Big Science.
• Emergent Research Questions:
  • How do architecture and data shape behavior?
  • Can scaling laws predict performance at larger scales?

Reflections​

• Most research occurs within API and fixed-weight confines, limiting exploration.
• Open-weight models offer immense value for interpretability and experimentation.
• Open-source efforts require collective funding and infrastructure support.

Final Takeaway​

Access shapes the trajectory of innovation in foundation models. To unlock their full potential, researchers must question data, architectures, and algorithms while exploring new models of collaboration and resource pooling.

Key Challenges with LLMs​

• Difficulty with tasks requiring complex planning (e.g., travel itineraries, meeting schedules).
• Performance declines with increasing task complexity (e.g., more cities, people, or constraints).

Three Proposed Solutions​

1. Scaling Law
   • Increase data, compute, and model size.
   • Limitation: High costs and diminishing returns for reasoning/planning tasks.
2. Hybrid Systems
   • Combine deep learning models with symbolic solvers. Symbolic reasoning refers to the process of solving problems and making decisions using explicit symbols, rules, and logic. It is a method where reasoning is based on clearly defined relationships and representations, often following formal logic or mathematical principles.
   • Approaches:
     • End-to-End Integration: Unified deep model and symbolic system.
     • Data Augmentation: Neural models provide structured data for solvers.
     • Tool Use: LLMs act as interfaces for external solvers.
   • Notable Examples:
     • MILP Solvers: For travel planning with constraints.
     • Searchformer: Transformers trained to emulate A* search.
     • DualFormer: Switches dynamically between fast (heuristic) and slow (deliberative) reasoning.
     • SurCo: Combines combinatorial optimization with latent space representations.
3. Emerging Symbolic Structures
   • Exploration of symbolic reasoning emerging in neural networks.
   • Findings:
     • Neural networks exhibit Fourier-like patterns in arithmetic tasks.
     • Gradient descent produces solutions aligned with algebraic constructs.
     • Emergent ring homomorphisms and symbolic efficiency in complex tasks.

Research Implications​

• Neural networks naturally learn symbolic abstractions, offering potential for improved reasoning.
• Hybrid systems might represent the optimal balance between adaptability (neural) and precision (symbolic).
• Advanced algebraic techniques could eventually replace gradient descent.

Overall Takeaway​

The future of decision-making AI lies in leveraging both neural adaptability and symbolic rigor. Hybrid approaches appear most promising for solving tasks requiring both perception and structured reasoning.

Key Themes and Context​

Enterprise Workflows

• Automation levels range from scripted workflows (minimal variation) to agentic workflows (adaptive and dynamic).
• Enterprise environments, such as those supported by ServiceNow, involve complex, repetitive tasks like IT management, CRM updates, and scheduling.
• The adoption of LLM-powered agents (e.g., API agents and Web agents) transforms these workflows by leveraging capabilities like multimodal observations and dynamic actions.

LLM Agents for Enterprise Workflows

• API Agents
  • Utilize structured API calls for efficiency.
  • Pros: Low latency, structured inputs.
  • Cons: Depend on predefined APIs, limited adaptability.
• Web Agents
  • Simulate human actions on web interfaces.
  • Pros: Greater flexibility; can interact with dynamic UIs.
  • Cons: High latency, error-prone.

WorkArena Framework

• Benchmarks designed for realistic enterprise workflows.
• Tasks range from IT inventory management to budget allocation and employee offboarding.
• Supported by BrowserGym and AgentLab for testing and evaluation in simulated environments.

Technical Frameworks​

Agent Architectures

• TapeAgents Framework

  • Represents agents as resumable modular state machines.
  • Features structured logs (the "tape") for actions, thoughts, and outcomes.
  • Facilitates optimization (e.g., fine-tuning from teacher-to-student agents).

• WorkArena++

  • Extends WorkArena with more compositional and challenging tasks.
  • Evaluates agents on capabilities like long-term planning and multimodal data integration.

Benchmarks

• WorkArena: ~20k unique enterprise task instances.
• WorkArena++: Focused on compositional workflows and data-driven reasoning.
• Other tools: MiniWoB, WebLINX, VisualWebArena.

Evaluation Metrics

• GREADTH (Grounded, Responsive, Accurate, Disciplined, Transparent, Helpful):
  • Prioritizes real-world agent performance metrics.
• Task-Specific Success Rates:
  • For example, form-filling assistants evaluated at 300x lower cost than GPT-4 through fine-tuned students.

Challenges for Agents in Workflows​

• Context Understanding
  • Enterprise tasks require understanding deep hierarchies of information (e.g., dashboards, KBs).
  • Sparse rewards in benchmarks complicate learning.
• Long-Term Planning
  • Subgoal decomposition and multi-step task execution remain difficult.
• Safety and Alignment
  • Risks from malicious inputs (e.g., adversarial prompts, hidden text).
• Cost and Efficiency
  • Shrinking context windows and modular architectures are key to reducing compute costs.

Future Directions​

Augmentation Models

• Centaur Framework:
  • Separates AI from human tasks (e.g., content gathering by AI, final editing by humans).
• Cyborg Framework:
  • Promotes tight collaboration between AI and humans.

Unified Evaluation

• Calls for a meta-benchmark to consolidate evaluation protocols across platforms (e.g., WebLINX, WorkArena).

Advancements in Agent Optimization

• Leveraging RL-inspired techniques for fine-tuning.
• Modular learning frameworks to improve generalizability.

Opportunities in Knowledge Work​

• Automation of repetitive, low-value tasks (e.g., scheduling, report generation).
• Integration of multimodal agents into enterprise environments to support decision-making and strategic tasks.
• Enhanced productivity through human-AI collaboration models.

This synthesis connects the theoretical and practical elements of enterprise workflow agents, showcasing their transformative potential while addressing current limitations.

Software’s Impact​

• Software is transforming industries, as predicted by Marc Andreessen (2011).
• Potential impact of enabling everyone to write software to achieve their goals.

Software Development Workflow​

• Time allocation:
  • 17% Coding
  • 36% Bugfixing
  • 10% Testing
  • 8% Documentation/Reviews
  • 14% Communication
  • 15% Other tasks

Development Tools​

• Copilots:
  • Synchronous support for writing code (e.g., GitHub Copilot).
• Development Agents:
  • Autonomous tools for coding (e.g., SWE-Agent, Aider) and broader tasks (e.g., Devin, OpenHands).

Challenges in Coding Agents​

• Defining the environment.
• Designing observation/action spaces.
• File localization and code generation.
• Planning, error recovery, and ensuring safety.

Software Development Environments​

• Actual Environments:
  • Source repositories, task management software, office tools, communication tools.
• Testing Environments:
  • Focused on coding, sometimes includes browsing tasks.

Metrics and Datasets​

• Pass@K (Chen et al., 2021): Measures success rates of generated code passing unit tests.
• Semantic Overlap Metrics:
  • BLEU, CodeBLEU, CodeBERTScore.
• Key Datasets:
  • HumanEval, ARCADE, SWEBench, Design2Code.

Solutions for File Localization​

1. User Input: Relies on experienced users to specify files.
2. Search Tools: Integrated search capabilities (e.g., SWE-Agent).
3. Repository Mapping: Prebuilt maps (e.g., Aider repomap).
4. Retrieval-Augmented Generation: Combine retrieved code and LMs.

Planning and Recovery​

• Hard-coded Processes: Predefined steps for file localization, patch generation, etc.
• LLM-Generated Plans: Use LMs for planning and execution (e.g., CodeR).
• Revisiting Errors: Automated fixes based on error messages (e.g., InterCode).

Safety Measures​

1. Sandboxing: Limit execution environments (e.g., Docker).
2. Credentialing: Principle of least privilege.
3. Post-hoc Auditing: Security analysis using LMs and other tools.

Future Directions​

• Enhance agentic training methods.
• Expand human-in-the-loop approaches.
• Address broader software tasks beyond coding.

Resources​

• OpenHands Repository: GitHub

Introduction​

• Two kinds of AI applications:

  • Generative AI: Creates content like text and images.
  • Agentic AI: Performs complex tasks autonomously. This is the future.

• Key Question: How can developers make these systems easier to build?

Agentic AI Frameworks​

• Examples:

  • Applications include personal assistants, autonomous robots, gaming agents, web/software agents, science, healthcare, and supply chains.

• Core Benefits:

  • User-Friendly: Natural and intuitive interactions with minimal input.
  • High Capability: Handles complex tasks efficiently.
  • Programmability: Modular and maintainable, encouraging experimentation.

• Design Principles:

  • Unified abstractions integrating models, tools, and human interaction.
  • Support for dynamic workflows, collaboration, and automation.

AutoGen Framework​

https://github.com/microsoft/autogen

• Purpose: A framework for building agentic AI applications.

• Key Features:

  • Conversable and Customizable Agents: Simplifies building applications with natural language interactions.
  • Nested Chat: Handles complex workflows like content creation and reasoning-intensive tasks.
  • Group Chat: Supports collaborative task-solving with multiple agents.

• History:

  • Started in FLAML (2022), became standalone (2023), with over 200K monthly downloads and widespread adoption.

Applications and Examples​

• Advanced Reflection:
  • Two-agent systems for collaborative refinement of tasks like blog writing.
• Gaming and Strategy:
  • Conversational Chess, where agents simulate strategic reasoning.
• Enterprise and Research:
  • Applications in supply chains, healthcare, and scientific discovery, such as ChemCrow for discovering novel compounds.

Core Components of AutoGen​

• Agentic Programming:
  • Divides tasks into manageable steps for easier scaling and validation.
• Multi-Agent Orchestration:
  • Supports dynamic workflows with centralized or decentralized setups.
• Agentic Design Patterns:
  • Covers reasoning, planning, tool integration, and memory management.

Challenges in Agent Design​

• System Design:
  • Optimizing multi-agent systems for reasoning, planning, and diverse applications.
• Performance:
  • Balancing quality, cost, and scalability while maintaining resilience.
• Human-AI Collaboration:
  • Designing systems for safe, effective human interaction.

Open Questions and Future Directions​

• Multi-Agent Topologies:
  • Efficiently balancing centralized and decentralized systems.
• Teaching and Optimization:
  • Enabling agents to learn autonomously using tools like AgentOptimizer.
• Expanding Applications:
  • Exploring new domains such as software engineering and cross-modal systems.

Trajectory and potential of LLM agents

Introduction​

• Definition of Agents: Intelligent systems interacting with environments (physical, digital, or human).
• Evolution: From symbolic AI agents like ELIZA(1966) to modern LLM-based reasoning agents.

Core Concepts​

1. Agent Types:
   • Text Agents: Rule-based systems like ELIZA(1966), limited in scope.
   • LLM Agents: Utilize large language models for versatile text-based interaction.
   • Reasoning Agents: Combine reasoning and acting, enabling decision-making across domains.
2. Agent Goals:
   • Perform tasks like question answering (QA), game-solving, or real-world automation.
   • Balance reasoning (internal actions) and acting (external feedback).

Key Developments in LLM Agents​

1. Reasoning Approaches:
   • Chain-of-Thought (CoT): Step-by-step reasoning to improve accuracy.
   • ReAct Paradigm: Integrates reasoning with actions for systematic exploration and feedback.
2. Technological Milestones:
   • Zero-shot and Few-shot Learning: Achieving generality with minimal examples.
   • Memory Integration: Combining short-term (context-based) and long-term memory for persistent learning.
3. Tools and Applications:
   • Code Augmentation: Enhancing computational reasoning through programmatic methods.
   • Retrieval-Augmented Generation (RAG): Leveraging external knowledge sources like APIs or search engines.
   • Complex Task Automation: Embodied reasoning in robotics and chemistry, exemplified by ChemCrow.

Limitations​

• Practical Challenges:
  • Difficulty in handling real-world environments (e.g., decision-making with incomplete data).
  • Vulnerability to irrelevant or adversarial context.
• Scalability Issues:
  • Real-world robotics vs. digital simulation trade-offs.
  • High costs of fine-tuning and data collection in specific domains.

Research Directions​

• Unified Solutions: Simplifying diverse tasks into generalizable frameworks (e.g., ReAct for exploration and decision-making).
• Advanced Memory Architectures: Moving from append-only logs to adaptive, writeable long-term memory systems.
• Collaboration with Humans: Focusing on augmenting human creativity and problem-solving capabilities.

Future Outlook​

• Emerging Benchmarks:
  • SWE-Bench for software engineering tasks.
  • FireAct for fine-tuning LLM agents in dynamic environments.
• Broader Impacts:
  • Enhanced digital automation.
  • Scalable solutions for complex problem-solving in domains like software engineering, scientific discovery, and web automation.

The story of TianPan.co mirrors the evolution of web publishing itself - from simple HTML pages to today's AI-augmented content platforms. As we launch version 3, I want to share how we're reimagining what a modern publishing platform can be in the age of AI.

The Journey: From WordPress to AI-Native​

Like many technical blogs, TianPan.co started humbly in 2009 as a WordPress site on a free VPS. The early days were simple: write, publish, repeat. But as technology evolved, so did our needs. Version 1 moved to Octopress and GitHub, embracing the developer-friendly approach of treating content as code. Version 2 brought modern web technologies with GraphQL, server-side rendering, and a React Native mobile app.

But the landscape has changed dramatically. AI isn't just a buzzword - it's transforming how we create, organize, and share knowledge. This realization led to Version 3, built around a radical idea: what if we designed a publishing system with AI at its core, not just as an add-on?

The Architecture of an AI-Native Platform​

Version 3 breaks from traditional blogging platforms in several fundamental ways:

1. Content as Data: Every piece of content is stored as markdown, making it instantly processable by AI systems. This isn't just about machine readability - it's about enabling AI to become an active participant in the content lifecycle.

2. Distributed Publishing, Centralized Management: Content flows automatically from our central repository to multiple channels - Telegram, Discord, Twitter, and more. But unlike traditional multi-channel publishing, AI helps maintain consistency and optimize for each platform.

3. Infrastructure Evolution: We moved from a basic 1 CPU/1GB RAM setup to a more robust infrastructure, not just for reliability but to support AI-powered features like real-time content analysis and automated editing.

The technical architecture reflects this AI-first approach:

.
├── _inbox            # AI-monitored draft space
├── notes             # published English notes
├── notes-zh          # published Chinese notes
├── crm               # personal CRM
├── ledger            # my beancount.io ledger
├── packages
│   ├── chat-tianpan  # LlamaIndex-powered content interface
│   ├── website       # tianpan.co source code
│   ├── prompts       # AI system prompts
│   └── scripts       # AI processing pipeline

Beyond Publishing: An Integrated Knowledge System​

What makes Version 3 unique is how it integrates multiple knowledge streams:

• Personal CRM: Relationship management through AI-enhanced note-taking
• Financial Tracking: Integrated ledger system via beancount.io
• Multilingual Support: Automated translation and localization
• Interactive Learning: AI-powered chat interface for deep diving into content

The workflow is equally transformative:

1. Content creation starts in markdown
2. CI/CD pipelines trigger AI processing
3. Zapier integrations distribute across platforms
4. AI editors continuously suggest improvements through GitHub issues

Looking Forward: The Future of Technical Publishing​

This isn't just about building a better blog - it's about reimagining how we share technical knowledge in an AI-augmented world. The system is designed to evolve, with each component serving as a playground for experimenting with new AI capabilities.

What excites me most isn't just the technical architecture, but the possibilities it opens up. Could AI help surface connections between seemingly unrelated technical concepts? Could it help make complex technical content more accessible to broader audiences? Will it be possible to easily produce multimedia content in the future?

These are the questions we're exploring with TianPan.co v3. It's an experiment in using AI not just as a tool, but as a collaborative partner in creating and sharing knowledge.