2 篇博文 含有标签「Machine Learning」

Reasoning is pivotal for advancing LLM capabilities

Introduction​

• Expectations for AI: Solving complex math problems, discovering scientific theories, achieving AGI.
• Baseline Expectation: AI should emulate human-like learning with few examples.

Key Concepts​

• What is Missing in ML?
  • Reasoning: The ability to logically derive answers from minimal examples.

Toy Problem: Last Letter Concatenation​

• Problem

  : Extract the last letters of words and concatenate them.

  • Example: "Elon Musk" → "nk".

• Traditional ML: Requires significant labeled data.

• LLMs: Achieve 100% accuracy with one demonstration using reasoning.

Importance of Intermediate Steps​

• Humans solve problems through reasoning and intermediate steps.
• Example:
  • Input: "Elon Musk"
  • Reasoning: Last letter of "Elon" = "n", of "Musk" = "k".
  • Output: "nk".

Advancements in Reasoning Approaches​

1. Chain-of-Thought (CoT) Prompting
   • Breaking problems into logical steps.
   • Examples from math word problems demonstrate enhanced problem-solving accuracy.
2. Least-to-Most Prompting
   • Decomposing problems into easier sub-questions for gradual generalization.
3. Analogical Reasoning
   • Adapting solutions from related problems.
   • Example: Finding the area of a square by recalling distance formula logic.
4. Zero-Shot and Few-Shot CoT
   • Triggering reasoning without explicit examples.
5. Self-Consistency in Decoding
   • Sampling multiple responses to improve step-by-step reasoning accuracy.

Limitations​

• Distraction by Irrelevant Context
  • Adding irrelevant details significantly lowers performance.
  • Solution: Explicitly instructing the model to ignore distractions.
• Challenges in Self-Correction
  • LLMs can fail to self-correct errors, sometimes worsening correct answers.
  • Oracle feedback is essential for effective corrections.
• Premise Order Matters
  • Performance drops with re-ordered problem premises, emphasizing logical progression.

Practical Implications​

• Intermediate reasoning steps are crucial for solving serial problems.
• Techniques like self-debugging with unit tests are promising for future improvements.

Future Directions​

1. Defining the right problem is critical for progress.
2. Solving reasoning limitations by developing models that autonomously address these issues.

生成式 AI 的关键趋势​

• 机器学习进步重新定义计算能力
• 计算和硬件需求的演变
• 扩展（计算、数据、模型规模）改善结果

AI 能力的进展​

• 图像识别
  • 示例：“豹”分类，90.88% 准确率（ImageNet）
  • AlexNet 初始性能：63.3%
• 语音识别
  • 在 LibriSpeech test-other 数据集上的性能提升

Transformers 和基础模型​

• 关键技术
  • 自回归训练
  • 使用数万亿标记进行预训练
  • 示例：“猫坐在垫子上”
• 优化
  • 监督微调 (SFT)
  • 来自人类反馈的强化学习 (RLHF)

Gemini 模型​

• 项目启动于 2023 年 2 月
• Gemini 1.0 发布：2023 年 12 月
• Gemini 1.5 发布：2024 年 2 月
• 特点
  • 跨文本、图像和视频的多模态推理
  • 长上下文能力（最多 1000 万标记）
  • 降低幻觉率

企业 AI 趋势​

1. 随着数据需求的减少，加速 AI 开发
2. 从单一模态系统向多模态系统过渡
3. 从密集模型架构向稀疏模型架构转变
4. 可扩展和灵活平台的重要性
5. API 成本下降
6. LLMs 和搜索的集成

定制化和效率​

• 技术
  • 微调和参数高效调优（例如，LoRA）
  • 蒸馏以优化性能和延迟
• 挑战
  • 在部署中平衡成本、延迟和性能
• 函数调用
  • 集成 API、数据库和外部系统
  • 应用：数据检索、工作流程、客户支持

解决限制​

• 问题
  • 冻结的训练数据导致知识过时
  • 高幻觉率
  • 不一致的结构化输出
• 解决方案
  • 检索增强生成 (RAG) 框架
  • 以私有、新鲜和权威数据为基础
  • 带有引用的结构化输出

生成式 AI 的未来​

• 增强的多模态推理和扩展的上下文能力
• 优化以降低成本和提高可扩展性
• 改进输出的基础性和事实准确性