One post tagged with "Safety"

Anthropic’s History​

• Founded: 2021 as a Public Benefit Corporation (PBC).
• Milestones:
  • 2022: Claude 1 completed.
  • 2023: Claude 1 released, Claude 2 launched.
  • 2024: Claude 3 launched.
  • 2025: Advances in interpretability and AI safety:
    • Mathematical framework for constitutional AI.
    • Sleeper agents and toy models of superposition.

Responsible Scaling Policy (RSP)​

• Definition: A framework to ensure safe scaling of AI capabilities.
• Goals:
  • Provide structure for safety decisions.
  • Ensure public accountability.
  • Iterate on safe decisions.
  • Serve as a template for policymakers.
• AI Safety Levels (ASL): Modeled after biosafety levels (BSL) for handling dangerous biological materials, aligning safety, security, and operational standards with a model’s catastrophic risk potential.
  • ASL-1: Smaller Models: No meaningful catastrophic risk (e.g., 2018 LLMs, chess-playing AIs).
  • ASL-2: Present Large Models: Early signs of dangerous capabilities (e.g., instructions for bioweapons with limited reliability).
  • ASL-3: Higher Risk Models: Models with significant catastrophic misuse potential or low-level autonomy.
  • ASL-4 and higher: Speculative Models: Future systems involving qualitative escalations in catastrophic risk or autonomy.
• Implementation:
  • Safety challenges and methods.
  • Case study: computer use.

Measuring Capabilities​

• Challenges: Benchmarks become obsolete.
• Examples:
  • Task completion time relative to humans: Claude 3.5 completes tasks in seconds compared to human developers’ 30 minutes.
  • Benchmarks:
    • SWE-bench: Assesses real-world software engineering tasks.
    • Aider’s benchmarks: Code editing and refactoring.
• Results:
  • Claude 3.5 Sonnet outperforms OpenAI o1 across key benchmarks.
  • Faster and cheaper: $3/Mtok input vs. OpenAI o1 at $15/Mtok input.

Claude 3.5 Sonnet Highlights​

• Agentic Coding and Game Development: Designed for efficiency and accuracy in real-world scenarios.
• Computer Use Demos:
  • Coding: Demonstrated advanced code generation and integration.
  • Operations: Showcased operational tasks with safety considerations.

AI Safety Measures​

• Focus Areas:
  • Scaling governance.
  • Capability measurement.
  • Collaboration with academia.
• Practical Safety:
  • ASL standard implementation.
  • Deployment safeguards.
  • Lessons learned in year one.

Future Directions​

• Scaling and governance improvements.
• Enhanced benchmarks and academic partnerships.
• Addressing interpretability and sleeper agent risks.