Simple Reflex Agent

Governed by immediate perceptions, this agent operates on instinctual responses to stimuli. For instance, consider the basic act of blinking to clear the vision—it’s automatic, without deliberation.

Least complex, relying on hardcoded responses.

category: Misuse.

Could be easily misused due to their predictable nature.

Model-Based Agent

This agent remembers past states, like a chess player pondering future moves. It uses this information to assess and adjust to new situations, creating a dynamic model of the world as it changes.

More sophisticated, with some memory and anticipation.

category: Accidents.

Higher risk of accidents due to incorrect modeling.

Goal-Directed Agent

Purpose-driven, these agents are like marathon runners focused on the finish line. Each choice is strategic, designed to bring them closer to their objective, whether it’s a location, a state of affairs, or a numerical target.

Focused and strategic in its actions.

category: Accidents.

Could accidentally cause harm if the goal is misaligned.

Utility-Based Agent

Here, the agent is akin to a savvy investor seeking the highest return, not just completion of tasks. Their actions are calculated to maximize benefit, efficiency, and effectiveness.

Complex decision-making based on maximizing utility.

category: Structural.

Their complexity may lead to structural risks if their utility function is not aligned with human values.

Learning Agent

The most sophisticated of the group, learning agents adapt and evolve. They accumulate knowledge from experiences, refining their approach to challenges, much like a scientist formulating theories from experimental data.

Most advanced, with the ability to learn and adapt over time.

category: Agential.

Highest agential risk due to their advanced learning capabilities, which may lead to unexpected and possibly dangerous behaviors if not properly aligned.

4 Paths for the Future

1. Balancing Act (Preferred):

• Key Characteristics: Low-impact, moderate to low likelihood, decentralized diffusion, regional dominance (US/EU), retreat from globalization, protectionism, cyber war threats.
• Drivers: COVID-19 pandemic aftermath, increased international tensions, decline in cooperation.
• AI Development: Slow and gradual, primarily centered on regional innovation. Low likelihood of high-level AI before 2062.
• Risks: Sporadic outbreaks of conflict, increased cyber-attacks, race dynamics of isolation, long development timeline.
• Benefits: Safer, more controlled AI development, potentially less disruptive impact on society.

2. Accelerating Change (Middle-Ground):

• Key Characteristics: Moderate impact, moderate likelihood, multipolar AI competition, technology outpacing governance, shift in power from nations to corporations, embodiment as driver.
• Drivers: Current AI research trends, technology giants consolidating control over industries.
• AI Development: Faster than Balancing Act, with advanced AI arriving between 2042 and 2062.
• Risks: Structural risks to society from rapid technological advancements, monopolization, safety concerns with outpacing governance.
• Benefits: Potential for faster progress and innovation, but with increased risks and challenges.

3. Shadow Intelligent Networks:

• Key Characteristics: Subtle impact, variable likelihood, distributed network of intelligent agents, self-organization risks, control concerns, based on Eric Drexler’s “CAIS” framework.
• Drivers: Planned smart grid and next-generation network technologies, potential insight into intelligence.
• AI Development: Hybrid AI paradigm, distributed diffusion, AI ecology emergence, advanced AI between 2042 and 2062.
• Risks: Goal alignment issues, runaway distributed systems, monopolization, safety concerns.
• Benefits: Potentially less disruptive, distributed AI systems offering new capabilities, but with unique control and safety challenges.

4. Emergence (High Impact, Low Likelihood):

• Key Characteristics: High impact, low likelihood, rapid AI takeoff in non-Western state, AI arms race, centralized control, regional power shift (East/South Asia).
• Drivers: Economic and geopolitical competition, nationalization of AI research, resource-prohibitive and compartmentalized AI technology.
• AI Development: Extremely fast, with high-level systems expected within 20 years (before 2042).
• Risks: System failures, inner misalignment, power-seeking AI, regional arms race, high potential for disruptive impact.
• Benefits: Potentially fastest advancements in AI, but with extreme risks and potential for significant societal disruption.