The Value of Abstraction

Made for a reading group at the Center for Safe AGI.

The Value of Abstraction

Mark K. Ho, David Abel, Thomas L. Griffiths, Michael L. Littman


Agents that can make better use of computation, experience, time, and memory can solve a greater range of problems more effectively.

A crucial ingredient for managing such finite resources is intelligently chosen abstract representations.

But, how do abstractions facilitate problem solving under limited resources? What makes an abstraction useful?

To answer such questions, we review several trends in recent reinforcement-learning research that provide insight into how abstractions interact with learning and decision making. During learning, abstraction can guide exploration and generalization as well as facilitate efficient tradeoffs—e.g., time spent learning versus the quality of a solution.

During computation, good abstractions provide simplified models for computation while also preserving relevant information about decision-theoretic quantities.

These features of abstraction are

Keywords: abstraction, reinforcement learning, bounded rationality, planning, problem solving, rational analysis