W10. Learning 02
Today, we introduce the concept of utility, or the ability to evaluate different actions based on current state. From a single-agent perspective, we can evaluate whether our actions are reasonable given feedback from the world, and a way to value our state as a result of our actions. By keeping a record of actions and value, we can start to model uncertainty in the world and make predictions.
Pre-readings and Videos
These videos demonstrate various simulations of evolution given a concept of basic utility and resource constraints. Importantly, sometimes the utility function is explicitly defined, and sometimes it is implicitly enforced through emergent processes.
Simulating Evolution
Emergent processes can determine agent traits, given a bit of randomness in a system. This version of utility is implicit, just defined as survival.
Simulating Markets
This video demonstrates emergent processes of dominance of traits, whereas utility and valuation is explicit.
Summary of the Day
- Class notes. Available here
Learning Goals
- Be able to define and discuss utility both at an abstract and concrete level.
- Design a simple utility function that translates a state and an action into a reward.
- Design an experimental protocol that associates an action with a positive or negative reward state.