An Adapting Threshold Model of Utility (with Lorne Whitehead)

We consider a simple model of decision making in which there is a limited capacity to make fine distinctions. We show that there are readily implementable mechanisms by which utility functions adapt in real time. That is, if the distribution of rewards shifts, the perceived utility function reacts, to maintain optimal use of the limited capacity to discriminate. Such rapid adaptation of utility is analogous to well-known properties of the visual system, and might well be implemented by similar neural circuitry. In the case that all that matters is the probability of error, the mechanism is particularly simple. If the criterion is instead to maximize expected fitness, we find a mechanism that is approximately optimal. Although such adaptive utility functions take an hedonic interpretation of preference seriously, and incorporate large swings in levels, they are at once approximately consistent with conventional economics. That is, the possibility of suboptimal choice stems only from the limited capacity to make fine distinctions. As this capacity improves, utility remains adaptive, but choice becomes fully optimal.