Figures
Citation: Powell K (2003) Economy of the Mind. PLoS Biol 1(3): e77. https://doi.org/10.1371/journal.pbio.0000077
Published: December 22, 2003
Copyright: © 2003 Kendall Powell. This is an open-access article distributed under the terms of the Public Library of Science Open-Access License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Frans de Waal's laboratory monkeys won't work for unequal pay. If a partner monkey gets a grape (big bucks) for little or no work (trading a token), a monkey will reject her measly cucumber pay from her human “boss.” And she makes her disdain known, hurling her cucumber or token out of her cubicle—even though she would happily gobble down cucumbers in other circumstances.
De Waal's work at the Yerkes Primate Center at Emory University in Atlanta has shown an aversion to inequality in non-human primates (Figure 1), drawing an evolutionary link between how humans and monkeys make decisions. Humans reject inequality, too, even if it means walking away empty-handed. This behavior cannot be explained by classical economic theory that says both monkeys and humans should take whatever reward they are offered to maximize gain. But in species like de Waal's monkeys and humans that rely heavily on cooperation for survival, evolution has favored a complex calculus for even simple decisions.
(Photo courtesy of Frans de Waal.)
In a simplified way, de Waal's experiments and others blend neurobiologists' ability to track behavior and brain processes with economists' models of the cost–benefit analyses behind every decision made by an animal. The two fields have each been working toward explaining decision-making behavior, using widely different approaches for decades. Recently, researchers in both fields have recognized that using tools from the other trade might speed their own work along, resulting in the emerging field of neuroeconomics.
Teaming Up
The principle of Expected Utility says that a person facing uncertainty will rank the possible payoffs or outcomes as a function of their expected values and probabilities of happening. Using this principle, experimental economists tested the idea that humans should interact with a self-interest that gives the highest possible gain. In the Ultimatum game, one person is given a sum of money and must decide how much of that sum to share with a second person. The second person can then decide to accept or reject the offer, but the catch is that if he rejects the offer, neither player gets any money.
Although rational-decision theory predicts that the first player should make a low offer and the second player should accept because it would maximize how much each player leaves with, the results were resoundingly irrational. Most first players offered close to half of the money and most second players rejected sums lower than half. Economists were stumped when their models fell far short of explaining human decision-making.
“Standard economic theory uses models where players are calculating complicated numbers, thinking far ahead to figure out what the other person will do, and there are no temptations,” explains Colin Camerer, a behavioral economist at the California Institute of Technology in Pasadena. Those models tended to be mathematically simple, but realistically hard on the players, he says. “People aren't that smart. An 18-year-old doesn't plan out his entire lifetime savings.”
Camerer has teamed up with neurobiologists looking at brain scans of people while they play games like Ultimatum. The results of such experiments should reveal new mechanisms at play in the brain during decisions, like aversion to inequality, that economists can add to their models to reflect the sophistication of human choices more accurately.
On the other side of the decision-making fence, neurobiologists in the last decade had begun to look beyond mapping how the brain processed sensory input or motor output and began asking questions about what was happening in between those two systems. Once they turned away from simple experiments in which a single stimulus elicits a uniform response, giving meaning to neural activity was no longer easy. For example, Paul Glimcher and his colleagues at New York University in New York City gave monkeys a visual cue indicating that a gaze shift either to the left or right would result in some level of juice reward. All things being equal, monkeys had no reason to favor one side or the other. However, when the experimenters increased the amount of juice reward for one side on random trials, the same visual cue now elicited a very different pattern of movement, favoring that side. And the neural activity they recorded appeared to reflect the monkeys' sense of how they could get the most reward, rather than any clear association with sensation or action.
To account for these results, Glimcher and others turned to economic models of decision-making that took into account the probability of a reward, the size or value of the reward, and the cost of work to get the reward. These variables, the neuroscientists hypothesized, might lie between environmental stimulus and action and be the link between sensory neurons and motor neurons in the brain.
“We know that to make efficient decisions, you have to know the utility of the decision,” says Glimcher. “Economists had beautiful computational models to describe all of these processes” used to calculate utility. Now, he says, the next step is to look for these variables in the brain at a cellular level.
Cell Decisions
One laboratory has shown that neurons can indeed “code” for some of the variables weighed during simple decisions or choices. Wolfram Schultz and his colleagues at Cambridge University in the United Kingdom studied dopamine-releasing neurons in the ventral midbrain of monkeys. Dopamine neurons have long been implicated in reward-seeking behavior and are targets of highly addictive drugs like nicotine and cocaine.
In their experiment, monkeys responded to five distinct visual stimuli that matched the probability of the juice reward. Each cue represented either a probability of 0 (no reward), 0.25, 0.50, 0.75, or 1 (certain reward). Because uncertainty and probability are inherently linked—that is, uncertainty is highest at a probability of 0.50 and lowest at 0 and 1—the researchers could then look for neuronal responses to both variables.
What they found were two distinct ways in which the same dopamine neurons code for probability and uncertainty. With decreasing probability of reward, the monkey's dopamine neurons fired stronger bursts at the time of the reward delivery. At the same time, with greater uncertainty of reward, a sustained increase in activity occurred between the flashing of the visual cue and the reward delivery. In other words, this intervening activity was not seen when the probability equaled 0 or 1 and was greatest when the probability of getting a reward was 50/50, the highest level of uncertainty.
“To make decisions about rewards or money, a person has to make predictions about the future, and in any prediction there is some uncertainty that is critical,” says Christopher Fiorillo, a neurophysiologist in Schultz' lab who led the study. “This is the first demonstration of a single neuron coding uncertainty.” Fiorillo says there are probably many other types of neurons in the brain that can code uncertainty, but the fact that dopamine neurons do it adds another intriguing layer to decision-making behavior.
Dopamine neurons, he explains, have been shown to have a reinforcing effect, so that an animal will seek out stimuli or actions that are followed by a release of dopamine. So, Fiorillo says, he was surprised to see the activity of dopamine neurons increased by uncertainty about a reward, as if uncertainty itself were rewarding in some way. The finding might help explain why people are drawn to gambling even though they tend to lose money on average. Fiorillo and his colleagues speculate that outside the artificial conditions of a laboratory or a casino, an uncertain situation presents a learning opportunity that may help the decision-makers “beat the odds” the next time they face it. And so evolution would favor paying attention to highly uncertain scenarios.
In another experiment, Glimcher and colleagues had monkeys play a “game” in which there were two ways of getting juice, with each choice having a different probability of reward. After 100 trials, the probabilities were changed.
“Basically, he's playing a two-armed slot machine whose payoff rates are constantly switched on him,” says Glimcher. “His behavior looks pretty erratic, but he's getting it about right. He spends two-thirds more time on the one that is two-thirds more likely to give him juice.” And, he says, an economic model of this choice predicts the monkey's behavior with 90% accuracy. This might indicate that, subconsciously, humans tally probabilities, expected gain, and the cost of the work to get the reward in all manner of simple choice decisions. At the level of neurons, we might all be math-whizzes.
But game-theory work has shown that humans rarely think ahead in complex interactions far enough to arrive at the most rewarding decision. Glimcher, by applying the principles of the utility decision theory of maximal gain, has found neurons that may code the variables that go into such a decision. But, others say, since humans do not always act in a way that maximizes their gain, other computational models may give a better answer to how we make decisions in more complex contexts.
“The difference is a question of perspective, since we're really all interested in the same issues,” says Joshua Gold, a neuroscientist at University of Pennsylvania in Philadelphia. He and his collaborators have used a mathematical framework called Banburismus to model decision-making in monkeys performing a difficult visual task.
British codebreakers used Banburismus in World War II to break the German navy's Enigma code (Figure 2). It consists of three components: a method to quantify the weight of evidence, a method to update this quantity with additional evidence, and a decision rule that determines when there is enough evidence to make a decision. Gold and his colleagues apply the framework to monkeys watching a cluster of dots moving across a screen of randomly moving dots. The monkeys earn juice by shifting their gaze in the same direction as the dot cluster. By increasing the number of randomly moving background dots, they can push the monkeys' visual system to its limits.
Alan Turing and colleagues at Bletchley Park broke the German Navy's unbreakable Enigma code with the help of a mathematical framework they called “Banburismus.” Some neuroscientists think this framework could also help to break the neural code. (Image modified from the National Cryptologic Museum of the National Security Agency; www.nsa.gov/museum/enigma.html.)
It is at this point, Gold says, that other factors besides visual cues come into play as the monkey decides how to answer. “By getting the monkey to work in a regime where he's coming close to guessing, then we see much more influence by extraneous factors such as bias [i.e., the monkey's previous experiences] and size of reward,” says Gold. The computational model gives a way to represent these factors mathematically and can also predict the error rates and reaction times of the monkeys' decisions.
“If it really does explain behavior mathematically,” adds Gold, “it will be a nice way of studying how those variables predict behavior.” These models, whether based in economic decision theory or statistics like Banburismus, give physiologists good candidates in their search for decision-making functions in neuronal circuits. Neurobiologists chasing the perfect model that can incorporate all the factors that go into a decision say it will show how humans calculate the mental “currency” that allows us to literally compare apples to oranges and decide which to buy.
Some biologists, however, are cautious about translating what happens in an economics-based neurobiology experiment in the lab to more complicated human behavior in business or courtroom decisions. “We don't want to say things that are wrong, incomplete, or could be miscontrued,” says Jeff Schall, a vision researcher at Vanderbilt University in Nashville, Tennessee. “And we don't need to, to make scientific progress.”
His lab has found two sets of neurons in the anterior cingulate cortex that respond when a monkey shifts his gaze to a target—those that signal success and those that signal mistakes. He has also recorded similar signals from electrodes placed on humans performing the same task. These signals can be thought of as the “oops” or “high-five” feeling that tells an animal how to proceed in the next trial. Both humans and monkeys slow down on the next trial after an “oops” signal. Schall says his work shows another aspect of decision-making, adaptation, not accounted for in classical economic views of reward influences.
“Neuroeconomics is just part of the bigger picture of goal-directed action,” he says. Others say economic theories lack another critical component—how to calculate how much value the reward has to the decision-maker.
“There's a lot of emphasis on game theory and it's very exciting, but there's one flaw that everyone recognizes,” says Barry Richmond, a systems neuroscientist at the National Institute of Mental Health in Bethesda, Maryland. “How do you measure value at any given moment when it is changing both because of personal situation and because of external things?” Richmond sees that monkeys, like humans, exhibit different levels of aversiveness to work.
Richmond's monkeys have been trained to learn a visual cue, a brightness bar, that indicates how much work is left before getting the reward. As the reward gets closer, its “value” appears to go up, because the monkeys work harder (by making fewer errors) in the last trials before the reward. Obviously, value pivots on the timing of the reward, among many other considerations.
It is easy to see a little bit of ourselves in this monkey business. Students study furiously the night before an exam to be rewarded with a grade. As a project deadline looms, employees put in longer hours in order to keep their job. But some experimentalists have gone even further in making connections between neurons firing in a monkey's brain and what's going on in ours.
Let Me Pick Your Brain
Kevin McCabe, a neuroeconomist at George Mason University in Fairfax, Virginia, was among the first social scientists to set up a neurobiology experiment to answer his questions about how humans make decisions. His early work showed that if you changed the Ultimatum Game into the Dictator Game, where the first person simply dictated how much the second person got, then humans still gave a fairly large sum away, about one-third of the total. Only when the experimenter and the second person could not see the decision of the dictator did the dictator begin acting in the rational, self-serving manner of giving away tiny amounts. Only in the socially isolated context did the dictator follow economic principles.
“We wanted to design an imaging experiment to demonstrate that when people reciprocate, brain processing is different than when they are not cooperating,” says McCabe. The subsequent experiment, where people played the Ultimatum Game inside a scanner that takes a functional magnetic resonance image (fMRI), showed that blood flow and, by proxy, neuronal activation increased in the frontal brain areas of cooperators. These areas included human homologs of the lateral intraparietal area that Glimcher had seen activated by reward size and probability and the anterior cingulate cortex that Schall found to send success or failure messages (Figure 3).
The ventral midbrain is active when humans receive an unpredictable juice reward. Monetary rewards, although defined by cultural agreement, also engage the same subcortical reward processing structures. (fMRI image courtesy of P. Read Montague.)
McCabe's experiment hints that humans are wired to cooperate. “We're biologically endowed to engage in personal exchange,” he says. “And what makes economies run so well is not personal exchange per se, but our ability to trade with people we don't even know—to buy food at the grocery store from a farmer we've never met.”
Another group, led by Read Montague, director of the Human Neuroimaging Laboratory at Baylor College of Medicine in Houston, Texas, has also looked at brains of cooperators in the Trust Game. Here, an investor decides to trust a trustee with some of her money. The investment is increased by the experimenter and then the trustee decides how much to give back to the investor. This game is played out ten times by two people who meet each other at the beginning and whose brains are scanned simultaneously as they play.
The researchers wanted to see what happens in each player's brain when the trustee's decision is revealed to both on a computer screen. “The trustee's brain shows the visual cortical activity only of seeing the screen,” Montague explains. “But the investor's brain goes haywire, with both emotional and cognitive reactions to what they see.” Presumably, the activity represents the investor trying to assimilate the information into her decision of how much to invest in the next round.
Montague, a physicist by training, says he's found a home in the computational nature of neuroeconomics, which adds a “fresh look at a bunch of problems that were previously only at the margins of behavioral psychology.” But he also sees the advantages that the field brings to economists by shoring up their models with physical evidence: “Let's face it, they don't have good models now or they could tell you what's going to happen [in the stock market] tomorrow. This is starting to give economists a way to loop back into experiments—they realized they've got to crack the head open.”
Montague's collaborator Camerer agrees that knowing how individual humans make decisions could certainly improve our understanding of larger markets. After all, global trade institutions are still run by individuals who draw on their own ability to trade and make decisions. Unraveling the decision-making code would open windows on economic questions ranging from the global (Why do certain countries enjoy economic growth?) to the very personal (What causes compulsive behavior when reward systems go bad?).
Camerer sees neuroeconomics as trying to “make a one-to-one mapping from economic theory to the brain. We have a head start, but it's very difficult to produce clear neuroscience that also has economic significance.” In just a few decades, he envisions that economic theory may look very different, perhaps throwing out utility altogether and instead having a system of mechanisms found in the brain that interact to help a shopper decide, “What's for dinner?”
And the knowledge coming out of the fledgling field—how the brain codes motivation and reward value—could be used to increase work output, promote more effective addictive drug rehab programs, and stabilize economies. Camerer adds, “This work can really go from synapses seen in brain imaging to explaining the most important thing in the world—why is Africa poor and Singapore rich?”
Further Reading
- 1. Brosnan S, de Waal F (2003) Monkeys reject unequal pay. Nature 425: 297–299.
- 2. Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299: 1898–1902.
- 3. Gold JI, Shadlen MN (2002) Banburismus and the brain: Decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36: 299–308.
- 4. Liu Z, Murray EA, Richmond BJ (2000) Learning motivational significance of visual cues for reward schedules requires rhinal cortex. Nat Neurosci 3: 1307–1315.
- 5. McCabe K, Houser D, Ryan L, Smith V, Trouard T (2001) A functional imaging study of cooperation in two-person reciprocal exchange. Proc Natl Acad Sci U S A 98: 11832–11835.
- 6. Platt ML, Glimcher PW (1999) Neural correlates of decision variables in parietal cortex. Nature 400: 233–238.
- 7. Sanfey AG, Rilling JK, Aronson JA, Nystrom LE, Cohen JD (2003) The neural basis of economic decision-making in the Ultimatum Game. Science 300: 1755–1758.