Does it pay to be nice?

The maths of altruism

An ebook by Plus, the online maths magazine (http://plus.maths.org).

This ebook is © University of Cambridge 2013.  Copyright of the individual articles is held by the authors and they are reproduced here with permission.

Table of contents

Does it pay to be nice?

The maths of altruism

by Rachel Thomas

Mathematical Mysteries: Survival of the nicest

by Helen Joyce

Guilt counts

by Marianne Freiberger

Trust me, I’ve evolved

by Marianne Freiberger

But wait there’s more...

Does it pay to be nice?

Yes it does.  And we’re not just talking about that warm fuzzy feeling inside. Maths indicates that it pays in evolutionary terms too.  One of the most puzzling aspects of human behaviour is cooperation, in situations where backstabbing and selfishness would seem to be more rewarding.  But it appears that the mathematics of evolution proves that being nice is unavoidable, even on our grumpiest of days.

Plus (http://plus.maths.org) is an award-winning free online maths magazine that aims to open a door onto the world of mathematics for the general public.  Our articles, interviews and podcasts feature some of the best writers, mathematicians and scientists, including Marcus du Sautoy, Ian Stewart, Stephen Hawking and John Conway.  And we are also a starting point for many budding science writers and researchers speaking to the public for the first time.  Covering topics including medicine, politics, science, philosophy and music – we try to reveal how mathematics is the language of our Universe.

This is one of our first e-books, collections of Plus articles on particular themes that have intrigued, challenged and entertained our online readers.  You can find many more at http://plus.maths.org/ebooks. We’d love to hear your feedback at plus@maths.cam.ac.uk and we hope you enjoy the read!

The maths of altruism

by Rachel Thomas

Does it pay to be nice? Yes, it does. And we're not just talking about that warm fuzzy feeling inside, it pays in evolutionary terms of genetic success too. In fact being nice is unavoidable; research suggests that humans, or any population of interacting individuals (including animals, insects, cells and even molecules) will inevitably cooperate with each other.

Professor Martin Nowak, from the Program for Evolutionary Dynamics at Harvard University, sees cooperation everywhere. So much so, that he believes cooperation is the third process necessary for evolution, after mutation and natural selection: "You might say that whenever evolution wants to come up with some creative new solution, such as the emergence of the first cell, going from single-celled organisms to multicellular organisms, or the emergence of insect societies or human societies, cooperation plays a role."

It took cooperation to get from single-celled to multicellular organisms, such as the nematode C. elegans (stained to highlight the nuclei of all cells.) (Image from PLos Biology)

Over the last 30 years Nowak and his colleagues have created and investigated a series of evolutionary games, each based on how populations interact in the real world. To his surprise, cooperation has emerged as the most successful behaviour in game after game. And this isn't a question of moral judgment, this is all judged in the cold hard terms of the mathematics of evolution.

Book-keeping…

Using mathematics to describe a human behaviour, such as cooperation, might at first seem quite absurd. But it is possible thanks to the convergence of two very important areas of mathematics.

The first of these is the mathematics of evolution. Charles Darwin expressed his theory of evolution in words when it was first published in 1859, and for many years this revolutionary theory could only be discussed verbally. Today evolutionary theory has a strong mathematical basis. "Just talking about evolution wouldn't be using the level of precision that is actually available," says Nowak. It would be like physicists talking about the Solar System only in words and not using the equations of Newton or Einstein.

"Evolution is very precise and every idea needs to be couched in this language of the mathematics of evolution," says Nowak. The maths you need is a bit like book-keeping, describing how different strategies, different mutants, reproduce in a population and keeping track of who will be abundant and at what time.

…and playing games

Nowak's work is also based on game theory. This area of mathematics, founded by mathematician John von Neumann and economist Oskar Morgenstern in the 1940s, revolutionised economics by making it possible to describe and understand the strategies people used when making decisions in a competitive environment.

Typically in game theory you would consider a game with two players, say you and I, where the outcome (in terms of a cost or benefit) of the game for me would depend on your decision as well as my own (and visa versa). This approach differed from previous economic theories in that it was the interaction of the two parties that was vital; the outcome for each party could not be considered in isolation. "So it's not just an optimisation problem for me and it's not just an optimisation problem for you," says Nowak.

Suppose we are playing a two player game where we each have to simultaneously choose between two choices: to cooperate or not to cooperate (not cooperating is known as defecting). Then the possible outcomes are: we both defect, I defect while you cooperate, I cooperate while you defect, or we both cooperate. You can define the payoffs and costs to each of us for each of these outcomes in a payoff matrix that completely describes the game. The values of the payoff matrix, where the first number in each pair is the benefit/cost to me and the second is the benefit/cost to you, can determine lots of different situations.

You cooperate

You defect

I cooperate

-2, -2

-4, -1

I defect

-1, -4

-3, -3

Direct reciprocity

The hardest situation to resolve in game theory is known as the Prisoner's Dilemma. (The payoff matrix above gives one example, and the more general form is given in the matrix shown in the photo below) In the Prisoner's Dilemma, if you defect then the best choice for me is to also defect, and if you cooperate the best choice for me is still to defect: either way the logical thing for me is to defect. You will come to the identical conclusion, and so, we will inevitably both defect. It seems irrational to choose to cooperate despite the fact that we both would have had a much better outcome if we had cooperated. This paradoxical outcome lies at the heart of the Prisoner's Dilemma. (You can read more about the Prisoner’s Dilemma in the next chapter.)

Martin Nowak and the Prisoner's Dilemma

The Prisoner's Dilemma seems to leave no hope for cooperation if the players act rationally. But surprisingly, in certain situations, cooperation can triumph. "Cooperation doesn't emerge in the Prisoner’s Dilemma unless you have some mechanism in place," says Nowak. The simplest mechanism that allows this is called direct reciprocity.

In a direct reciprocity game we are now no longer playing the Prisoner's Dilemma only once. Instead we will keep playing, deciding if the current round will be the last with some probability. For example, we could roll a dice and if it is even then we will play another game. (If you knew that this round was definitely the last it is logical to treat it as a one-off game, leading you to defect. The same argument then works for the previous round, and the previous round, resulting in you always defecting as you would in a single play game.)

The defining feature of a direct reciprocity game is that our strategies will depend on the past history of our opponent's behaviour:

If I cooperated last time, you will cooperate this time with probability p.

If I defected last time, you will cooperate this time with probability q.

"So your strategy doesn't depend on what you did, it just reacts to what I did," says Nowak, "and it is completely defined by just two parameters: p and q."

The political scientist Robert Axelrod explored this iterative version of the Prisoner's Dilemma in the 1970s. Instead of just having the game between two players, Axelrod invited people to submit strategies that would all compete as players in a tournament. Each player (representing one of the submitted strategies) would play every other player in each round and the winning strategy would be the one with the highest payoff at the end of the tournament.

"The undisputed champion in this [tournament] was Tit-for-tat", says Nowak. If you play this strategy then at our first meeting you will always cooperate. Thereafter you will repeat my last move: cooperating if I cooperated in our last game and defecting if I defected in our last game. This eye for an eye strategy is described by p=1 and q=0.

Stickleback fish rely on the Tit-for-tat strategy when approaching a potential predator to determine how dangerous it is. As pair of stickleback approach in short spurts, each spurt can be thought of as a round of the Prisoner's Dilemma. (Image by Ron Offermans)

In the 1980s, Nowak and Karl Sigmund (his PhD supervisor at the University of Vienna) explored what happened in these tournaments when some important ideas from evolution were introduced. The first of these was mutation: "The old tournaments had these artificial strategies, rather than allowing natural selection to generate strategies by random mutation." Instead Nowak started off with a random distribution of strategies, choosing the values of p and q for each initial strategy at random. Then the payoff in each round could be thought of as reproductive success – the more successful a player was the more offspring they produced to play the next round. And the strategy of each of these offspring would be based on their parent's strategy, but allowing for some random mutations in the values of p and q.

The other problem Nowak identified was that the tournaments didn't allow for any errors to occur. Errors come in two flavours, says Nowak, trembling hand – where you mean to do one thing and accidentally do something else – and fuzzy mind – where you incorrectly remember the past. "Fuzzy minds leads to the scenario where you and I have different interpretations of history. That's a typical feature of human interaction where there are two parties in conflict and both know, for sure, that the other one started it!" Nowak introduced the possibility of errors into his tournament by giving each player a probability that they would make these errors.

The history of human conflict?

Nowak set his evolutionary mathematical model of the repeated Prisoner's Dilemma to run on his computer and sat back to watch what would happen. "The first thing we saw was the emergence of Always Defect: no matter what you do I'll always defect." It is intuitively obvious that the defectors were the early victors in a game starting from a population of random strategies: if they are going to play randomly it is always better for you to defect.

The defectors were not on top for long. Very suddenly the population swung over from Always Defect to Tit-for-tat, the winning strategy in Axelrod’s original tournaments. But most interestingly it didn't end there; Tit-for-tat didn't last for long, just 10 or 20 or so generations. "Tit-for-tat was just like a catalyst, speeding up a chemical reaction but disappearing immediately after," says Nowak.

What remained was Generous Tit-for-tat, a more cooperative strategy that incorporated an idea of forgiveness. "In this strategy if you cooperated last time then I will definitely cooperate this time, so p=1. And if you defected last time I will still cooperate with a certain probability. So I will always cooperate if you cooperate, and I sometimes cooperate even if you defect, and that is forgiveness." This probability of forgiveness was just the probability q in each strategy. The most successful strategy to emerge from the tournament had p=1 and q=1/3.

Evolutionary mathematics can explain the cycles of war and peace in human history. (Engraving by Hans Holbein the Younger)

This was a dramatic difference to Axelrod’s old tournaments where Tit-for-tat reigned supreme. Even more surprising was that the system carried on evolving, the society becoming more and more cooperative, more and more lenient, until it was dominated by players who always cooperated. "And once you have a society of Always Cooperate it invites the invasion of Always Defect," says Nowak. All it takes is a few mutations in the strategies as they reproduce and the whole cycle will start again.

"It is very beautiful because you have these cycles of cooperation and defection." Nowak's first observations in the field have since been confirmed by many other studies over the years: cooperation is never fully stable. "So we have a simple mathematical version of oscillations in human history, where you have cooperation for some time, then it is destroyed, then it is rebuilt, and so on."

Your reputation precedes you

Nowak had already discovered that playing according to the rules of direct reciprocity leads to the rise of cooperation. In direct reciprocity, my strategy for playing with you depends only on how you played in your previous games with me. But what if we had never met before and may never meet again? Instead, I might base my decision on how you played in all your previous games against other players. That is, my decision of how to play is based on your reputation.

This more complex game is called indirect reciprocity. A number of games are played in each round, where players are randomly paired such that one is acting as a donor and the other as a recipient. The donor then either chooses to help the recipient (cooperate), giving the recipient a benefit b at a cost of c to themselves (where b>c), or chooses not to help them (defect). Now our priority isn't the immediate payoff from each game, instead we want to improve our reputation in order to increase our future success within the round (in terms of number of offspring, which depends on the total costs and benefits we accrue during a round). "I might help someone even though I don't expect it to be returned by that person, but it gives me a better reputation," says Nowak.

Strategies for playing indirect reciprocity games consist of a social norm and an action rule. The social norm gives players a way to judge other players' reputations and interpret their actions. These social norms might be very simple, such as "All players are good" or "All players are bad". Or they might be very nuanced, considering a player's behaviour and the reputations of their past opponents. For example: "A player is good if they helped those with good reputations and didn't help those with bad reputations". (Simple approaches use the same social norm for all players. Allowing social norms to evolve for individual players is a more mathematically challenging problem.)

Mathematically Nowak defined the social norm to be how a player's reputation changes, in our eyes, as we observe them playing with other opponents. A simple example would be that our assessment of a player's reputation is a number, r, which we set to zero until we observe them playing the game. Then their reputation increases by one unit each time we see them help and decreases by one unit each time we see that they don't (r can be any integer: positive, negative or zero). A more complex social norm might have a player's reputation increase (or decrease) only if we see them help (or not help) players with reputations larger than a certain value.

The action rule defines how I will act in a game. It is similar to the probabilistic strategies of direct reciprocity but this time my decision as a donor to help a recipient depends on how I perceive their reputation, rather than on their past history with me. For example, my strategy might be to only help those recipients whose reputation I judge to have at least some value, k (so I would help someone if their reputation r ≥ k).

Cooperative strategies are defined as those that would help a recipient, even if the donor didn't know anything about the recipient's reputation (and so the recipient's reputation r=0). So I would be considered as cooperative if my value for k was at most zero (k ≤ 0). This means that, not only would I help players with a positive reputation score, I would also help players that were playing for the first time or that I hadn't observed playing before (whose reputation score was, therefore, zero). And a strategy is more and more cooperative the lower the value of k.

If, however, my value for k was greater than zero (k > 0) then I would not help someone whose reputation was zero (eg if they were playing for the first time). An all-out defector would have k = ∞ and would always defect against all other players.

When Nowak ran computer tournaments based on indirect reciprocity he again found, when allowing for mutation and errors, that the population would cycle from unbending defectors, through cooperative strategies, to unconditional cooperators and back again. But the most successful, in terms of how long they remained dominant in the population, were those strategies that behaved cooperatively and discriminated on the basis of their opponents' reputation, that is, those with k ≤ 0.

Indirect reciprocity can explain many altruistic behaviours, such as acts of charity. "For example, suppose the university goes to a donor to ask for a donation," says Nowak. "The donor might not consider this based on repeated interaction with the university but rather, I'll give this donation and get a reputation as a valuable member of society." So giving to others not only makes us feel better about ourselves, but also makes others in our community think more highly of us, and our enhanced reputation might benefit us in the future.

It's good to gossip

Although there are examples of indirect reciprocity in animals (such as cleaner fish and their clients), it is humans who have taken this behaviour to a whole new level with the development of language. "Animals have a more elementary version of indirect reciprocity," says Nowak. It seems that animals can't share their experiences of others' past behaviour. Instead they have to observe the behaviour of others directly. "No other organism has the full blown indirect reciprocity because of the lack of human language. They can have indirect reciprocity by observation, but not augmented by communication."

Cleaner fish benefit from having a good reputation. Their clients, such as this cod, are more likely to let themselves be cleaned by those cleaner fish that have been observed to have done a good job on others. (Image by Richard Ling)

Humans, however, are great at talking and we also love nothing more than a bit of social gossip, passing on who has done what to whom and trying to understand why. "We humans are very good at that," says Nowak. "And that is what I find fascinating: what could have been the [evolutionary] selection pressure that made humans? It's not just [group] hunting as that is done by other animals. It really is the complicated politics of indirect reciprocity." Nowak believes that the cooperative force of indirect reciprocity is not only responsible for the evolution of human language (see the further reading list below for more on this), but that it also drove the development of our understanding of social complexity. "Indirect reciprocity drove the selection of human language and social intelligence."

Family, friends and neighbours

As well as the games involving direct and indirect reciprocity, cooperation also emerges in a number of other evolutionary games. For example, spatial structure can change the outcome of games. In games where defection would win straight out in a well mixed population, says Nowak, cooperation triumphs if the players are restricted to only interacting with their neighbours: "Neighbours help each other."

The cooperative mechanisms of group and kin selection have been studied as part of evolution for decades but have also caused much controversy. Darwin himself investigated group selection, the idea that a trait that is beneficial to a group (as opposed to an individual) will be favoured by natural selection. Many examples were given to support this theory, for example vervet monkeys calling to warn their group of a predator, despite putting themselves in greater danger, or people sacrificing themselves to save the lives of strangers. However, there was much argument over when and why this behaviour arose. "[Previously] there wasn't a good mathematical approach to group selection, it was only a verbal discussion. But when you look at the mathematics it is clear when [group selection] is valid and when it isn't. The discussion disappears and mathematics decides the argument."

The advantages of kin selection, helping a relative (rather than just an unrelated member of your social group, as in group selection), seem obvious. "For me, kin selection makes a lot of sense if you present it in a precise way," says Nowak. "I recognise my brother and I behave differently to my brother than to a stranger." However, things are not quite as straight forward as they seem. Nowak and his colleagues Corina Tarnita and Edward Wilson caused much controversy in 2010 when they published a paper criticising the standard mathematical argument for kin selection, called inclusive fitness theory (see the further reading list below). They showed that inclusive fitness theory had much greater limitations than people had realised. In particular, inclusive fitness could not be used to explain the evolution of the highly complex cooperative insect societies, such as leaf cutter ant colonies, where millions of ants work and die so that only one individual, the queen ant, can reproduce.

Evolutionary mathematics can explain the altruism in insect societies, such as leafcutter ant colonies, but it doesn't involve inclusive fitness. (Image by Adrian Pingstone)

The analysis of group and kin selection demonstrates the contribution of maths to evolution and biology. "That's what the role of mathematics is in the sciences. If you have a verbal discussion, you make certain assumptions and you think it leads to the following conclusions, but it's not rigorous." Nowak, like many other mathematicians and scientists, believes that if you can't give a clear mathematical description you haven't really understood the science. "Real understanding in science in terms of mathematics is ultimately elegant, it's ultimately simple, so we understand the situation well if we have a simple description. If we don't, then I'm not quite satisfied."

Nowak's research has ranged from studying human, animal and insect behaviour, to the development of language, and even delved into cancer and cell biology. "People often ask me: how can you work on so many different questions in so many different fields? It is always the same questions, it's always the mathematics of evolution. The maths of populations that are reproducing, competing and interacting. It's the approach that I'm using to study questions in the life sciences, in economics, and in the origin of life. It seems to me that evolution is such a fundamental principle that it should be everywhere, in every scientific description of the world, not just the biological. I'd be very curious [to see if I could find it] in the fundamental laws of physics!"

Can we save the world?

It seems that wherever you apply the mathematics of evolution, there is cooperation. These different mechanisms – direct and indirect reciprocity, spatial game, kin and group selection – that influence how people, animals, insects, or even how molecules or cells interact all lead to emergence of cooperation. Moreover, Nowak is convinced that cooperation isn't just an outcome of evolution, it is necessary for evolution. "I have slowly realised that cooperation plays a role in so many aspects of evolutionary organisation. That's why I made the argument that cooperation can be seen as the third fundamental pillar of evolution, next to mutation and natural selection. Without cooperation you wouldn't get the construction that is evident [in the world]."

Can we cooperate to save the world? (Image from NASA)

And cooperation might not just have been vital in our evolution, but also in our very survival. "The biggest problem we are facing is not one of medical research, though it would be great to cure cancer and many infectious diseases. And the biggest problem isn't economic, how to fix the economy. The biggest problem is how to maintain the stability of intelligent life on this planet." And the only way to solve that problem, Nowak thinks, is that we all become supercooperators, in the sense that we must cooperate with many other individuals everywhere on the globe, now and in the future. It is no longer enough to just play the evolutionary game with the people we know or even the people who are alive today. "Everything we are doing now has an implicit cost on the people to come. Eventually they will have to pay it, and maybe they will not be able to. That is the biggest problem we are facing."

So if cooperation is the answer, can this new mathematical understanding of its evolution show us how to become supercooperators? "I hope so. I hope it will lead to a global rational analysis of the situation where mathematics can play a role to identify the problems and identify the solutions." Let's hope that Nowak and his colleagues discover a clear and elegant mathematical argument for why we should cooperate with future generations, before it's too late.

Further reading

1. You can read a fascinating paper by Nowak and colleague Natalia Komarova giving a mathematical model that describes how language evolves.

2. This paper by Corina Tarnita and Edward Wilson published in the journal Nature in 2010 sparked controversy by criticising the standard mathematical argument for kin selection.

3. This statement by Martin Nowak, Corina E. Tarnita, and Edward O. Wilson addresses the controversy surrounding their paper.

4. Read our review of Nowak’s book SuperCooperators: Evolution, altruism and human behaviour or Why we need each other to succeed, written with Roger Highfield.

About this article

Martin Nowak and his colleagues at the Program for Evolutionary Dynamics

Martin Nowak is Professor of Biology and of Mathematics at Harvard University and Director of Harvard’s Program for Evolutionary Dynamics.

Rachel Thomas, editor of Plus, interviewed Martin Nowak in Boston in January 2012.

Mathematical Mysteries: Survival of the nicest

by Helen Joyce

Survival of the nicest?

One of the most puzzling aspects of human behaviour is cooperation, in situations where backstabbing and selfishness would seem to be more rewarding. From the point of view of evolutionary theory, the very existence of altruism and cooperation appear mysterious. The mechanics of evolution seem to imply that rugged competition is the order of the day; that, given an opportunity to benefit by cheating someone, or by defaulting on a deal, we will inevitably do so. Surely to do otherwise would mean relegation to the sidelines of the evolutionary game — and in that game, demotion means extinction.

In fact, as even the most cynical observer must admit, cooperation is rife in human society. Even if you sometimes despair of human nature, you must admit that the "dog-eat-dog" scenario conjured up by the phrase "survival of the fittest" doesn't bear much resemblance to life as we know it. So it must be that, from a purely self-interested point of view, cooperation can actually be good for us.

To highlight this puzzle, consider the Prisoner's Dilemma. To summarise, this very famous paradox in Game Theory describes two people suspected of being accomplices in a crime. They are held prisoner in separate, non-communicating cells. The police visit each prisoner, and tell both that if neither confesses, each will be sentenced to two years in jail. However, if only one prisoner confesses, implicating the other, the one who confesses will get off scot-free as a reward, and the other, who didn't confess, will receive a punitive sentence of seven years. If each confesses and implicates the other, both will be sentenced to three years.

What should a prisoner do in this situation? Suppose the other prisoner doesn't confess. Then the best course of action is to confess, and go free. Even if the other prisoner does confess, it will be better to have done likewise — at least the sentence will be lower. Both prisoners will reason thus, so both will confess and end up serving sentences of three years — even though, if both had remained silent, both would have served sentences of only two years.

If you think this dilemma is very far from your everyday life — after all, you are law abiding and will never be thrown in jail! — think again. Every time you make a bargain, you are potentially facing the prisoner's dilemma. What is to stop you — or, more to the point, the person you are making the bargain with — from defaulting? Surely both of you will be tempted by the prospect of getting something for nothing, and afraid that if you are honest the other person won't be, and you'll get landed with the so-called "sucker's payoff" — getting nothing for something? It's all very well to say that "honesty is the best policy" but surely this is a luxury that only the civilised and comparatively rich can afford?

Well, the good news is that we are not dependent on the benevolence of others, as the prisoner's dilemma would seem to suggest. In fact, cooperation can spontaneously break out even among fundamentally selfish agents — provided you assume that people meet each other more than once, and can remember what the other person did last time they tried to strike a bargain.

To explore this sort of situation, political scientist Robert Axelrod invented the game of Iterated Prisoner's Dilemma — Prisoner's Dilemma played repeatedly against the same opponent — and set up a tournament, inviting academics from all over the world to devise strategies. First Axelrod compared various strategies by pairing them and seeing who won; then he held a meta-tournament, in which there were many agents, each with its own strategy which it was allowed to modify in response to what was going on around it, for example if it saw that other agents had more successful strategies.

Over the long term, Axelrod discovered that selfish strategies tended to do very badly, as did foolishly generous strategies. Defecting encouraged others to defect; not punishing others for defecting only encouraged them to do so again. One of the most successful and stable (in other words, successful against many different strategies and in many different environments) was Tit for Tat. This strategy involves cooperating the first time you meet another agent, and after that always repeating your opponent's last move. So if your opponent defaults on one turn, you punish them by defaulting on the next; if they cooperate on one turn, you reward them by cooperating on the next.

A slightly better strategy — because it avoids the possibility of getting trapped into long cycles of retaliation — is Tit for Tat with forgiveness. This is Tit for Tat with a small randomised possibility of forgiving a defaulter by cooperating anyway. Forgiveness is particularly helpful if you introduce the possibility of misinformation into the game — that is, if moves are sometimes randomly misreported.

The submitted strategies varied in many ways — initial hostility, tendency to forgive or retaliate, complexity, how much past behaviour they took into account, and so on. No one strategy will always be best because how a strategy does depends on who the other players are — if you're playing against mugs, you may as well be a freeloader, and if you're playing against sharks, you may as well get your retaliation in first! And research into human behaviour is ongoing, with biologists, economists and mathematicians studying phenomena such as spiteful behaviour, altruism, and kin selection (generosity between close relatives, which is evolutionarily useful since their genes are similar). But Tit for Tat did well or best in Axelrod's tournament against very many different opponents — showing how cooperation could evolve using only the selfish mechanisms of natural selection.

About the author

Helen Joyce is former editor of Plus.

Guilt counts

by Marianne Freiberger

Guilt, so some people have suggested, is what makes us nice. When we do someone a favour or choose not to exploit someone vulnerable, we do it because we fear the guilt we'd feel otherwise. If this is the case, then guilt is what holds together human society, as society is to a surprising extent based on cooperation and trust. It would be interesting to know what neural processes generate guilt, not just to answer fundamental psychological questions, but also to understand disorders that are associated with an excess or lack of guilt, such as anxiety and psychopathy.

A team of neuroscientists, psychologists and economists have this month produced some new results in this area, using a model from psychological game theory. "One idea is that most people cooperate because it feels good to do it. And there is some brain imaging data that shows activity in reward-related regions of the brain when people are cooperating," says psychologist Luke Chang, one of the scientists behind the research. "But there is a whole other world of motivation to do good because you don't want to feel bad. That is the idea behind guilt aversion."

To test this idea the researchers used a commonly-studied mathematical game, called the Trust Game. It involves two people, an investor and a trustee. The investor gives a certain amount of money to the trustee. The amount is then multiplied by some factor, usually 3 or 4, and the trustee then has the chance to return some, all, or none of the money to the investor.

How can we predict how people will behave in this game? The traditional approach to game theory assumes that players are rational and completely selfish. Under this assumption trustees keep all the money, abusing the investors' trust. Realising this, investors don't hand over any cash in the first place, so no interaction takes place at all.

In this new study the researchers replaced the selfishness assumption with one of guilt aversion. They assume that the trustee simultaneously tries to maximise their financial pay-off and minimise the guilt they expect to feel if they let their partner down. Guilt is defined as the failure to meet the partner’s expectations. So if the investor expects to get an amount E_1 back from the trustee and the trustee returns S, then guilt can be quantified as E_1 - S if S < E_1 and 0 otherwise.

However, in a realistic situation, the trustee does not know exactly how much money the investor expects, instead they will base their decision on the amount of money E_2, they believe the investor wants back. So a better quantification of guilt is E_2 - S if S < E_2 and 0 otherwise.

Now suppose the investor hands over a certain amount of money, which gets multiplied to give a total T. After returning S, the trustee is left with a pay-off of T-S. The idea that the trustee tries to simultaneously maximise pay-off and minimise anticipated guilt is captured by a utility function U, which measures how happy the trustee feels with the interaction. It is defined as the trustee’s financial gain minus a term measuring the guilt they feel:

U = (T - S) - ϴ(E_2 - S).

The number ϴ, which is positive, measures just how sensitive the trustee is to feeling guilt: the higher ϴ, the heavier the guilt factor weighs in. It is different for every trustee-investor pair, reflecting the fact that some trustees are more conscientious than others and that the amount of guilt we feel when abusing someone’s trust depends on who we’re dealing with.

The trustee tries to maximise the utility function, that is, he or she looks for the value of S which gives the largest value of U. This value gives the best possible trade-off between pay-off and guilt.

If you plot U versus S for fixed values of ϴ, you get two different types of graphs. For ϴ < 1 the graph has a maximum at S = 0. Thus, the model we’ve just constructed predicts that trustees less sensitive to guilt maximise their utility function by returning no money at all. For ϴ > 1 the graph has a maximum at S = E_2, predicting that guilt averse people will return the amount they believe the investor wants back.

Suppose the total T = 40 and the trustee believes that the investor expects to receive an amount E_2 = 20. The plot shows U versus S for ϴ = 1.5 (blue) and ϴ = 0.5 (pink). For ϴ = 1.5 there is a maximum at S = 20 and for ϴ = 0.5 there is a maximum at S = 0.

The researchers tested their model on 30 volunteers who played repeated rounds of the trust game. On the whole, the volunteers behaved as the model predicts: they typically returned close to the amount they believed the investor expected back. After playing the games, they also reported that they would have felt more guilty had they returned less. This, so the researchers say, suggests that anticipated guilt really does play a role in decisions to cooperate.

The researchers also used fMRI scans to monitor the brain activity of trustees during games. They found that participants who chose to honour trust by returning close to the amount that was expected of them showed increased activity in one network of brain components, while those that abused trust showed increased activity in another network. These two networks compete, but on the whole the network connected to honouring trust wins out.

An fMRI image showing areas of the brain associated with the competing motivations of minimising guilt (yellow) and maximising financial reward (blue) when participants decide whether or not they want to honor an investment partner's trust. (Image courtesy Luke Chang/UA psychology department.)

The results chime with existing evidence. Previous studies have linked the brain network associated to an abuse of trust to neural processes that compute value and reward. The network associated to honouring trust has been linked in previous studies to feelings of guilt, anger, social distress and empathy for others. "These studies support our conjecture that the prospect of not fulfilling the expectations of another can result in a negative affective state, which in turn ultimately motivates cooperative behaviour," say the researchers in their paper. "Perhaps the function of this frequently observed network is to track deviations from expectations and bias actions to maintain adherence to the expectation such as a moral rule or social norm."

So the study suggests that when you do someone a favour without expecting anything in return, it's because the relevant parts of your brain signal that falling short of the other person's expectations would lead strong feelings of guilt. There's a caveat though. Rather than guilt — feeling bad for not meeting expectations — the driving emotion may be empathy — the ability to "feel" the other person's disappointment when their expectations aren't met. It's a subtle difference and more work is needed to prise apart these two emotions.

The game theoretical model used in this study may seem surprisingly simple, but it's got an edge over traditional models used in economics, which have often been criticised for failing to take account of human nature. They usually assume that "players" are rational, self-interested and undeterred by complex social emotions such as guilt. This is where the inter-disciplinary approach involving both psychologists and economists can lead to useful results. "In the end, it's a two-way exchange," says economist Martin Dufwenberg, who co-authored the study. "Economists take inspiration from the richer concept of [humanity] usually considered in psychology, but at the same time they have something to offer psychologists through their analytical tools."

Further reading

The study, Triangulating the neural, psychological, and economic bases of guilt aversion by Luke Chang, Alec Smith, Martin Dufwenberg and Alan G. Sanfey, has appeared in the journal Neuron.

About the author

Marianne Freiberger is editor of Plus.

Trust me, I’ve evolved

by Marianne Freiberger

Why are some people generous and others selfish? There's no doubt that both strategies pay off under certain circumstances, but research (as well as everyday experience) shows that we are not mere opportunists — some people simply are nicer than others. This raises a question which intrigues evolutionary psychologists: is there a selective force that works in favour of a wide range of personalities, preventing us from all evolving the same optimal character trait? In 2008 a possible answer was suggested by mathematicians from the universities of Bristol and Exeter. Their game theoretical model of human interaction shows that diversity, once it has taken root in a population, can be self-perpetuating and ensure its own existence for generations to come.

"What we found is that watching each other's behaviour produced individuals who were more socially aware," explains John McNamara, who led the research, "which in turn exaggerated the personality traits of both players. Some became more cooperative — because they became aware of the impact their decisions were making on their reputations — while others became less cooperative and exploited trusting individuals for personal gain."

McNamara's team, which also includes Philip Stephens and Sasha Dall, work in evolutionary game theory, an area that over the last few decades has gained popularity with people who study evolving systems, including economists, biologists, sociologists and psychologists. As the name suggests, the theory likens inherited traits to strategies in the great game of evolution. An advantageous trait has a positive payoff in terms of fitness to survive and have offspring, and this is reflected in a larger proportion of the next generation having that particular trait. Using simple mathematical rules to describe the payoff and replication of traits in the next generation, researchers can sit back and let a computer work out whether a particular trait survives, becomes dominant, or eventually dies out.

McNamara, Stephens and Dall based their model on a simple game of trust and cooperation. Imagine that two individuals, call them Alice and Bob, have been thrown together and have to decide whether or not to cooperate on a certain task. Alice is the one who has to make the first move. She has to signal whether or not she is willing to trust Bob. If she does, and Bob reciprocates, then both receive a certain payoff, which is higher than the payoff they would both get if Alice had decided not to trust Bob. If, however, Bob decides to defect once Alice has decided to trust him, he will receive a higher payoff still, while Alice will get nothing.

As it stands, only a very naive Alice would decide to trust Bob, since he doesn't have much incentive to cooperate. In terms of evolution, you would expect naive Alices to eventually die out, which would in turn make altruistic, cooperating Bobs redundant. But now imagine that social awareness enters the game. If Bob is aware that he is being watched by future Alices, he may well decide to cooperate if trusted, for if he doesn't, he may never be trusted again and therefore always lose out on the higher reward. If Alice has some way of finding out how Bob has behaved on previous occasions, she may be more willing to trust. Being a trusting Alice and a cooperating Bob may become viable evolutionary strategies. The question is whether Alices with varying degrees of trust and Bobs with varying degrees of trustworthiness can exist stably alongside each other.

To find out, the researchers imagined a population of individuals who are randomly paired off in Alice-Bob situations. Each individual plays the role of Alice in some interactions and the role of Bob in others. When cast in the role of Alice, an individual always behaves in the same way, according to an inherited trait: some always trust Bob, some never trust Bob, some trust Bob if he has been seen to cooperate in at least one of n previous occasions, some trust him if he has been seen to cooperate in at least two of n previous occasions, and so on. The latter classes represent Alices with social awareness and varying degrees of trust. When cast in the role of Bob, each individual may or may not cooperate if trusted, and the individual's chance of cooperating is measured by a number p between zero and one. The continuously varying values of p represent Bobs of varying degrees of trustworthiness. The researchers assumed that an individual's degree of trust, when being Alice, and trustworthiness, when being Bob, are independent. This isn't entirely realistic, but it frees the researchers from having to keep track of the association between genes that control the traits.

Each interaction comes with a certain payoff for the two individuals involved. The exact value of this payoff depends on the individuals, but using statistics it's possible to calculate a mean payoff, a sort of average, for the traits in each role. For example, when acting as Alice an individual who uses some social awareness to assess Bob may have a higher mean payoff than one who trusts Bob unquestioningly. Similarly, an individual who in the role of Bob has a 50% chance of cooperating may have a higher mean payoff than one whose chance of cooperation is zero. These mean values are used to calculate the proportion of the next generation that display a certain trait: the researchers devised a formula which ensures that the higher the expected payoff for a trait (such as the degree of trust in an Alice, or the degree of trustworthiness in a Bob), the higher the proportion of the next generation with that same trait.

With these rules in place, it would be possible to calculate how the various degrees of trust and trustworthiness perform over many generations, but there is another important issue to take into account: in real-life evolution individuals don't just acquire traits through inheritance, but also through mutation. This is important because a society of nice altruists could be turned on its head within a few generations by the influence of a selfish mutant. The researchers built mutation into their model by introducing mutation rates which slightly adjust the replicating formulae for the next generation.

The next step was to set the computers chomping. Starting with some initial distribution of traits and values for the various parameters that define the game — the payoffs that result from trusting and cooperating, and the rates of mutation — the researchers waited to see how the traits fared generation after generation. For certain initial values, the proportions of individuals with the various traits eventually settled down — they hardly changed at all from generation to generation. Such an unchanging configuration of traits was deemed to be evolutionary stable.

Unsurprisingly, when the researchers initialised their model without social awareness — with only universally trusting or untrusting Alices — the number of trusting Alices became negligible, while the most popular chance p of cooperation was zero. Without social awareness, it is impossible to maintain trustworthiness and trust. However, in stable situations that involved social awareness, two types of Bob could be seen to exist alongside each other: a non-cooperating type, and a type with quite a high degree of trustworthiness. Universally trusting Alices were also seen to exist alongside universally untrusting ones, and those who display social awareness and varying degrees of trust — a whole range of character traits existing in an evolutionary stable situation.

The researchers put the phenomenon down to a positive feedback loop: "Variation encourages social awareness," explains Dall, "which favours greater personality divergence, maintaining the need for social awareness. In other words, because people operate in such different ways, we need information to decide whether or not trust to them. This encourages a really diverse range of responses which, in turn, makes social awareness all the more important."

Real-life people and their interactions are of course a whole lot more complicated than the virtual individuals in the model. But what the research shows is that diversity is no mystery: it can arise from the very rules of the game and maintain itself. The research also shows that models of human behaviour, used widely in economics, can no longer ignore complex and changing interpersonal relationships, as they have tended to do so far.

About the author

Marianne Freiberger is editor of Plus.

But wait there’s more...

We hope you’ve enjoyed this ebook.  You can find out more about this and other topics online at Plus.  You can also tell us what you think and find many more ebooks to download at http://plus.maths.org/ebooks.  We’d love to hear your feedback!