Adversarial systems, fake news and decision making

How we set up adversarial systems to externalise our thinking

I

Making decisions is hard. It needs you to have a clear view of what is present today, the factors that might affect it, how people might perceive the decisions you make, and optimise amongst all of them. Turns out, this is kinda tough. You can't ask people to think about everything.

So what do you do? You externalise some of the thinking needed. That's where laws, and rules, and norms come into place. They limit the search-space within which you need to apply your brain and figure out your best strategy. Let's look at a few examples:

• Driving on a highway is hard. You need everyone to show some level of cohesion, and that is a coordination problem that can only happen by fiat. If everyone tried to figure all of it out themselves there would be chaos. So instead we have speed limits and road rules. So if your aim is to get somewhere as fast as you can, you can still do that, but you need to plan a bit differently. You choose your lane wisely, or take the right side roads with less traffic, or don't zigzag like a maniac. The objective hasn't changed, but the strategies have.

• Treating patients in a hospital is hard. Even if you're a crack doctor with a great medical team, you have clear (human) limits on what you'll be able to do, compared to the tsunami of patients that come your way. You can't solve everything for everyone, starting from the first visit. So you create a system of specialists and a triage-pyramid. This helps navigate the patient towards the place where they'll get the best care they need in the best time. There will still be time when you need to go all the way down that pyramid and start from scratch, but only for a few and not for everyone.

• Tax systems are hard. Nobody wants to pay them, but the government can't run without it (not looking at you Saudi Arabia). You can create some rules about what people ought to pay, but that gets tripped up pretty soon. Just trying to figure out all the rules from scratch is a pain in the ass, both to the rule-setter and to the rule-follower. But what if there was a relatively sensible system set up upfront (not looking at you USA). You could try optimise within it a little bit and take some breaks that the society sees as 'desirable', but otherwise your goal is to play within the rules and optimise what you can.

• Playing sports is hard. For one thing you have to extremely athletic and in top physical shape. But you also need to be quick on your feet and think of smart manoeuvres. But you can't play, say, basketball if everyone doesn't agree on what the definition of 'basketball' is. If the goal is just to get the ball inside the basket at the other end, you could just bring twenty tall players and hand it off one to another. So you make some rules. And you make some more rules to figure out how to apply those rules. And the teams that win try to figure out ways of playing within those rules (e.g., take more 3 point shots ffs) that can still get them to win.

• Running a business is hard. You have to make your employees, your customers, and your owners happy. None of them are an undemanding bunch either. And the pressure is on you to make this happen, get the profits to go up and to the right, otherwise it's your ass on the line. Sure, you could make this happen by doing all sorts of unethical stuff, but that's ... well ... unethical. So there are regulations put in place that say things like don't employ children in your factory, don't dump your pollutants into our drinking water, try not to spread chemicals in the air, and so on. And within those regulations, you try to continue to win!

• Politics is hard, especially running for office. Everyone wants something. Donors want access, constituents want their voices heard, and you and your staff want some recognition as being all around awesome people. Sure, you could start from the basics by asking everyone what they want and start to make a list, but that'll take forever and like painting the Golden Gate Bridge by the time you're finished you'll have to do it all over again. So what do you do? Join a party with some pre-established norms that everyone broadly seems to like, and then you can build and tweak it a bit to fit your situation. And you use those tweaks to win as much as you can.

I could probably go on with most of the things we do in society. But these are probably enough. Every single one shows the problem we face regularly. When we ask things like 'why can't Facebook police its own behaviour and kick out the neo-nazis', or 'why do these people vote against their self interest' or 'why did Exxon keep misleading us about climate change'.

The system is not set up to make them do that.

And what's more, that's not a bug, that's actually the feature.

II

So why is it a feature? Most systems we have set up for ourselves are adversarial in nature.

The theory that underpins much of democracy and most of economics lies in the theses of comparative advantage, where each party should focus on doing what they are best at, and argumentation as a path to truth, where each side tries to win and their competition allows a third party to actually choose the correct side.

The idea is that we can't all do things ourselves, so why don't we try to find a way to do what we are good at, and let someone else be in charge of telling us what we can and cannot do.

We don't choose every single one of the laws that we are supposed to follow. Mostly we just follow what's been told to us, and try make tweaks and changes in the direction we think it ought to go. And in order to make those changes we fight tooth and nail, against adversaries who also fight tooth and nail.

The idea is that debate is good. And that adversarial debate systems allow each side to present its strongest arguments so that we (collectively) decide on the course of action.

Is this wise? Seems like it, at least so far. More importantly, it's unclear if there are better ways to actually accomplish what we want to do. If we want the best ideas to actually win out, we have to allow the best ideas to fight against all ideas and see if they actually win.

The problem with the system is that there is an overwhelming level of interest on behalf of both sides to try change the rules, bend the rules, or selectively break them, in order to win. After all, like every toddler knows, one definite way to tie a game of chess is to knock over the board and then argue over who won.

There's a second problem. We know that nobody's got the time to figure out the veracity of claims people make. That's why they try and figure out whom to trust and just go with that. There's an issue in epistemology about how to figure out what 'truth' is. Turns out it's bloody hard, even for the more trivial things. Reading the papers it's a wonder any of us can even wake up and have a cup of coffee without falling to the ground clutching our heads.

Our reliance on particular sources for information and reliance on particular people to carry out their duties are not 'dumb'. Nor are they a symptom of lack of discernment or lack of education. There is a flippant response whenever someone gets fooled by a Madoff or a politician that the issue here is a lack in our schooling system. But it belies the fact that everyone relies on particular sources of information for their news. Everyone relies on people.

When my faucet leaks, I trust my plumber to be able to suss out the problem and fix it. When my computer breaks, I can mess about with it, and I know a bit more than plumbing, but I still trust the repairman to do what's needed. If she says I need to change the motherboard, I'm not going to second guess that. If this happens often enough I might try find a replacement. But even in that case, how would I do that? Believe in online ratings to find a good one, or believe in the recommendations of my friends?

We can't get away from the problem of unreliability through insistence on 'doing your own work', because that's not how most of us live most of our lives. We live lives of strong interdependence on each other. With a high degree of trust and overall we are not the victims of some con. That overall we are not being lied to constantly.

You could always be the person with the tinfoil cap and try think from 'first principles' about everything. It will be better for some things. But it is not a panacea. Nor is it something that is even remotely plausible.

When we move media paradigms our old intuitions die and new ones have to be born. When word of mouth got replaced with pamphlets we had to update the believability of strangers for believability of friends. When newspapers were popular we had to figure out which papers to trust, and on what. When radio got popular we had to learn that friendly sounding voices could lie and we needed to withhold judgement. When TV got popular we learnt to trust our eyes about what could constitute truth, and got wary of sweaty politicians. And when social media got popular, we haven't quite figured out what that meant yet.

Because it took us back full circle. It is teaching us to be wary of word of mouth all over again. It's trying to reactivate our atrophied muscles regarding pieces of information spread to us by our friends and family. Our predisposition to trust the sources of information is what's been weaponised. And it's hard, because the problem isn't just that some pieces of information is false. It's that they're coming to us from highly trusted sources. When your brother or sister or mom sends you something, that comes with a check mark of veracity.

III

What this means is that we are entrenched in a world where the information we get is unreliable, there is immense levels of noise, where most systems are set up such that we're reliant on them policing each other to work, and fighting to change, bend or break the rules is part of the game.

What this also means that the loss here isn't just one of information, or disinformation, or even antagonistic and bad-faith actions. It's that trust has fundamentally been lost.

If you cannot independently verify each action a counterparty is taking, trust is the only solution.

If you cannot figure out which pieces of information to actually put your faith in, then trust in a source is the only solution.

And trust, weirdly enough, presents itself as a barbell. You trust those closest to you, and those in your immediate sphere of knowledge. If you're a rural schoolteacher, then you would trust those immediately around you (you might not believe them, but their thought process is at least not opaque). You would also trust yourself in matters that affect you, whether occupational such as school funding or personal. However you would also trust those much further away if you believe they are acting in your interests. If someone who represents your county, your district, your state, has consistently given you what you think you want, they are the one you trust.

Disproving this with statistics is impossibly hard. You spend as much time convincing someone that your statistics works as on the issue itself, and that's if you're lucky. You could argue the order of the facts, but that leads to a complex fight about what started what and who's the first mover, which is almost impossible to answer even in simple cases like two kids fighting. Or you could argue the weight of the facts on each side saying in total, one sides clearly outboxing the other, but that then leads into questions of measurement and proportionality.

Disproving it with stories is harder. For the first you have to convince someone that your statistics is better than their lived experience. And for the second you have to convince that your stories are equally or more real than theirs.

There are biases at work here. But that doesn't make it unique. Biases are at work everywhere. People want to believe good things about themselves, they don't want to be made fools of, and they want to put their trust in something or someone who at least looks like is on their side. And just because they picked a side doesn't mean they are onboard with everything that person does either. It just means that trust oftentimes has to be earned.

In politics this is a lesson that gets taught again and again. The same people who voted in droves for Obama are now called racists because they voted for Trump. It can only be that they don't care about the people calling them racists, because clearly they have no faith in them. They have no faith in media, whose narratives seem hyper partisan. They have no faith in the government itself, so might as well vote in a manner that soothes themselves.

Parenthetically, voting is a weird act. It's highly asynchronous, infrequent and with a winner takes all dynamic, which makes it hard to use it as a method of approval or disapproval of any particular act in office. This isn't particularly different from other walks of life either. In business, quarterly board meetings aren't enough to act as a direct moratorium on specific decisions, but become referendums on management instead.

The answer to a lack of trust is not a 'coming-together'. It's an adversarial system. Trust has to be earned. And for trust to be earned it has to start from some semblance of authenticity. The one common thread amongst every successful politician is that the voter can believe that they are who they say they are. Authenticity is essential.

In the business world, the same thing applies. Facebook gets a lot of ire, but they are playing by the rules set down by the society. It's not their job to both create the rules that police them and to play by them. If their job is to maximise shareholder returns, then the world needs to tell them which guardrails to play within. Not as a punishment, but as setting out the rules of the road. Speed limits aren't a punishment for drivers, they're what keep everyone safe. Without that, there is no trust. Facebook's measures to police themselves will never be trusted. Ergo, there is really only one solution. You need some level of authenticity, an acknowledgement that companies focus on what they do well, and they are reliant on the world to set their guardrails.

We could ask ourselves the question as to whether adversarial decision making and trust-based systems are the right way for us as a society to decide things. And those are good questions. But until now, nobody has come up with a better solution. We might be able to tinker and manage things at the margins through technology and clever social engineering, but ultimately we won't be able to wash our hands off the process by pretending it is something other than it is. Utopia was never the goal, and perfection cannot be the enemy of reality.