No Developer is an Island
People behave with hidden motivations and flawed reasoning.
Let’s look at one of the underlying causes behind why many employees, teams, and companies don’t behave the way we want: the principal-agent problem.
“It is difficult to get a man to understand something when his salary depends on his not understanding it” - Upton Sinclair
The principal-agent problem arises when the person/group asking for help (the principal) has different incentives from the person/group offering help (the agent). It is even more common in situations where the agent has more expertise, such as a hired professional or specialist.
The reason is that behavior, both individual and collective, changes to follow incentives. Individual will, morality, ethics, and integrity are all altered by circumstance and incentives. The principal-agent problem is already part of your life:
Going further, you find examples of the principal-agent problem leading to control fraud:
It is so pervasive that economics majors study its theory. I see the principal-agent problem as a force to be fought and minimized. We each have great potential to change our situation and the world around us, especially if we work hard and think carefully.
“Know your enemy and know yourself and you can fight a hundred battles without disaster” - Sun Tzu
To solve a problem you must know your options. To know your options you must know the context.
“The best revenge is to not be like your enemy.” - Marcus Aurelius
You can fight the principal-agent problem with data. Since it often involves information asymmetry , having more information is empowering.
A powerful way to use data is comparison shopping. We have options to acquire goods & services, and vote with our wallets. Doing so in an informed way helps you choose a superior product/service and support companies whose practices you like.
Luckily for principals, the amount of data and number of skilled analysts are increasing rapidly. However, the vast majority of valuable data is not public. Imagine if detailed insurance company data was public: customers would rapidly switch to the most customer-friendly companies. Companies seeking to maximize profit will hide critical data. Therefore we must get information from unconventional sources or public institutions.
A brilliant report by Stephen Brill found that the medical costs are insane. This has been corroborated by many other independent journalists and bloggers. Drug makers make deals to prevent generic (i.e. cheaper) versions of their drugs. It’s sometimes cheaper to pay cash. Visitors from other countries are often appalled by our health care system.
It is easy to skew numbers and hide inefficiency when data isn’t available. Conversely, it’s easy to make systems more efficient when data is available. I’m a big fan of startups that are working on this problem, like Castlight. However, even then the data is not really public.
Data is potentially powerful, and therefore potentially risky. Any individual, group or company with sufficient skill can use data for their own ends. This can be for good reasons, like making health care affordable for the average person, or for nefarious reasons, like figuring out who is more likely to have health issues, so they can be discriminated against (ahem, “priced appropriately”).
Any effort to expose, assemble, or publicize data should involve thought about how it can be used for good or evil. Powerful organizations can handle the risk of embarrassing disclosure. Individuals can’t.
One of the big changes in the past decade is the rise of social networking. Facebook, Twitter, Tumblr, YouTube and Reddit enable a single person to tell their story and have that story visible to the entire world. Amazon, Yelp, and eBay and other sites enable a person to write a review that can influence many future purchases.
Publicity works because of human psychology. We are a social, tribal species. We are identify with people who seem like us. We love the underdog. Sympathy is powerful.
I’ve never met a company or PR agent that is as sympathetic as a normal person. Companies in general, and PR agents in particular, aren’t authentic. Much of that is cultural; the language that agents use is framed by their worldview (profit, marketing, damage control, shareholder value, etc). I’m glad; it makes fighting the principal-agent problem with publicity really easy.
Publicity works for a second reason: unequal risk. Bad publicity costs far, far more than good customer service does. This is doubly true if an agent lies, argues with their customers, and denies things they have done.
Let’s look at two examples.
Individuals (the principals) in this situation have minimal power. The average person can’t afford lawyers to fight police corruption. However, individuals collectively have massive power; they can vote out the politicians running these towns and states. Even the threat to do so (opinion polls) has tremendous clout.
Let’s look at what happened:
Let’s guess that 1% of the 70K Twitter users were also BA customers, and the average ticket costs around $1000.
Let’s guess that each person knows ~150 people. We can guess the impact:
700 People * 150 People in Network * 50% Chance of Mentioning * $1000 ticket price 20% are BA customers = $10.5 million
It’s still not an even playing field. But the odds are better.
Businesses care about money. People care about their jobs. Publicity gives a principal leverage against an agent’s primary weak spot: their wallets.
While researching this post, I found that most successful publicity campaigns had common elements:
“Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has.” - Margaret Mead
The third way to fight the principal-agent problem is with other people. Other people have skills, ideas and connections you don’t. A small team is far more capable than an individual.
The Internet makes organizing far easier than ever before. Anyone can make a site, and anyone else can find it. It’s relatively easy to identify competent professionals and bypass layers of middlemen / bureaucracy.
There are already sites that act as online watering holes and gathering places:
Online gathering places are still in their infancy. Here are some examples of what I can’t find:
The tools for Internet-based gathering places are largely mature:
Let’s look at another area with a bad principal-agent problem: finance.
In the US, the financial system is huge and corrupt. Here’s a partial list of what financial institutions have done over the last 10 years:
The power and influence that exists in finance comes from the collective money of the average person (i.e. principals). It’s a horrific example of the principle-agent problem.
Financial institutions are vulnerable to customers moving their money somewhere else. Their influence comes from our checking accounts, credit cards, loans, and 401Ks. Move it, and you end up with better service and less organizational corruption. Win win. One recent effort has been the Move Your Money project.
Let’s look at why credit unions are better for customers than banks.
People behave according to incentives. Moral courage is admirable, and rare. Incentives encourage what they measure, and care must be taken to avoid unintended consequences.
Here are my questions to identify incentives when choosing an agent:
When looking at banks and credit unions, incentives are the big difference. Banks are designed to take money from their customers, as profit. The customers aren’t owners. In contrast, credit unions have aligned incentives. They’re nonprofits. The customers are owners. Plus, customers can vote to fire the the board of directors and top executives.
Here are types of organizations with good incentives:
We are all principals and agents. As principals, we want the best service for the price. As agents, we should provide the best service we can.
Since we are more often principals than agents, we should remember tactics to fight the principal agent problem:
I am privileged to say that I have been chosen to give a presentation at SQL Saturday 212, in Redmond WA, on May 18, 2013.
The topic? Machine Learning for Mere Mortals.
This will be a gentle introduction to the world of machine learning. I will be covering topics including supervised vs. unsupervised learning, clustering, decision trees, association rules, and learning at scale. I shall also provide examples and demos of real-world machine learning applications.
Finally, I’ll provide resources to help people get started.
I hope to see you there!Permalink