Policy, as in many areas of life, is about tradeoffs. To take a simple example, consider arguments that some conservatives and progressives might make regarding welfare. I’ve heard friends that lean progressive say things like “How can we let someone who is really trying and down on their luck go hungry? We need to increase the availability of SNAP [food stamps].” On the other side, I’ve heard friends that lean conservative say some version of “I’ve seen people who get food stamps just waste them on things like cookies, cake, soda and chips – we need to reduce their use.” Who is right?
The answer, of course, is that they both are. People come in all shapes and sizes. They also vary in their behaviors, values, and ethics. This is what makes policy so difficult: you have one policy, but how people behave in response to that policy can vary widely.
There are various ways to deal with this. One is to refine the policy. For example, current SNAP policy does not allow folks to buy alcoholic beverages with those funds. This can work well when there is fairly broad agreement that such a refinement makes sense. But this can easily end up getting very complicated as you attempt to refine further and further until you end up with a complex mess that is difficult for the consumer to understand and for the regulator to enforce, and where the interaction effects between the various rules can cause unintended outcomes. (Tax policy, anyone?)
Error Types
Beyond some basic “common sense” refinements, however, a better approach is simply to acknowledge that any policy is going to have some “error” in it, and to ask which type of error is more acceptable, and how much? This is basically the same thing as thinking about Type I and Type II model errors in hypothesis testing.
Using the example above, would you rather someone get food stamps that didn’t really need them or not give someone food stamps that really did need them? To be clear, not everyone may agree on the answer to this question, but at least we’re now starting to have a real conversation.
Let’s say you think that it’s better to err on the side of being generous, even if it means some abuse of your generosity will happen. What ratio are you wiling to accept? For example, if there’s one abuse for every 10,000 people you truly help, that seems reasonable. What if it’s 5 people helped for every 1 abuse? 1 to 1? What if it’s 5 abuses for every 1 person truly helped? What if it’s 10,000?
Standards of Proof
Some parts of our legal system are already explicitly like this (or at least try to be). In the justice system there are known various ‘standards of proof’ that are required depending upon what’s going on. For example, a police officer is required to have a ‘reasonable suspicion’ before stopping and questioning an individual. A ‘probable cause’ is required to issue a search warrant or arrest someone. A ‘preponderance of evidence’ or ‘clear and convincing evidence’ is required in civil court (and sometimes in criminal). And ‘proof beyond a reasonable doubt’ is the standard required for a criminal charge.
By placing such a high bar for evidence, we as a society have made the choice that we would rather let a guilty party go free than convict an innocent one. According to Wikipedia, it is estimated that between 2.3 and 5 percent of all U.S. prisoners are innocent. Is that an acceptable error rate? That’s an open question, but at least its a tractable one.
I’m not saying that the details of individual policies don’t matter. Clearly they do. And of course there are other real considerations, such as cost. But when there is disagreement it may help to start the conversation by asking “which type of error are we more willing to make?” and “by how much”?