How much difference does your policy make?
There are a couple of important lessons that I would like to share with consumers of public policy analysis, lessons gleaned from my own coursework and work experience. The immediate context is the rise of policy analysis in the blogosphere. I strongly believe that ideas have a fringes-to-mainstream transition. And it is a good sign that the pace is picking up within the blogosphere. As always you have a few policy analysis "lemons" out there which need to be vetted, and these "user-friendly" points could help them. I will be glad to receive any criticism of them.
1> You start with asking what is it that makes a difference.
2> How much of a difference does that make?
3> If it makes a difference, is it worth investing in creating that difference?
4> Who is the party best suited to creating that difference?
The first and fourth are based on a qualitative understanding but the second and third have a quantitative approach. An example is the Minimum Wages Act. One could argue from both quarters (a) it makes a positive impact or (b) it makes a negative impact. When you read an article that doesn't measure the impact of the difference, often it may be hard to tell which will prevail. More importantly, most articles don't inform you the cost of making that difference. There are trade-offs everywhere. And the fourth point (subtleties often lost in most conversations) is the nature of the agency best suited to undertake creating the difference. While it may feel good to think that Factor A will make a difference in x people's lives, you need to think about the other three factors again.
* Considering the realistic nature of its implementation, will it create a difference? And a difference for whom: intended or unintended beneficiaries?
* Okay, so what amount of difference will it make for the intended beneficiaries?
* What are the short-term and long-term costs of making the difference? Below is a stylized table to help you think about it.
1> You start with asking what is it that makes a difference.
2> How much of a difference does that make?
3> If it makes a difference, is it worth investing in creating that difference?
4> Who is the party best suited to creating that difference?
The first and fourth are based on a qualitative understanding but the second and third have a quantitative approach. An example is the Minimum Wages Act. One could argue from both quarters (a) it makes a positive impact or (b) it makes a negative impact. When you read an article that doesn't measure the impact of the difference, often it may be hard to tell which will prevail. More importantly, most articles don't inform you the cost of making that difference. There are trade-offs everywhere. And the fourth point (subtleties often lost in most conversations) is the nature of the agency best suited to undertake creating the difference. While it may feel good to think that Factor A will make a difference in x people's lives, you need to think about the other three factors again.
* Considering the realistic nature of its implementation, will it create a difference? And a difference for whom: intended or unintended beneficiaries?
* Okay, so what amount of difference will it make for the intended beneficiaries?
* What are the short-term and long-term costs of making the difference? Below is a stylized table to help you think about it.
Targets | Cost | Benefit |
| Intended | | b |
| Unintended | | |
An idea of a,b, c and d will greatly clarify your thought process of evaluating a suggested policy.
The fourth is the nature of the agency. Operationally it means understanding the incentives of an agency to influence a result.
Most articles that you read online will have qualitative content on the first and the fourth points. So often you have arguments on what to do and who should do it. Usually they result in govts or markets doing X difference. An idea of the measure of the difference X will help clarify issues rather than the initial discourse. It is a tougher question but it will definitely help you from getting misguided. A couple of caveats. First, by measure, I do not necessarily imply a number but a dimension that will help you compare across various actions. Second, governments often fudge their measure of difference by framing it in inputs, at most outputs but usually not in terms of outcomes.
So the next time somebody tells you X will do such and such difference and should be done by Y agency, ask him to give an idea of the magnitude of the difference and the costs involved. That is when you are no longer an amateur consumer of policy analysis!
Cross-posted on the Indian Economy blog.
Labels: policy analysis, primer
posted by Naveen Mandava at 4:50 PM - Permalink
<< Home