Friday, September 12, 2008

Heckman on the econometrics of prayer and God

So I was trolling the IZA website for new working papers today and found a really interesting little paper by James Heckman - prolific Chicago economist.

It estimated (if I understand him right) the effect of prayer on God's attitude towards man, by assuming that prayer is an increasing function of God's attitude towards man. As the attitude becomes more positive, people pray more. You can derive God's attitude towards man as a function of prayer from this prayer function and the population distribution of prayer (as captured by survey data).

I'm still a little confused by how all this works, but Heckman is a smart guy. I'm sure his math is right, I'm just not sure what implicit assumptions he's making to make the math right.

But it does make me wonder - we've seen all kinds of studies about how "prayer makes people in hospitals heal faster" - but you don't see social scientists doing this much. You don't see anything on the effect of prayer on neighborhood crime, or income, etc. It would be an interesting exercise simply because of what it would take to identify an unbiased estimate. After all, conceivably you'll pray more if you have a higher inherent likelihood at having a rough go at things - so that should negatively bias the effect of prayer. You'd have to find some instrument or exclusion principle predicting prayer. But it could work and that would be an interesting thing to write about.

But even once you get the unbiased estimate you still have Weber's conundrum - did you really estimate that Providence was shining down on the faithful, or did you just pick up some normative or cultural artifact that is positively (but spuriously) correlated with the prayer itself. And again - simply the exercise of working through these possibilities would be interesting, but its doubtful you could come out with anything conclusive.

So why don't more economists do this? Why don't we look at the effect of prayer intensity in different metropolitan areas on the performance of sports teams? Or national religiosity and whether they're victorious in a war? Or just a simple "health and wealth gospel" look at the effect of religiosity on earnings. I'm sure we've had studies that say "religious people on average earn X more dollars than non-religious people", but that's a different endeavor from trying to identify an unbiased estimator for the causal effect of prayer.

And of course I forgot to mention - Heckman's results! He finds a positive effect of no prayer on God's attitude, with a negative effect of some but very little prayer, and a positive effect of a significant amount of prayer.


Evan said...

You social scientists are silly. Why approach something in a scientific manner with methods inadequate for their object? Until Heckman can verify in any reliable sense God's attitude towards humanity, it's a fun but useless exercise. You don't even need to make a gesture to the correlation/causation issue... this applies to miraculous healing and football games, but it doesn't even apply to the question as Heckman has posed it.

Looks interesting, though. Hope I haven't burst your bubble by not being too impressed!

Evan said...

Okay, so I don't get this at all.

Could you explain in general what he's trying to say? If Y is an unobserved variable, how is he using it? What does his scale correspond with? I'd be interested to check out Greeley... he's a well known Catholic sociologist, but it's been a while since 1972- I wonder if there are more update sociological examinations of prayer.

dkuehn said...

ya - economists do goofy stuff more than you'd expect, actually.

the point he alluded to in the paper is that you can estimate an unobserved variable from the distribution of an observed variable if you make certain assumptions about the functional form of the observed variable. I don't think it's meant to be any kind of "proof of God" or anything - its just a neat math trick that may have applications elsewhere... or if not, its just a neat exercise.

i'll read it again later and see if i can explain it better... I'm not sure I even totally got it all.

dkuehn said...

This is hopefully a clarity post, Evan:

RE: "If Y is an unobserved variable, how is he using it?":

There are lots of unobserved things out there that we wish we could study - at the end of the paper he mentions two others that economists have worked wtih a lot - happiness and democracy. How do you measure those??? Think of physics, though, and early atomic researchers. Remember that experiment they did where they shot some atoms onto a think gold sheet and they could see the effects of the atom and derive information from that? Same idea here. If we are willing to make assumptions about how the unobserved variable interacts with an observed variable, then we can look at changes in the unobserved variable by tracking the observed variable. The question, of course, is "how good are the assumptions you make" - and obviously that's where this little exercise will probably fail as a "proof of God". So it shouldn't be lookedd at that way.

RE: "What does his scale correspond with?":

I know this won't satisfy you, but it doesn't matter. Its just a scale he created ranging from 0 to 1. Obviously, a zero to one scale has a lot of desirable mathematical properties (because the points no the scale are a proportion). So a lot of people use this. I'm looking at job matching models now where they assume a "job search intensity" between 0 and 1. The neat thing is, its easy to reduce other scales to a 0 to 1 scale. For example, in a logistic function: "z = (e^x/1+e^x)", z will have the exact same ordering of values as x will, but where x can have any range, z is forced to be between 0 and 1. So that's how a lot of people transform scales - or something similar to that. So z isn't a perfect representative of x, but it is great for answering questions like "when x increases what happens to y" if x and y both have to be converted to a zero to one variable.

RE: "You social scientists are silly. Why approach something in a scientific manner with methods inadequate for their object?"

Our methods are almost always inadequate for their object. Think about how many forces cause an object to move through space... pretty complicated, huh? I remembere physics - it wasn't a cakewalk. Now think of how many forces cause the stock market to fall, or lead to someone having a certain income level. EXTREMELY complicated. Half the game of empirical economics is justifying an imperfect solution and arguing that although it's biased it's a lot better than what anyone else has done, and it approximates the correct solution. Its also nice if you can say "I know it's biased and I think its biased in this direction about this much". That's just how the game is played. You're never going to be able to observe everything that's going on or measure everything so you invent mathematical tricks like this to work your way around it. Econometrics is full of mathematical shortcuts like this - instrumental variables, regression discontinuities, propensity score matching, etc. None of these are wrong - they're exactly right if you are willing to assume certain things about what you do and don't observe. If those assumptions are credible, then you have a damn good paper.

RE: "Until Heckman can verify in any reliable sense God's attitude towards humanity, it's a fun but useless exercise."

I would argue that it's fun, but not useless. He specifically mentions applications to happiness and democracy at the bottom. A lot oof work on both of those has been done in economics, and this could help some of that... and since we have surveys where people claim how happy they are, we can use this technique to try to predict the unobserved variable "happiness" and see how well it matches to their claims - further advancing that line of research. But the point is, this Father Greely and the Singh guy came up with this approach with work on prayer, so Heckman has some fun goofing around with prayer data first. I don't think he was intending to say anything conclusive about the power of prayer - it was just a neat exercise.

RE: "You don't even need to make a gesture to the correlation/causation issue... this applies to miraculous healing and football games, but it doesn't even apply to the question as Heckman has posed it"

Why not? Conceivably, prayer could cause God to change his attitude, or God's attitude could cause people to pray... but prayer could also be correlated with a Protestant work ethic which causes economic success - and people have also found economic success to be correlated with God's positive attitude. Doesn't have to be right - but this causation/correlation conundrum is believable enough that anybody writing about the power of prayer will HAVE to grapple with it. And not just economists either - health and wealth gospel types have to justify why what they're saying is true, and its not just the Protestant ethic or something similar (maybe just "the power of positive thinking") at work.

RE: "economists do goofy stuff more than you'd expect, actually"

Freakonomics is another book that's filled with these kinds of exericises. He goes through totally random things, but a lot of the point is to explore new ways of using data. And like this paper, its a fun read in the process.

Evan said...

Hey. I have Sophia on my lap so I don't know how long she'll be still and allow me to write! If I don't finish here I'll chat more later.

As always you speak wisely of those damned social sciences... can't argue with your explanation of its usefulness, I only hope more of your ilk use their tools in the same way.

On the 0-1 scale, I understand the use of the scale, and don't have a problem with that. I know that's a normal enough thing. I guess what I didn't understand from his paper was, how does he derive x to plug it into the equation in the first place and get a z between o and 1? I suppose I may just have to read Greeley for that. My question was more how he got his numbers, rather than the scale itself.

On the correlation/causation issue, my point was that you have a solid score at a football game, or you have a clear ability to know whether a patient is healthy or not... and at this point the question of cause or coincidence comes up. But with this example, the economist is speaking of the "attitude" of God towards humanity (or "mankind", in his words... you economists aren't ones for gender neutrality, eh?) But how do we even know God's attitude in a way that would allow us to come to the question of causation v. coincidence with prayer? God allows rain on the just and unjust alike, eh? Just semantics, I know... but papers like this invite interesting dialogue, right?

Whew, got to say everything. Sophie is sprawled across me asleep, so I was free to type.

dkuehn said...

1. Oh ya - it was survey data that gave him the distribution of prayer in the population. NORC produces that stuff - I think they mention it in the subtitle.

2. That's what's assumed, though - the functional relationship between God's attitude, Y and prayer, X. and that's the underlying relationship you have to accept.

il blog di Valentino said...

Thank you for posting this. I would like to know if you have more information about the math used, and the assumed realation between the observed and unobserved variable.

Economics Specialist said...

I am very thankful to you that you post this article and i hope that you also post the relationship between the quantity and price. I am waiting for your this topics.

Anonymous said...

The paper is meant as a joke. Heckman is being tongue in cheek. He is trying to show the danger of making parametric assumptions to "identify" something that is inherently unobservable