Looking at studies that are critical of energy efficiency: identifying useful findings and where they fall short

Several papers in the last few years claim to show that particular energy efficiency programs and policies do not work or are too expensive. We have commented on some of these in blog posts (see here, here, and here), noting that some of them do have useful insights, but also that many of them make serious mistakes.

Have we turned the corner on criticisms of energy efficiency programs and policies?

In recent months, there have been signs that such criticisms may be abating or softening. For example, the E2e project, a source of several of these critical studies, is turning toward some commercial and industrial programs, which tend to have lower costs than the residential programs they have been criticizing. Initial results from one of their residential studies (abstract available here under Allcott) have found that residential weatherization programs are of borderline cost-effectiveness when considering only energy cost savings. This finding is consistent with other studies on these programs (see residential retrofits here), leaving researchers to debate the other benefits of these programs such as improved resident comfort and health.

Unfortunately, not everyone is taking a more measured approach. There are some questionable studies about automobile fuel economy that we will be commenting on in a forthcoming blog post. And, just last week, a study was published by the Fraser Institute in Ontario implying that efficiency programs there are not cost effective. Their conclusion is not based on data from Ontario, but cites previous critical studies of other programs, in particular a controversial study from 1992. Their analysis ignores or downplays other more recent studies that found much lower costs.

New ACEEE paper

Since we expect some studies like this to continue, we thought it would be useful to prepare a paper reviewing a few of the recent studies, identifying useful findings, but also pointing out a variety of recurring mistakes, such as misunderstanding the programs and markets they are examining or unreasonably extrapolating their findings to areas they did not study. This short paper, which we are releasing today, is written for people who are not evaluation experts, but who are trying to understand what conclusions they should take from these studies. That said, we acknowledge that not all energy efficiency programs are stellar, and that sound evaluation is critical in identifying what is working well and what needs improving.

We conclude our new paper with suggestions for constructively moving forward, such as the need for academic economists and energy efficiency evaluation experts to better understand where the other side is coming from, and to explore opportunities to find a middle ground, potentially even working on some projects together. With cooperation and combined expertise, energy efficiency studies can achieve their intended purpose: to understand what works, and to improve what falls short.