A tale of a poorly designed program and overly judgmental evaluators

Blog Post | November 04, 2014 - 9:47 am
By Steven Nadel , Executive Director

The E2e project released a working paper recently that evaluates the State Energy Efficient Appliance Rebate Program (SEEARP), funded in 2009 by the American Recovery and Reinvestment Act. In the paper, authors Sebastien Houde and Joseph Aldy find that the SEEARP program provided little economic stimulation or energy savings. These results are not surprising, given problems with the design of the program (points that ACEEE made when states were crafting their versions of the program). However, Houde and Aldy’s conclusion that incentive programs as a class are costly and ineffective is an unjustified case of guilt by association.

Houde and Aldy examined sales of three appliances (refrigerators, clothes washers, and dishwashers) that accounted for 65% of SEEARP expenditures. So their conclusions apply only to those products and not to others, such as water heaters, furnaces, air conditioners, and heat pumps, which were included in some state programs.

The appliances included in their study represent major purchases and small incentives (Houde and Aldy find that incentives averaged 12-15% of product cost); too small to spur large numbers of consumers to make major purchases they were not planning to make, particularly during a recession. Not surprisingly, Houde and Aldy find that most program participants were planning to buy an appliance anyway, and they estimate that appliance sales increased only 1-2% above what would be expected absent the program. The appliances that Houde and Aldy didn’t consider had eligibility levels that were often set more stringently, reducing free riders.

Eligibility levels under the program were based on ENERGY STAR®. For the three appliances examined by Houde and Aldy, ENERGY STAR market share was very high (46-75% of sales prior to the program), and thus “free rider” levels would be expected to be very high. Free riders are consumers who took a rebate, but would have purchased an ENERGY STAR appliance even if the program were not offered. Houde and Aldy estimate that free riders were 73-92% of program participants. Free riders do not contribute to energy savings relative to business-as-usual, so when free riders are very high, energy savings are usually low, which Houde and Aldy confirm. ACEEE generally recommends that rebate program eligibility be limited to only the most efficient products with limited current market share (e.g. typically less than 15%), to help keep free riders down.

The ENERGY STAR market share for products not examined by Houde and Aldy was generally lower, so we would expect that portion of SEEARP to perform at least somewhat better than for the three appliances they examine. Still, the bottom line is the program could have been designed much better.

Rather than limit their findings to a tale of one poorly designed program, the authors use their results to impugn incentive programs more broadly, and declare that their findings “add to the growing body of evidence suggesting that energy efficiency subsidies tend to have a high cost to society due to various unintended consequences.” They also seem to go out of their way to find such unintended consequences. For example, they discuss how rebates can lead consumers to purchase larger appliances that use more energy and therefore reduce the energy saved.

If you look at the details in their own paper, though, you’ll find that ENERGY STAR models increased in size by less than 1%. Such small changes do not support their extensive discussion on this alleged problem. Finally, Houde and Aldy miss an opportunity to tell us how to design better rebate programs. They note how energy savings were greater in some states than others, but do not explore why that is the case so we all might learn from that finding. The only glimmer they provide is to note that generally higher rebates resulted in higher energy savings, that providing rebates as a percent of purchase price (instead of a fixed rebate) reduced energy savings, and that eligibility levels stricter than ENERGY STAR increased energy savings for some products. Good evaluations should tell us how we can do better, not argue for ideas that are poorly supported by the evidence.

For one example of how incentive programs can be successful, take a look at the Northwest Energy Efficiency Alliance’s (NEEA) efforts to promote more efficient clothes washers. NEEA succeeded in raising the market share of eligible clothes washers from 2% to more than 20%, ultimately influencing ENERGY STAR specifications and federal minimum efficiency standards.