Evaluation, measurement and verification (EM&V) studies can provide a special opportunity for uncovering what is and is not working in an energy efficiency program. Energy efficiency program administrators look to EM&V studies to find out how their efficiency programs can be improved. To effectively identify opportunities for improvement, it is important for efficiency program administrators and evaluators to avoid the pitfalls of evaluation, measurement and verification.
Photo by Flickr user TomasNot planning far enough ahead for data collection and analysis
EM&V studies take time! More specifically, EM&V studies often require significant blocks of time (e.g. months) to execute effective outreach to efficiency program participants and to perform M&V during the appropriate seasons (e.g. heating or cooling season). Outreach to program participants can be a lengthy process of first identifying the participants, obtaining accurate and precise contact information, communicating requests for evaluation participation, waiting for responses from participants, and finally scheduling onsite M&V and/or survey administration. Consideration of the appropriate season(s) for an EM&V study can add six months to a year of data collection time, after evaluation participants have been recruited for onsite EM&V. Typically, procuring an EM&V study for completion in the same calendar year is an unrealistic plan.
Designing an evaluation that will not yield actionable results
Don’t bother investigating something that you can’t do anything about! There are many interesting things that program administrators and evaluators may like to know about the process and performance of their efficiency programs. However, only a subset of these “interesting things” can actually be acted upon or influenced by program administrators. EM&V studies should be focused on actionable results, so that evaluation dollars are not wasted and opportunities for improvement are not missed.
Not making use of established EM&V resources
Don’t reinvent the wheel! EM&V studies have been designed and executed by many firms and agencies around the world. Any firm or agency today can benefit greatly from the insights and lessons learned of prior studies and established, publicly available resources on EM&V. Identifying and leveraging these insights and lessons learned should be a standard practice for EM&V, much like the standard practice of a literature review in academic research or a prior art review in patent applications.
Receiving feedback as bad news
Image by Flickr user hobvias sudoneighmDon’t shoot the messenger! Findings of poor performance is actually good information, provided that the findings are actionable. EM&V studies should not be used to confirm what we already know. A good EM&V study is one that uncovers how an efficiency program can be adjusted for significant improvement in market impacts.
Avoiding pitfalls of evaluation, measurement and verification studies
Keep these possible pitfalls of EM&V studies in mind before embarking on your next project. If you have questions or would like more information, please use the comments below or get directly in touch with us!
