Anyone involved in the Measurement and Verification (M&V) of building energy savings has encountered Stratified Random Sampling (SRS), a statistical tool used to handle the huge volume of data from the large number of projects and measures in a typical building energy research effort. From calibrated energy modeling to program evaluation and from the field through to the whitepaper, the use of applied statistical principles will simplify and improve the accuracy and defensibility of your M+V work.
Resistance
Now, at first glance, what could go wrong? Engineers are smart and like math, right? However, the reality of seeing a statistical sampling plan through from start to finish can go against the grain for a typical engineer. Here’s a look at why this is an issue in for Measurement and Verification, and what you can do about it.
Why Sampling?
Consider an example project: an efficient lighting retrofit in a school. You are the lead engineer responsible for evaluating the actual energy savings realized in a lighting upgrade. Imagine that this is one of a dozen similar projects that you are responsible for addressing in a relatively short timeframe.
Chances are you will be given a list, by a statistician, with a few dozen different efficiency measures, all with different levels of use to be verified, including new fixtures in the gym, high performance lights in three kinds of classrooms (60 rooms total), the library, office, and auditorium, as well as new LED parking lot fixtures.
It will begin to dawn on even the most intrepid engineer that there may not be enough time and metering equipment available to complete their evaluation in a timely fashion with the level of engineering rigor one might prefer. Enter statistical sampling.
The Sampling Plan
Below is an SRS sample size calculation for a generic school lighting project. For each usage group, the estimated power or energy savings and the quantity of light control switches are used to obtain the number of ‘tests’ required for an acceptable level of statistical accuracy and precision. Note that instead of metering all 180 classroom lighting circuits, sampling allows us to meter only eight and still achieve valid results!
Figure 1 - Sample Size CalculationThe Site Visit
You check in at the front office and successfully complete the field portion of your evaluation, installing the specified number of loggers in the exact random locations shown on the floorplan. The installed equipment is exactly as documented in the project file.
Figure 2 - Installed Logger DocumentationThe Analysis
After all the energy loggers have been successfully retrieved two weeks later it is time to hunker down for the analysis. This is where an engineer begins to break faith with the sampling methodology. After all, now that we have The Data, hasn’t sampling reached the end of its useful life? However, for the sampling plan to be effective, it needs to be utilized in the final analysis too.
Figure 3 - Classroom Sample Group Average for 24 HourFor example, in the Classroom Sample Group graph above, the lights are on 80% of the time between 7 and 8 am, on average. However, since the Classroom 1 energy logger measured that the lights are always on 100% of the time between 7 and 8 am, it can be difficult for an engineer to use the 80% number in their savings calculations for Classroom 1. Yet that is what an SRS approach requires.
In another example, we may have introduced bias on site by choosing ‘the best’ or most convenient metering locations, rather than using random numbers to choose meter locations, as required for SRS. This may result in overestimating, or underestimating, evaluated classroom energy savings.
Statistics vs. Engineering
The reason following through with an SRS approach can be a challenge is that engineers have learned to rely on their judgment, and in fact have made a career of it. Codes of ethics generally prevent us from expressing a definitive opinion regarding events about which we have no direct knowledge, at least not without a defensible reason. Statistics, on the other hand, has everything to do with drawing precise conclusions about real events, such as lighting usage in a classroom that was not measured, using no judgment or reasoning, using only a random number approach.
If an engineer has been told that ‘no one really uses that room,’ and it was found locked with the lights off, it can take some convincing to persuade them to take measurements in that room. From the statistical point of view though, if the sampling plan was prepared correctly, it does not matter whether a particular room is used all day every day, or only twice every other Sunday. What is important is that the room was randomly chosen without bias, and not out of convenience, since that very randomness is what establishes the validity of the energy savings for the SRS project.
Conclusions
It may be helpful for building energy engineers using SRS for Measurement and Verification to keep three things in mind:
1) If site conditions are truly inconsistent with the sampling plan, the engineer can and should exercise their engineering judgment by re-sampling the project on site.
2) Remember that the metering results might surprise you! If you already knew the answer, you wouldn’t need to meter it.
3) Trust the methodology. Statistics is widely accepted by many as a valid approach. Strike a balance between a healthy level of professional vigilance in the interest of the highest public good, and trusting that your approved sampling plan has captured the primary variables of interest, for the purposes of an energy savings calculation.
In summary, the more we can learn about when, how, and why, buildings use energy, the better we can control it. Using SRS will allow you to stay ahead of the curve, and be the first to predict trends accurately enough to specify optimal energy control solutions, without depleting your time and financial budgets.
