What does the GHG Protocol say about Inventory Quality?

Summary: This is an extract from Chapter 7 of the GHG Protocol.

Companies have different reasons for managing the quality of their GHG emissions inventory, ranging from identifying opportunities for improvement to stakeholder demand to preparation for regulation. The GHG Protocol Corporate Standard recognizes that these reasons are a function of a company’s goals and its expectations for the future. A company’s goals for and vision of the evolution of the GHG emissions issue should guide the design of its corporate inventory, the implementation of a quality management system, and the treatment of uncertainty within its inventory.

A corporate GHG inventory program includes all institutional, managerial, and technical arrangements made for the collection of data, preparation of the inventory, and implementation of steps to manage the quality of the inventory. ****The guidance in this chapter is intended to help companies develop and implement a quality management system for their inventory.

Given an uncertain future, high quality information will have greater value and more uses, while low quality information may have little or no value or use and may even incur penalties. For example, a company may currently be focusing on a voluntary GHG program but also want its inventory data to meet the anticipated requirements of a future when emissions may have monetary value. A quality management system is essential to ensuring that an inventory continues to meet the principles of the GHG Protocol Corporate Standard and anticipates the requirements of future GHG emissions programs.

Even if a company is not anticipating a future regulatory mechanism, internal and external stakeholders will demand high quality inventory information. Therefore, the implementation of some type of quality management system is important. However, the GHG Protocol Corporate Standard recognizes that companies do not have unlimited resources, and, unlike financial accounting, corporate GHG inventories involve a level of scientific and engineering complexity. Therefore, companies should develop their inventory program and quality management system as a cumulative effort in keeping with their resources, the broader evolution of policy, and their own corporate vision.

A quality management system provides a systematic process for preventing and correcting errors, and identifies areas where investments will likely lead to the greatest improvement in overall inventory quality. However, the primary objective of quality management is ensuring the credibility of a company’s GHG inventory information. The first step towards achieving this objective is defining inventory quality.

Defining inventory quality

The GHG Protocol Corporate Standard outlines five accounting principles that set an implicit standard for the faithful representation of a company’s GHG emission through its technical, accounting, and reporting efforts (see chapter 1). Putting these principles into practice will result in a credible and unbiased treatment and presentation of issues and data. For a company to follow these principles, quality management needs to be an integral part of its corporate inventory program. The goal of a quality management system is to ensure that these principles are put into practice.

An inventory program framework

A practical framework is needed to help companies conceptualize and design a quality management system and to help plan for future improvements. This framework focuses on the following institutional, managerial, and technical components of an inventory (Figure 11):

Methods: These are the technical aspects of inventory preparation. Companies should select or develop methodologies for estimating emissions that accurately represent the characteristics of their source categories. The GHG Protocol provides many default methods and calculation tools to help with this effort. The design of an inventory program and quality management system should provide for the selection, application, and updating of inventory methodologies as new research becomes available, changes are made to business operations, or the importance of inventory reporting is elevated.

Data: This is the basic information on activity levels, emission factors, processes, and operations. Although methodologies need to be appropriately rigorous and detailed, data quality is more important. No methodology can compensate for poor quality input data. The design of a corporate inventory program should facilitate the collection of high quality inventory data and the maintenance and improvement of collection procedures.

Inventory Process and Systems:

These are the institutional, managerial, and technical procedures for preparing GHG inventories. They include the team and processes charged with the goal of producing a high quality inventory. To streamline GHG inventory quality management, these processes and systems may be integrated, where appropriate, with other corporate processes related to quality.

Documentation : This is the record of methods, data, processes, systems, assumptions, and estimates used to prepare an inventory. It includes everything employees need to prepare and improve a company’s inventory.

Since estimating GHG emissions is inherently technical (involving engineering and science), high quality, transparent documentation is particularly important to credibility. If information is not credible, or fails to be effectively communicated to either internal or external stakeholders, it will not have value.

Companies should seek to ensure the quality of these components at every level of their inventory design.

Implementing an inventory quality management system

A quality management system for a company’s inventory program should address all four of the inventory components described above. To implement the system, a company should take the following steps:

1. Establish an inventory quality team. This team should be responsible for implementing a quality management system, and continually improving inventory quality. The team or manager should coordinate interactions between relevant business units, facilities and external entities such as government agency programs, research institutions, verifiers, or consulting firms.
2. Develop a quality management plan. This plan describes the steps a company is taking to implement its quality management system, which should be incorporated into the design of its inventory program from the beginning, although further rigor and coverage of certain procedures may be phased in over multiple years. The plan should include procedures for all organizational levels and inventory development processes—from initial data collection to final reporting of accounts. For efficiency and comprehensiveness, companies should integrate (and extend as appropriate) existing quality systems to cover GHG management and reporting, such as any ISO procedures. To ensure accuracy, the bulk of the plan should focus on practical measures for implementing the quality management system, as described in steps three and four.
3. Perform generic quality checks. These apply to data and processes across the entire inventory, focusing on appropriately rigorous quality checks on data handling, documentation, and emission calculation activities (e.g., ensuring that correct unit conversions are used). Guidance on quality checking procedures is provided in the section on implementation below (see table 4).
4. Perform source-category-specific quality checks. This includes more rigorous investigations into the appropriate application of boundaries, recalculation procedures, and adherence to accounting and reporting principles for specific source categories, as well as the quality of the data input used (e.g., whether electricity bills or meter readings are the best source of consumption data) and a qualitative description of the major causes of uncertainty in the data. The information from these investigations can also be used to support a quantitative assessment of uncertainty. Guidance on these investigations is provided in the section on implementation below.
5. Review final inventory estimates and reports. After the inventory is completed, an internal technical review should focus on its engineering, scientific, and other technical aspects. Subsequently, an internal managerial review should focus on securing official corporate approval of and support for the inventory. A third type of review involving experts external to the company’s inventory program is addressed in chapter 10.
6. Institutionalize formal feedback loops. The results of the reviews in step five, as well as the results of every other component of a company’s quality management system, should be fed back via formal feedback procedures to the person or team identified in step one. Errors should be corrected and improvements implemented based on this feedback.
7. Establish reporting, documentation, and archiving procedures. The system should contain record keeping procedures that specify what information will be documented for internal purposes, how that information should be archived, and what information is to be reported for external stakeholders. Like internal and external reviews, these record keeping procedures include formal feedback mechanisms.

A company’s quality management system and overall inventory program should be treated as evolving, in keeping with a company’s reasons for preparing an inventory. The plan should address the company’s strategy for a multi-year implementation (i.e., recognize that inventories are a long-term effort), including steps to ensure that all quality control findings from previous years are adequately addressed.

Practical measures for implementation

Although principles and broad program design guidelines are important, any guidance on quality management would be incomplete without a discussion of practical inventory quality measures. A company should implement these measures at multiple levels within the company, from the point of primary data collection to the final corporate inventory approval process. It is important to implement these measures at points in the inventory program where errors are mostly likely to occur, such as the initial data collection phase and during calculation and data aggregation. While corporate level inventory quality may initially be emphasized, it is important to ensure quality measures are implemented at all levels of disaggregation (e.g., facility, process, geographical, according to a particular scope, etc) to be better prepared for GHG markets or regulatory rules in the future.

Companies also need to ensure the quality of their historical emission estimates and trend data. They can achieve this by employing inventory quality measures to minimize biases that can arise from changes in the characteristics of the data or methods used to calculate historical emission estimates, and by following the standards and guidance of chapter 5.

The third step of a quality management system, as described above, is to implement generic quality checking measures. These measures apply to all source categories and all levels of inventory preparation.

Table 4 provides a sample list of such measures.

The fourth step of a quality management system is source category-specific data quality investigations. The information gathered from these investigations can also be used for the quantitative and qualitative assessment of data uncertainty (see section on uncertainty).

Addressed below are the types of source-specific quality measures that can be employed for emission factors, activity data, and emission estimates.

Emission Factors and other Parameters

For a particular source category, emissions calculations will generally rely on emission factors and other parameters (e.g., utilization factors, oxidation rates, methane conversion factors). These factors and parameters may be published or default factors, based on company-specific data, site-specific data, or direct emission or other measurements. For fuel consumption, published emission factors based on fuel energy content are generally more accurate than those based on mass or volume, except when mass or volume based factors have been measured at the company or site-specific level. Quality investigations need to assess the representativeness and applicability of emission factors and other parameters to the specific characteristics of a company. Differences between measured and default values need to be qualitatively explained and justified based upon the company’s operational characteristics.

Activity Data

The collection of high quality activity data will often be the most significant limitation for corporate GHG inventories. Therefore, establishing robust data collection procedures needs to be a priority in the design of any company’s inventory program. The following are useful measures for ensuring the quality of activity data:

• Develop data collection procedures that allow the same data to be efficiently collected in future years.
• Convert fuel consumption data to energy units before applying carbon content emission factors, which may be better correlated to a fuel’s energy content than its mass.
• Compare current year data with historical trends. If data do not exhibit relatively consistent changes from year to year then the causes for these patterns should be investigated (e.g., changes of over 10 percent from year to year may warrant further investigation).
• Compare activity data from multiple reference sources (e.g., government survey data or data compiled by trade associations) with corporate data when possible. Such checks can ensure that consistent data is being reported to all parties. Data can also be compared among facilities within a company.
• Investigate activity data that is generated for purposes other than preparing a GHG inventory. In doing so, companies will need to check the applicability of this data to inventory purposes, including completeness, consistency with the source category definition, and consistency with the emission factors used. For example, data from different facilities may be examined for inconsistent measurement techniques, operating conditions, or technologies. Quality control measures (e.g., ISO) may have already been conducted during the data’s original preparation. These measures can be integrated with the company’s inventory quality management system.
• Check that base year recalculation procedures have been followed consistently and correctly (see chapter 5).
• Investigate whether biases or other characteristics that could affect data quality have been previously identified (e.g., by communicating with experts at a particular facility or elsewhere). For example, a bias could be the unintentional exclusion of operations at smaller facilities or data that do not correspond exactly with the company’s organizational boundaries.
• Extend quality management measures to cover any additional data (sales, production, etc.) used to estimate emission intensities or other ratios.

Emissions Estimates

Estimated emissions for a source category can be compared with historical data or other estimates to ensure they fall within a reasonable range. Potentially unreasonable estimates provide cause for checking emission factors or activity data and determining whether changes in methodology, market forces, or other events are sufficient reasons for the change. In situations where actual emission monitoring occurs (e.g., power plant CO2 emissions), the data from monitors can be compared with calculated emissions using activity data and emission factors.

If any of the above emission factor, activity data, emission estimate, or other parameter checks indicate a problem, more detailed investigations into the accuracy of the data or appropriateness of the methods may be required. These more detailed investigations can also be utilized to better assess the quality of data. One potential measure of data quality is a quantitative and qualitative assessment of their uncertainty.

Inventory quality and inventory uncertainty

Preparing a GHG inventory is inherently both an accounting and a scientific exercise. Most applications for company-level emissions and removal estimates require that these data be reported in a format similar to financial accounting data. In financial accounting, it is standard practice to report individual point estimates (i.e., single value versus a range of possible values). In contrast, the standard practice for most scientific studies of GHG and other emissions is to report quantitative data with estimated error bounds (i.e., uncertainty). Just like financial figures in a profit and loss or bank account statement, point estimates in a corporate emission inventory have obvious uses. However, how would or should the addition of some quantitative measure of uncertainty to an emission inventory be used?

In an ideal situation, in which a company had perfect quantitative information on the uncertainty of its emission estimates at all levels, the primary use of this information would almost certainly be comparative.

Such comparisons might be made across companies, across business units, across source categories, or through time. In this situation, inventory estimates could even be rated or discounted based on their quality before they were used, with uncertainty being the objective quantitative metric for quality. Unfortunately, such objective uncertainty estimates rarely exist.

Types of Uncertainties

Uncertainties associated with GHG inventories can be broadly categorized into scientific uncertainty and estimation uncertainty. Scientific uncertainty arises when the science of the actual emission and/or removal process is not completely understood. For example, many direct and indirect factors associated with global warming potential (GWP) values that are used to combine emission estimates for various GHGs involve significant scientific uncertainty. Analyzing and quantifying such scientific uncertainty is extremely problematic and is likely to be beyond the capacity of most company inventory programs.

Estimation uncertainty arises any time GHG emissions are quantified. Therefore all emissions or removal estimates are associated with estimation uncertainty.

Estimation uncertainty can be further classified into two types: model uncertainty and parameter uncertainty.

Model uncertainty refers to the uncertainty associated with the mathematical equations (i.e., models) used to characterize the relationships between various parameters and emission processes. For example, model uncertainty may arise either due to the use of an incorrect mathematical model or inappropriate input into the model. As with scientific uncertainty, estimating model uncertainty is likely to be beyond most company’s inventory efforts; however, some companies may wish to utilize their unique scientific and engineering expertise to evaluate the uncertainty in their emission estimation models.

Parameter uncertainty refers to the uncertainty associated with quantifying the parameters used as inputs (e.g., activity data and emission factors) into estimation models. Parameter uncertainties can be evaluated through statistical analysis, measurement equipment precision determinations, and expert judgment.

Quantifying parameter uncertainties and then estimating source category uncertainties based on these parameter uncertainties will be the primary focus of companies that choose to investigate the uncertainty in their emission inventories.

Limitations of uncertainty estimates

Given that only parameter uncertainties are within the feasible scope of most companies, uncertainty estimates for corporate GHG inventories will, of necessity, be imperfect. Complete and robust sample data will not always be available to assess the statistical uncertainty in every parameter. For most parameters (e.g., liters of gasoline purchased or tonnes of limestone consumed), only a single data point may be available. In some cases, companies can utilize instrument precision or calibration information to inform their assessment of statistical uncertainty. However, to quantify some of the systematic uncertainties ****associated with parameters and to supplement statistical uncertainty estimates, companies will usually have to rely on expert judgment. ****The problem with expert judgment, though, is that it is difficult to obtain in a comparable (i.e., unbiased) and consistent manner across parameters, source categories, or companies

For these reasons, almost all comprehensive estimates of uncertainty for GHG inventories will be not only imperfect but also have a subjective component and, despite the most thorough efforts, are themselves considered highly uncertain. In most cases, uncertainty estimates cannot be interpreted as an objective measure of quality. Nor can they be used to compare the quality of emission estimates between source categories or companies.

Exceptions to this include the following cases in which it is assumed that either statistical or instrument precision data are available to objectively estimate each parameter’s statistical uncertainty (i.e., expert judgment is not needed):

• When two operationally similar facilities use identical emission estimation methodologies, the differences in scientific or model uncertainties can, for the most part, be ignored. Then quantified estimates of statistical uncertainty can be treated as being comparable between facilities. This type of comparability is what is aimed for in some trading programs that prescribe specific monitoring, estimation, and measurement requirements. However, even in this situation, the degree of comparability depends on the flexibility that participants are given for estimating emissions, the homogeneity across facilities, as well as the level of enforcement and review of the methodologies used.
• Similarly, when a single facility uses the same estimation methodology each year, the systematic parameter uncertainties— in addition to scientific and model uncertainties— in a source’s emission estimates for two years are, for the most part, identical. Because the systematic parameter uncertainties then cancel out, the uncertainty in an emission trend (e.g., the difference between the estimates for two years) is generally less than the uncertainty in total emissions for a single year. In such a situation, quantified uncertainty estimates can be treated as being comparable over time and used to track relative changes in the quality of a facility’s emission estimates for that source category. Such estimates of uncertainty in emission trends can also be used as a guide to setting a facility’s emissions reduction target. Trend uncertainty estimates are likely to be less useful for setting broader (e.g., company-wide) targets (see chapter 11) because of the general problems with comparability between uncertainty estimates across gases, sources, and facilities.

Given these limitations, the role of qualitative and quantitative uncertainty assessments in developing GHG inventories include:

• Promoting a broader learning and quality feedback process.
• Supporting efforts to qualitatively understand and document the causes of uncertainty and help identify ways of improving inventory quality. For example, collecting the information needed to determine the statistical properties of activity data and emission factors forces one to ask hard questions and to carefully and systematically investigate data quality.
• Establishing lines of communication and feedback with data suppliers to identify specific opportunities to improve quality of the data and methods used.
• Providing valuable information to reviewers, verifiers, and managers for setting priorities for investments into improving data sources and methodologies.

The GHG Protocol Corporate Standard has developed a supplementary guidance document on uncertainty assessments (“Guidance on uncertainty assessment in GHG inventories and calculating statistical parameter uncertainty”) along with an uncertainty calculation tool, both of which are available on the GHG Protocol website. The guidance document describes how to use the calculation tool in aggregating uncertainties. It also discusses in more depth different types of uncertainties, the limitations of quantitative uncertainty assessment, and how uncertainty estimates should be properly interpreted.

Additional guidance and information on assessing uncertainty— including optional approaches to developing quantitative uncertainty estimates and eliciting judgments from experts — can also be found in EPA's Emissions Inventory Improvement Program, Volume VI: Quality Assurance/Quality Control (1999) and in chapter 6 of the IPCC’s Good Practice Guidance (2000a).