Results

What are the [results] evaluation factors?

The scoring of responses to Criteria Items and Award applicant feedback are based on two evaluation dimensions: (1) Process and (2) Results. Criteria users need to furnish information relating to these dimensions. The specific factors for Results: Levels, Rate/Breadth of Trends, Comparisons, and Linkage (L-T-C-Li), are described below.

Levels

Definition:

"Levels" refers to numerical information that places or positions an organization's results and performance on a meaningful measurement scale. Performance levels permit evaluation relative to past performance, projections, goals, and appropriate comparisons.

Features:

- Meaningful scale

- Related to what the applicant described in the Organizational Profile and Process Items [Categories 1-6].

Questions to ask in analyzing levels:

- Do the results provided address the Criteria requirements?

- Is the performance level excellent? Good? Poor?

- Are the axes and units meaningful?

Trends

Definition:

"Trends" refers to numerical information that shows the direction and rate (slope of trend data) and breadth (how widely deployed and shared) of performance improvements. A minimum of three data points generally is needed to begin to ascertain a trend. More data points are needed to define a statistically valid trend.

Features:

- Direction and slope of results over time

- Number of data points dependent on cycle time of process being measured

Questions to be asked in analyzing trends:

- Are results tracked over time? Are results sustained? Is there consistency of results—favorable, unfavorable, or no change? If there is no change, is it OK because the organization's results show sustained high performance?

- Do you understand the trends--remember that decreasing numbers can be better and the applicant may not have indicated the direction of desired results.

- Do the results include segmentation? If results are not segmented, does the applicant explain why not? What is happening within each segment? What is happening for a selected segment through the Item? If segmentation varies in the figures, why? Are the key segments appropriate? Are the results segmented in a way that can help the applicant improve what they are doing?

Comparisons

Definition:

"Comparisons" refer to establishing value of results by relationship to similar or equivalent measures. Comparison can be made to results of competitors, industry averages, or best-in-class organizations. The maturity of the organization should help determine what comparisons are most relevant. "Comparisons" also refer to your performance relative to appropriate comparisons and/or benchmarks

Features:

- Relevance

- Consistency

Questions to be asked in analyzing comparisons:

- Are comparisons provided for a few, some, or many to most figures? (Applicants do not need to provide a comparison for every figure.)

- Are comparisons provided for the most important results?

- Are the comparisons provided relevant?

- Are the comparisons with local, industry leaders, or best-in-class organizations?

Linkage

Definition:

"Linkage" refers to a connection to important customer, product and service, market, process, and action plan performance requirements identified in the Organizational Profile and Process Items [Categories 1-6].

Features:

- Relate to key requirements as described by the Applicant

- Listed in Item notes

Questions to be asked in analyzing linkage:

- Depending on the particular Results Item requirements and what the applicant stated was important, what would you expect to see (e.g., key measures, linkage to key segments, linkage to key processes, linkage to action plans)?

- Do the results link to responses in the Process Item Notes?

How do I use these [results] evaluation factors?

To ensure a more holistic evaluation of Results Items (Category 7), answer the questions [above] when drafting and before finalizing your comments. You should be able to strengthen and clarify comments using insights gained from answering the questions. Also determine how much information is missing. Is there a pattern in these "gaps" or is one key segment or area generally missing? Provide details related to gaps and evidence, including exact numbers and details from figures, to support your comments.

See also

• Results Scoring Guidelines

• Process Evaluation Dimensions