Determine Evaluation Metrics (33)

Description:

Develop metrics in parallel with system requirement specifications and the interface control document because those documents will specify data reported by the vendor and the ability to record data. Although state systems also record state data and compute metrics, those data may not appear in the interface control document, since it is internal to the state system. Vendor data must be reported to the state on a regular basis. Determine the reporting period for this data; reporting each month is often desirable, but some metrics can be generated more frequently.

Details:

This building block determines system performance metrics and customer service/service-level agreement metrics. System performance metrics include, at a minimum, the following:

  • Dollars charged (total, by vehicle weight type, and by individual vehicle/vehicle identification number)
  • Dollars collected, which may differ from dollars charged
  • Miles driven (total and by individual vehicle/vehicle identification number)
  • Count of enrolled vehicles
  • Count of active vehicles (i.e., vehicles that are enrolled and are actively reporting mileage)
  • Estimated or measured fuel consumed (only if fuel tax credits or refunds are offered)
  • Average vehicle fuel economy in miles per gallon (computed by dividing miles driven by fuel consumed for the appropriate vehicles)
  • Any error metrics available

Customer service metrics include, at a minimum:

  • Number of contacts per reporting period
  • Average wait time
  • Number of dropped calls
  • Number of emails per topic

Primary Use:

Determine how the system’s performance can be measured.

Best Practices/Lessons Learned:

  • Have many metrics; measure the system in as many ways as possible to get the greatest number of insights.
  • Have new, specific metrics for unique system features; anything unique should be measured.
  • Record metric values each reporting period, and research any unusual trends that occur.

State Government Context and Assumptions

The implementing RUC agency completes this task with consulting support in parallel with the development of system requirements and interface control documents.