5.2.3 Measuring and Improving Performance
Key performance indicators (KPIs) should be identified as part of the BIS that will provide a measure into whether key objectives are met both during project execution and upon completion. During execution, these KPIs provide visibility into the status of a project and can highlight issues that could cause delays. At completion, the KPIs can be used to measure and/or confirm the degree of project success or failure.
The 10 Tests for a good performance measure include:
- Truth test: are we measuring what we set out to measure?
- Focus test: are we measuring only what we set out to measure, and not things that are out of scope?
- Relevance test: are we measuring the right thing?
- Consistency test: does the measure give a consistent result whenever and by whomever it is applied?
- Access test: can the data be easily accessed and understood?
- Clarity test: is there any ambiguity in interpreting the measure or the results?
- ‘So-what’ test: will the result give us information that we can act upon?
- Timeliness test: will the data be available in time to affect action?
- Cost test: how much will it cost to measure and is this cost effective?
- Gaming test: what behaviours will the measure discourage and encourage? Is there a risk of unintended consequences?
Contract owners are typically accountable for the establishment and management of measures and metrics to measure contract performance. Supplier scorecards are recognized tools for monitoring and managing the contract and supplier performance.
Some potential areas for managing through scorecard metrics can include:
Quality:
- #’s of project change orders (unplanned);
- #’s of design field changes (unplanned);
- #’s and severity of non-conformances;
- #’s and severity of significant audit findings; % items requiring rework;
- % items received with issues (e.g. over shipped, short, damaged or discrepancies);
- Repeat issues or warning letters sent to vendor;
Cost:
- Levelized unit energy cost (LUEC);
- Cost performance index (CPI);
- Cost growth / savings index;
- Project cost / project capacity (e.g. MWe);
- Burn rates (costs per day for specific activities);
- Workers / phase cost;
- Forecast project cost efficiency;
- Construction cost efficiency;
- Cost of vendor related quality issues for purchaser (cost impacts of poor quality, rework, etc.)
Schedule:
- Schedule performance index (SPI);
- Schedule growth index;
- Schedule efficiency (e.g. duration / project capacity (e.g. MWe));
- Deliverable turn-around times;
Responsiveness:
- Project manager/ owner satisfaction;
- Deliverable turn-around times;
- Average age of non-conformances or vendor corrective actions;
Innovation:
- Value of savings to owner due to innovative changes identified by supplier / contractor;
Risk management:
- Use of front-end planning tools/methods;
- Use of lessons learned programmes;
- Use of risk assessment and management tools;
- Level of detail of schedules/estimates;
- Use of earned value reporting;
Occupational safety and health:
- Industrial accident rates (lost time accidents, medically treated, first aid treatment, TRIR, DART, etc);
- Near misses;
- Leading safety indicators;
- Use of zero accident techniques.
Notes:
- Forecasted project cost efficiency = Forecasted project cost / capacity
- Construction cost efficiency = Construction cost / capacity of project
- Cost growth = (Actual cost – planned cost ) / Planned cost {can be looked at for entire project or for a particular phase, e.g. construction}
- Cost performance index = Earned value/actual cost
- Earned Value = Budgeted cost of work performed (BCWP)
- Planned Value = Budgeted cost of work scheduled (BCWS)
- Forecasted schedule efficiency = Forecasted total duration / capacity of project
- Schedule efficiency = Duration / capacity
- Schedule growth index = (Actual duration – Planned duration ) / Planned duration
- Schedule performance index = Earned value/planned value
- Burn rate = Cost / duration of interest
- TRIR (Total Number of Recordable Injuries Rate) = (Total Number of Recordable Injury Cases × 200,000) / Total Site Work-Hours
- DART (Days Away/Restricted or job Transfer Rate) = Total Number of DART Cases × 200,000) / Total Site Work-Hours
- Capacity efficiency = (Capacity / reference project capacity) / (Installed quantity / reference project installed quantity)
- Forecasted project cost efficiency = Forecasted project cost / capacity
- Construction cost efficiency = Construction cost / capacity of project
When performance issues have been identified, an appropriate escalation process (example shown below) based on issue priority should be invoked. This ensures that the appropriate management level is aware of the issue and that it can bring necessary resources to address the problem.
