Skip to content

Distortion of performance measures: From ‘making the numbers’ to ‘making up the numbers’

April 28, 2014

One of the significant criticisms of performance measures is their propensity to distortion. A later post will consider how to at address this, but for now, let’s focus on the types of distortion and how they arise.

distortion

The first category is outright lying. It is always possible to simply invent numbers and assume that no one will check. This is, of course, a high risk strategy for the person making up the numbers, as if someone does check, there is very little defence that would not lead to the loss of employment. However, it does happen.

Sometimes the lying is a bit more subtle. Workers may shift achievements from one period to another in order to try to get the performance measures in the period to be closer to the target. Or if the target cannot be met, then maybe shift results to the next period, so there will be one very bad result followed by an improvement next time.
The second form of distortion is that of working to the target, not to the underlying intention. For example, many years ago, there was a target in a large Australian government instrumentality that 90% of all requests for formal advice would receive an answer within 30 days. The response was to review requests that got to around 25 days old and if they were answerable in time to focus on them. If they could not be answered in time, they would allowed to be late as part of the allowable failure ratio of 10%. But then there was no incentive to ever answer these questions – they sat there, getting to 60 days old, 90 days old or even more.

A related distortion is effort shifting. If a target has been met, then even if it is quite possible to do better, workers may choose to not exceed the target. This enables focus on other things, and also addresses workers fears that if targets are exceeded, they will be raised next time.

A third form of distortion can occur in the initial setting of targets. It is often the operators who are responsible for at least advising on the performance measures and their targets. In order to maximise their chance of success, the operators recommend measures and targets that are easy to achieve, or generate reasons for changing measures that are harder to achieve.

What are the controls on this? If different units are producing comparable outputs, then a degree of competition can be encouraged. If one is offering to produce a higher output per cost (more cost-effective production), then it can receive more funding. In most cases in the public sector, however, an alternative approach is necessary. This will be covered in a later post, but in brief, it depends on setting the tone in the organisation, transmitting it to all workers, having good management at the unit level, systems for monitoring and auditing, and good executive oversight.

© Numerical Advantage 2014
http://www.numericaladvantage.com.au

 

 

Advertisements

From → Articles

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: