The Phoenix audit we could have had – Part 3
This is my last post on the Phoenix audit by the Office of the Auditor-General. In the first of three parts (The Phoenix audit we could have had – Part 1), I talked about governance and oversight. Part two (The Phoenix audit we could have had – Part 2) dealt with the level of details provided in terms of the state of pay. In both areas, there were missed opportunities galore.
Today I want to talk about the way forward.
What were the criteria?
There really weren’t any forward-looking ones, at least not upfront. They had some generic elements under governance, but that was it.
What the REAL criterion should have had
It is pretty simple — is there a plan in place going forward that addresses major issues, is risk-based, and is written down. There are lots of bells and whistles beyond that, things like cost and timelines, but the most basic element is “Do they have a plan?”
What did the audit find?
The audit found that
- Departments and agencies had significant difficulties in providing timely and accurate pay information and in supporting employees in resolving pay problems
- A sustainable solution will take years and cost much more than the $540 million the government expected to spend to resolve pay problems
What COULD the audit have found?
I need to digress for a minute and talk about the audit process. Generally speaking, auditors come forward and say, “Okay, here are the terms of reference for the audit, i.e. this is what we’re going to look at”. There may be some back and forth with the department to say, “Wait, what about this?” or more likely “Wait, that isn’t part of this project” — it looks at what is in scope and what is out of scope.
Then the actual audit process begins, there are lots of documents and meetings, and preliminary findings are shared with the Department. This is the opportunity for the auditors to say, “Based on the docs we have, and the info we have been given so far, this is what we’re thinking we might say.” At this time, departments go crazy and say, “whoa, THAT’S not true, did you read this doc and this doc and this doc?”, often three docs that the auditors were never given. So they’re wrong about some aspect because they didn’t have any evidence from what they had seen so far. A gap, if you will.
Then they come back with their draft audit findings, they go through some iterations where the department gets to agree or disagree with some of the wording, often saying, “Wait, if you say THAT, with that language, we have to disagree, it goes too far”, and the auditors balance out their wording with their findings. While some people get their backs up that this is interfering with the independence of the auditors, it is often more along the lines of the auditors saying, “We examined the building plans for a green cabinet, a blue cabinet, and a yellow cabinet, and we found no evidence of cost analysis.” And the department says, “wait a minute, we had full analysis for yellow, and partial for blue, but agree with nothing for green.” And the auditors go back and look at their evidence and come back with revised wording that likely says “Not all projects had full cost analysis in place and there were significant gaps for most.” They’re still slapping someone, just making sure they’re slapping the right someone with the right language. And to be candid, some of it is seeing how much pain the department can handle. Can it handle 4 slaps or only 2? So the auditors start by saying “YOU COMPLETELY SUCK” and water down the language a bit at a time until the department stops whining, somewhere between “YOU MOSTLY SUCK” and “YOU’RE KINDA SUCKY IN CERTAIN AREAS”.
Because after the audit is done, there are two things the department has to produce and the level of work depends on which of those phrases the department could live with:
- A management response; and,
- A management action plan.
The management response is a simple response where the departments says “We agree” or “We disagree” with the recommendations. The cycle of responses over the years has ebbed and flowed, with some periods existing where no DM ever wanted to disagree with an audit recommendation. Even if they thought the auditors completely misunderstood the situation, they would say “We agree” and then in the prose explain how they were planning to either not do what they said or the exact opposite of agreement. The responses were somewhere between a sorry/not sorry situation and a Sir Humphrey response from a Yes Minister episode.
In more recent years, and changes in Auditor Generals, DMs feel more comfortable to say, “Wait, hold on a minute, we grudgingly agree with your findings, but NOT your recommendations on how to fix it…so we disagree.” But auditor generals NEVER want the report to say the department disagrees, as it basically means they’re saying fairly confidently that the AG didn’t understand the subject matter or project. If a department disagrees, this means they seriously disagree and then suddenly the AG jumps into the project directly, often working to massage the language enough to be so much “motherhood and apple pie”-type statements, that NO deputy could ever disagree with the recommendation. And then they’re back to bland recommendations that the department can agree with easily. Yawn.
But, regardless of the MR, the department also has to create a management action plan. And there is one relatively universal truth to MRs and MAPs — a “plan to have a plan” is not a plan in and of itself. The department cannot say, “Oh, yeah, that’s a good idea, we’ll look it, develop a plan, and then implement it.” They are SUPPOSED to say, “Hey, good idea, we’re going to do THIS and THIS to implement.” In other words, you need to have some content and an actual plan, not a plan to have a plan.
What would this look like in this case? They would have recommendations for a clear set of roles and responsibilities between players. Which this audit did recommend, except that the response is that they’ll create such a plan. A plan to have a plan, not the plan itself.
They would have clear recommendations relating to the role of other departments who send the pay files to Phoenix. The audit found that “Departments and agencies contributed to the problems; however, Public Services and Procurement Canada did not provide them with all the information and support to allow them to resolve pay problems to ensure that their employees consistently receive their correct pay, on time” so they did articulate a problem. Yet the response is a plan to have a plan to fix that.
There should be clear recommendations on transparency, risk-based triaging, cost breakdowns and service delivery mechanisms. Again, there are some strong hints to do that, and the response is “We’ll develop a plan.”
DEVELOP A PLAN???? What the heck have they been doing for the last two years or even the last 8 months while the audit was busting their chops internally? The Department *knew* that the audit finding was coming, and they should have had the plan relatively complete in certain areas. It should be ready to go.
Heck, the language was so watered down, it looked more like “we’re working on a strategy on how to develop a work plan that will lead us to a complete plan to respond to the challenges identified …. zzzzz”. Their plan is to develop a plan to have a plan. They haven’t even developed the PLAN for the plan, let alone the actual plan.
Conclusion
No governance in place, but watered-down wording that could possibly lead to little concrete change.
No transparency in data, so employees are still wondering what the state of pay is, and no recommendations or commitments to change that reporting.
Hardly any commitment at all of anything, other than a plan to have a plan.
It’s unfathomable that such an audit passed even the most basic internal tests at the OAG. Based on the actions committed to, it seems more like the audit equivalent of a hangnail than a project that is way over cost and a disaster on the ground. The system is bleeding out, but good news, they think they might know someone who can come up with a plan to develop a strategy to stop bleeding in general. But let’s not rush into anything resembling a solution.
