|
Canadian Defence Procurement – December
2004
Auditing the Upgrades to the CF-18 Fighter Aircraft
[Part 2]
The Auditor-General reviews the CF-18
Incremental Modernization Project
CF-18 Incremental Modernization Project, Public Works, & Treasury Board
The auditors' primary concern about the involvement of PWGSC (Public Works & Government Services Canada) in the
CF-18 Incremental Modernization Project was risk management. Neither DND
nor their PWGSC overseers developed a risk management plan at the outset of the CF-18 IMP. Instead, in
the earliest stages of the project, PWGSC relied entirely upon DND "to identify and manage risks to the
successful completion of the CF-18 modernization".
National Defence is at an early stage of risk management activity and
only recently drafted its plan to track and manage risks ... [PWGSC] and the Treasury Board (TB) Secretariat relied on
National Defence for the assessment of risk at the funding approval stage. National Defence, in turn, relied on the
contractor who had identified their risk exposure as low. Therefore, all three organizations used a low risk rating
... |
The OAG evaluated both DND and PWGSC risk management practices. Criteria were based on TB risk management practices as
well as federal and departmental risk management policy requirements. Neither DND nor PWGSC fully met criteria for good
risk management. Both these departments require improved practices.
... we expected [DND and PWGSC] to be able to provide us with evidence
to support the consultative process used to establish the government's low risk rating. For instance, we expected [DND]
to have identified critical risks that could affect the project schedule, such as lack of staff, techni-cal
difficulties, or delays in validating and verifying the [upgraded CF-18] aircraft for acceptance and airworthiness.
Similarly, we expected PWGSC to...provide us with documents to support the risks it had identified and how they had
been mitigated through the various clauses within the related contracts. |
Instead, the auditors found that DND, which claimed to identify and deal "with potential risks on a continuing
basis", made the development of a plan for risk management the responsibility of the contractors. Despite apparent
contractual risks, DND, PWGSC, and the Treasury Board worked from the contractors' plan.
Although critical of DND's reliance upon contractors for risk management plans, the auditors did not find the
CF-18 IMP itself suffered unduly from this practice. (This may be more to do with luck than planning. When it is time
to review DND project management of the Victoria-class submarines,
OAG auditors will likely cast a more jaundiced eye upon contractor-developed risk management plans.)
The auditors were also concerned about the lack of documentation involving the 'fairness monitor' engaged by
PWGSC to review the procurement process for the Advanced Distributed Combat Training System. PWGSC officials
believed that the monitor's qualifications and involvement in the process should be sufficient. This left the
auditors with no way to assess the fairness monitor's reliability. In the end, the ADCTS contract decision was
taken by the losing bidder before the Canadian International Trade Tribunal. PWGSC was vindicated by the CITT but such
confrontations are hardly a desirable method of demonstrating fairness.
Overseeing the Overseers – Treasury Board Secretariat's Oversight Role
The TB Secretariat is responsible for ensuring "the integrity of the expenditure process [and making sure that
DND] identifies and assesses project risks and puts in place measures and strategies to deal with risks". But,
officials from the TB Secretatiat emphasized that it is not their role to manage project risk.
The [TB] Secretariat did not provide us with evidence to support its
review of National Defence's identification and assessment of project risks for the ECP-583 project. [TB]
Secretariat staff told us that they attended ... ECP-583 briefings and that this demonstrated due diligence in its
oversight role. While attending [DND] briefings does represent some oversight, we also expected the Secretariat to
provide evidence supporting the other work they do in carrying out their oversight role. |
The auditors found that supporting documentation for the CF-18 IMP became more available starting in 2003 – two
years into the project. This they attributed to government approval of a CF-18 project being made conditional.
Funds were to be forthcoming only if DND submitted annual reports on the CF-18 IMP high- lighting 'milestone'
progress, significant changes to the project, and an updated risk assessment. The auditors saw this as a positive
development.
While auditors took the policy lapses and lack of documentation seriously (and rightly so) the overall sense was
that the CF-18 IMP was now being reported on properly by DND, Public Works, and the Treasury Board Secretariat.
However, the auditors had other concerns about DND's approach to project management.
|