This archived Web page remains online for reference, research or recordkeeping purposes. This page will not be altered or updated. Web pages that are archived on the Internet are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats of this page on the Contact Us page.
The CCA has established forms for interim and final reporting that are clear, and the expectations for these reports appear to be well understood by funding recipients. The CCA ensures that all organizations complete these reports before being issued holdback payments. Although the level of reporting required is generally seen as appropriate, when added to application requirements, it is still considered burdensome by some organizations, in particular by institutions with small projects, where the reporting is considered too onerous for the amount of funding allocated. However, these reporting requirements reflect the increased federal government pressure for results-based management.
Recipients are generally aware of the importance of reporting, and they are reporting information that will be used to measure performance of the program. Final reports completed by project recipients are one of the main tools for collecting performance measurement data. In the first year of the program, the final reporting forms did not describe the performance measurement being undertaken, so recipients were not necessarily aware of NADP's performance measurement requirements. The final reporting form was modified for the second year of the program to include descriptions of the expected immediate and long-term outcomes. Recipients are asked to link their projects to the realization of these outcomes. These modifications should help increase awareness of performance measurement for funding recipients.
In addition to describing expected outcomes, the CCA Board and Secretariat modified the form over time to capture better performance measurement data. The current form includes specific questions on performance measurement (e.g., number of new fonds level descriptions contributed to the National Catalogue, linear metres of material re-housed), and more general questions (e.g., how did the project increase participation of ethno-cultural groups in archival heritage and thereby their representation in Canada's archival systems). It appears that reporting on performance indicators does not require much extra work for recipients, given that they need to complete a final report anyway.
NADP reporting activities feed into the broader performance measurement strategies.
The 2005 NADP RMAF-RBAF identified challenges in program monitoring as a risk to the success of the program. The clarity of performance measures and challenging old assumptions were identified as being important to consider in developing a strategy. The first few years of the program were aimed at capacity development, and performance measurement was planned to become more of a focus after the formative evaluation. This is what has taken place.
The LAC and the CCA have established a performance measurement plan, including an identification of indicators, steps involved in data collection and timelines for activities and reporting. The concept of performance measurement and the steps needed to be taken to address it are well understood by the Board of the CCA and LAC. Performance targets are in the process of being set. The LAC and the CCA Board and CCA Secretariat are working together to establish these targets.
The CCA has received two years worth of final report forms. At the time of this study, it appeared that most of the data in the reporting forms had not been compiled or analyzed. National baseline data had been established for a few indicators under Objectives 1 and 2, but not for Objectives 3, 4 and 5.
The following is a summary of performance measurement as it pertains to each of the national objectives.
Objective 1 – Increase access to Canada's archival heritage through the National Catalogue
Performance measurement is a particular issue with Objective 1, increasing access through the National Catalogue. Currently, the Catalogue is capable of including descriptions at the fonds level only, which limits its usability as a performance indicator. Interviewees commented that projects could be contributing descriptions at lower levels, and that this would not be reflected in National Catalogue contribution numbers. Funded projects are asked to report on their contributions to the National Catalogue, which is planned to be used as a means to measure performance. Other indicators under Objective 1 (e.g., the extent of material described, number of series described) may be more representative of the work being done. These are being collected through the reporting forms and should be used in performance measurement reporting by the CCA and LAC.
Institutional participation in the national catalogue and participation in the Provincial/Territorial database are other indicators to be used under Objective 1. A careful examination of the baseline data is necessary here; interviews demonstrated that in some regions, institutional participation in the National Catalogue is already very high. Baseline data collected to date under Objective 1 include institutional participation in the National Catalogue and data on user experience with the national catalogue.
Objective 2 – Increase awareness and broaden use of Canada's archives
Surveys are planned to help in measuring performance related to this objective. A general survey led by LAC was conducted in 2007, to determine public awareness about archives. A repeat of this survey at a later date is planned to measure change. This indicator will also help to determine effects of the CBC television show being broadcasted, entitled: Who do you think you are? This survey provides some baseline data under Objective 2; however, distinguishing the effects of this television show, NADP awareness projects, and other activities within the program may be difficult. Observed changes in numbers of archives users may not be directly related to NADP awareness-raising projects. Surveying new archives users about their reasons for use may be more beneficial.
Obtaining output numbers relating to awareness activities from final reports will give an idea about the reach/extent of awareness activities. For example, an organization can report on producing and distributing 1,000 brochures to raise awareness. However, the effects the brochures have on their audience needs to be understood. Funding specific projects through the NADP could help to assess performance at the local level. For example, a project funded under Objective 2 on increasing awareness could have a component of their project that includes performance measurement, such as conducting a survey on archival awareness in their community.
Objective 3 – Increase the representation of Aboriginal peoples and under-represented ethno-cultural groups in Canada's archives
As with Objective 1, the number of National Catalogue contributions may not be a completely representative indicator. Other data being gathered include the number of new Aboriginal/under-represented ethno-cultural group descriptions (at any level) as an indicator, which is more representative. The final reporting form for project recipients was modified for 2007-2008 to include a section asking for numbers of descriptions at all levels (i.e., fonds, series, subseries, file and item levels). The 2006-2007 reporting only requested the reporting of fonds contributions, which means that the more comprehensive baseline data set for this indicator is available beginning in the second year of the program. It does not appear that baseline data under Objective 3 have been established.
Objective 4 – Increase the capacity of archival networks to undertake strategic and development activities
Data for measuring performance under this Objective will come largely from the CCA. A section of the final report form requests project recipients to describe the strategic, promotional and educational activities undertaken during that year. The section includes collecting qualitative and quantitative data on the subject. Having information such as reach, numbers of workshop participants, and tools developed will help measure changes in capacity. Archives Advisors and Provincial/Territorial Councils will also be asked to report on activities undertaken that were not funded by the NADP, which will also help to determine capacity. However, this strategy has been developed only recently, and adequate results may not be available by the time of the summative evaluation.
Objective 5 – Increase the capacity of archival institutions to preserve Canada's archival heritage
Percentage of "at-risk" media types that have been conserved or re-formatted is an indicator for this objective, and this information is requested from project recipients in the final report form. Analysis and reporting for this indicator is straight-forward, as the quantitative information can be gathered directly from the reports and summarized. In addition, archival institutions having completed global preservation assessments in the period from 1999-2006 will be surveyed. There are also plans in place to develop a database drawing from a sample of these global preservation assessments. The survey and database will help to determine the amount of at-risk media in Canada and is intended to shed light on the percentage conserved or re-formatted. Since the data extend back to 1999, these lines of evidence could be important for baseline data development, creating a more complete measure of performance by the time of the summative evaluation. However, it appears that these data have not yet been summarized.