Library and Archives Canada
Symbol of the Government of Canada

Institutional links

ARCHIVED - About Us

Archived Content

This archived Web page remains online for reference, research or recordkeeping purposes. This page will not be altered or updated. Web pages that are archived on the Internet are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats of this page on the Contact Us page.

Formative Evaluation of the National Archival Development Program
Final Report

Executive Summary

Introduction

In 2006, Library and Archives Canada (LAC) established the National Archival Development Program (NADP) in response to the changing needs of the archival community and priority shifts within LAC. There are two streams of NADP funding: $1,140,000 is directed toward funding projects for ultimate recipients, and the Canadian Council of Archives receives $580,000 for administering this funding and developing archival capacity. As part of its evaluation strategy, LAC scheduled this formative evaluation to be conducted in the second year of the program. For the purposes of this study, the two funding streams are evaluated together because they are closely related. This document constitutes the formative evaluation's final report.

Purpose of evaluation

This evaluation addresses two main questions. First, it aims to determine whether the NADP is being implemented as designed. Next, it examines performance measurement activities of the program to determine whether appropriate performance measurement data is being collected, captured, and used. This evaluation is based on data from the first two years of the program.

Profile

LAC supported the CCA Secretariat when it was first created over twenty years ago. Prior to the inception of the NADP, the CCA administered a similar Grants and Contributions program through receipt of transfer payments from LAC (at the time, National Archives). Results from an audit conducted in 2003 and an evaluation conducted in 2004 showed a need for increased accountability in the Grants and Contributions program. In addition, there was a need to have more clearly defined program objectives. The Terms and Conditions of the program were also in need of updating for compliance with Treasury Board policy. To address these issues, the NADP replaced the Grants and Contributions program in 2006.

At the core of the program are five main objectives, and all proposed projects are required to demonstrate their support for one or more of these objectives. The NADP's five objectives are: to increase access to Canada's archival heritage through the national catalogue, increase awareness, increase the representation of Aboriginal peoples and under-represented ethno-cultural groups, increase capacity of archival networks to undertake strategic and development activities, and increase the capacity of archival institutions to preserve Canada's archival heritage. Projects submitted under these objectives are expected to generate outcomes, which are measured through performance indicators. All organizations receiving funding report on these outcomes. An annual $1.71 million is planned for allocation between 2006-2007 and 2010-2011. This amount covers both CCA activities and funding for archival projects.

Methodology

This evaluation employed the following methodologies:

  • Document review – official program documents provided by LAC and CCA management were reviewed, as well as project files. The project file review was conducted on site and included examining 20 project files, comprised of applications, contributions agreements, interim and final reports, correspondences and any deliverables produced.
  • Key Informant Interviews – interviews were conducted with 16 key informants, including representatives from the CCA, LAC, Provincial and Territorial Councils, and Project Recipients.

Findings

In the first year of the program, 94 applications were approved with funding and 46 additional applications were approved but had no funding available. In the second year of the program, 118 applications were approved with funding and 41 additional applications were approved but had no funding available. Data for approved, non-funded projects were not collected before the inception of the NADP.

There is a general sense of satisfaction with the five national objectives, and provincial and territorial archival institutions/organizations are satisfied with the current system of setting priorities at the local level. Objectives 2 and 3, the newest objectives, had the least number of projects in both years of the program. The number of projects addressing these objectives increased in the second year, and interviews revealed that there is growing support for Objective 2 (increasing awareness). There also appears to be an important perception from the community to continue funding projects that in part address backlog reduction, which was an objective in itself under the previous Grants and Contributions program prior to the NADP.

The application process was determined to be onerous, especially when considering the relatively small amount of money involved in many of the projects. The increased application requirements are in place because of increased federal government accountability expectations. Since many organizations are dependant on NADP funding, applying is considered worthwhile. Obtaining « matching funds » for projects is difficult, especially for small, non-profit organizations that are often operated by volunteers or one archivist with limited annual budgets. Submitted applications required many clarifications, which is time-consuming for the CCA. However, the two-tier adjudication/review process seems to be working well overall. It helps ensure consistency in the program. Many errors in applications that pass through the P/T adjudication phase are addressed with the national-level review.

Roles and responsibilities of the different stakeholders within the NADP are clearly defined, and support provided is generally appropriate for all levels of the program. However, the onerous application process and P/T adjudication of projects are often done by volunteers and with little resources, which affects the number of errors passing through to the national review stage. Simplifying the application forms would be beneficial.

Some organizations and institutions experienced delays in receiving payments for their projects, which was sometimes problematic in financing and conducting the planned work within proposed timelines. The cause of these delays can be attributed to a number of factors, including errors in the application forms that need further clarification and delays in reporting. The main factor contributing to payment delays is the newness of the program. Progress has been observed and efforts are being made to prevent delays in the future.

Reporting by recipients is being completed using forms that were designed to capture performance measurement data. A performance measurement plan is in place, and LAC and the CCA are working together to implement the plan. Performance data have been collected in the final reporting forms, but at the time of this evaluation, these data did not appear to have been compiled or analyzed. The process of compiling, summarizing, and using these data was beginning at the time of this evaluation.

Some baseline data have been collected at the national level under Objectives 1 and 2; however, it appears that, in general, little baseline data for the indicators have been summarized.

Conclusion

The following conclusions were generated from the findings.

Evaluation question: Is the program being implemented as designed?

  • There is a general approval of the five national objectives by all stakeholders within the NADP. The system of setting local priorities at the provincial and territorial levels is working well and should continue.
  • There is a continued need for the NADP, as it is either the only source or a major source of funding for many groups.
  • The application form, which was considered to be onerous by applicants, should be simplified to the extent possible, in collaboration with LAC, to ensure that all the information currently being requested is necessary.
  • Given the large number of small-sized projects, and the fact that many applicants are volunteers, the program could consider providing grants, instead of contributions, for small-sized projects, when the program is renewed.

Evaluation question: Is appropriate performance information being collected, captured and used?

  • A comprehensive performance measurement strategy is in place. However, at this point, work is required to collect, organize, and report on performance of the NADP.
  • Data for a number indicators have been collected and exists in completed recipient reporting forms. The LAC and CCA should continue to work together to establish the baseline data from the information that has already been collected.
  • Consideration could be given to funding NADP projects, under the five objectives, that address performance measurement at the local level. Such projects could help to gain an additional perspective and familiarize project recipients with performance measurement. A portion of a funded project could be directed toward developing the tools and capacity to measure performance under the project's objective.
  • A more detailed explanation of the rationale for performance measurement within the NADP could be described in the application forms and reporting forms.
  • Given the current circumstances with regard to performance measurement, there is some doubt that sufficient evidence will be available at the end of five years to accurately report on all of the designated performance indicators. Enhanced efforts in performance measurement will strengthen the case for program renewal.

Management response

1. Evaluation question: Is the program being implemented as designed?

  • There is a general approval of the five national objectives by all stakeholders within the NADP. The system of setting local priorities at the provincial and territorial levels is working well and should continue.
    • Management Response: Accepted.
    • LAC is pleased with the success of the national objectives with all stakeholders. The system is flexible enough at the local level to allow a wide variety of projects to be funded under the national objectives. The current system will stay in place.
  • There is a continued need for the NADP, as it is either the only source or a major source of funding for many groups.
    • Management Response: Accepted.
    • LAC is aware that the NADP is one of the few options available for the archival community to receive funding for various and important activities and/or projects. LAC is committed to make the NADP as successful and relevant as possible to the applicants and beneficiaries.
  • The application form, which was considered to be onerous by applicants, should be simplified to the extant possible, in collaboration with LAC, to ensure that all the information currently being requested is necessary.
    • Management Response: Accepted.
    • LAC and CCA have already made significant changes to the various application documents and have also worked on providing improved guidelines and assistance for applicants. The wide range of quality in applications and perceived level of burden by applicants indicates that some of these changes have been beneficial. LAC commits to collaborate with the CCA to further improve the application forms to help reduce the burden as much as possible for applicants, while ensuring that the necessary reporting requirements for contributions programs are met.
    • Completion date: Improved application forms in place for fourth year of the program (2009-2010).
  • Given the large number of small-sized projects, and the fact that many applicants are volunteers, the program could consider providing grants, instead of contributions, for small-sized projects, when the program is renewed.
    • Management Response: This option will be examined when the program is renewed.
    • The question of grants versus contributions was examined when the program was re-designed in 2006. LAC chose to make the NADP a contribution program based on the findings and recommendations of the audit and evaluation of the previous version of the program, done in 2003 and 2004, as well as advice from the Treasury Board Secretariat. LAC commits to examine the situation and gauge the various options available when the program will be reviewed and evaluated for its renewal.
    • Completion date: A decision on whether grants will be included in the program for certain projects will be made by end of fiscal year 2010-2011.

2. Evaluation question: Is appropriate performance information being collected, captured and used?

  • A comprehensive performance measurement strategy is in place. However, at this point, work is required to collect, organize, and report on performance of the NADP.
    • Management Response: Accepted.
    • Developing a Performance Measurement Plan was in fact a successful accomplishment for both the CCA and LAC during the 07-08 fiscal year. The implementation of the plan was a priority as soon as it was approved, one of the first steps was to modify the structure of certain parts of the program (forms, reporting) to allow the capture of data required for the indicators. Substantial progress has been made in the past few months and a process is now in place for the majority of indicators. Organization and analysis have taken place for about one third of the indicators contained in the plan. LAC commits to provide the necessary resources to complete the process so that organization and analysis are in progress for all indicators before the audit of the program.
    • Completion date: End of fiscal year 2009-2010.
  • Data for a number of indicators has been collected and exists in completed recipient reporting forms. The LAC and CCA should continue to work together to establish the baseline data from the information that has already been collected.
    • Management Response: Accepted.
    • Both LAC and CCA understand the need to make progress on baseline data using what is already available in reporting forms. Work is already underway to establish baseline data for these indicators during the 08-09 fiscal year. Both the CCA and LAC have demonstrated flexibility to assign responsibility so as to facilitate implementation of the plan.
    • Completion date: In progress
  • Consideration could be given to funding NADP projects, under the five objectives, that address performance measurement at the local level. Such projects could help to gain an additional perspective and familiarize project recipients with performance measurement. A portion of a funded project could be directed toward developing the tools and capacity to measure performance under the project's objective.
    • Management Response: Accepted
    • LAC will examine this possibility in conjunction with the CCA. Although performance measurement in itself is not part of the program objectives, an evaluation criterion could be added to favour projects that would include performance measurement information, in addition to the ones already required through forms and reports.
    • Completion date: A decision will be taken with the CCA on this matter in October 2008.
  • A more detailed explanation of the rationale for performance measurement within the NADP could be described in the application forms and reporting forms.
    • Management Response: Accepted
    • LAC recognizes that the archival community (beneficiaries/applicants) requires a better understanding of the need and context of performance measurement. LAC commits to collaborate with the CCA to present an overview of the process, the plan and the analysis of results to the next CCA Annual General Meeting in October 2008 as well as consideration towards modifying the forms to provide more detailed explanation. Other communication options will also be explored.
    • Completion date: End of fiscal year 2008-2009
  • Given the current circumstances with regard to performance measurement, there is some doubt that sufficient evidence will be available at the end of five years to accurately report on all of the designated performance indicators. Enhanced efforts in performance measurement will strengthen the case for program renewal.
    • Management Response: Acknowledged.
    • LAC recognizes the significant cultural and work change which performance measurement has meant for the archival community and is working closely with the CCA to implement the performance measurement plan and to ensure that proper data is collected and analyzed for all indicators to facilitate the renewal of the program in 2011. LAC will review the progress in collaboration with the CCA and make appropriate adjustments as required to ensure that the plan is fully implemented by the end of the fourth year of the program.
    • Completion date: end of fiscal year 2009-2010.

Table of Contents