ARCHIVED - About Us

Audits and Evaluations

Results-based Management and Accountability Framework

For LAC's Management of the Documentary
Heritage of Interest to Canada
Program Activity 1.2

4.2 Evaluation Plan

4.2.1 Overall Strategy and Evaluation Issues

This last section describes the evaluation strategy for the Management of the Documentary Heritage of Interest to Canada. An evaluation is a special study conducted to assess the relevance, success and cost-effectiveness of a program, policy or activity area. The federal Evaluation Policy calls for evaluations of all programs within five year cycles, at a minimum.

For this Program Activity, an evaluation of the Disposition of the government information of continuing value activity is suggested for 2011. At this stage, it is assumed that the Program Activity 1.2 would be evaluated as a stand-alone. The evaluation would be governed by an evaluation committee and would be guided by the following evaluation issues listed below. The methodologies used to address the issues are also outlined below.

4.2.1 Overall Strategy and Evaluation Issues

Evaluation Issues	Methodologies
1. To what extent does the program activity support LAC's mission and priorities? Are they consistent with federal government priorities?	Document review Key informant interviews
Governance and Delivery
2. Are appropriate resources and processes in place to acquire collection items?	Document review Key informant interviews Focus groups International benchmarking
3. Are appropriate standards and targets in place to ensure effective acquisition, description and care of collection?	Document review Key informant interviews Expert panel
4. Are acquisitions, descriptions and preservation processes meeting international standards, LAC service standards, priorities and targets?	Document revieww Key informant interviewsw Focus groupsw Expert panel
5. What is the state of the collection from a preservation perspective? Is the long-term access to the collection at risk?	Document review Key informant interviews Focus groups
6. Is LAC meeting the digitization challenge in terms of coverage and format? Are the right digitization decisions taken?	Document review Key informant interviews Focus groups Expert panel
7. Are there appropriate IT and HR strategies in place to support the Management of the Documentary Heritage Sub Activity?	Document review Key informant interviews Focus groups
8. Are the proper mechanisms/measures available to monitor the success of the Sub Activity? Is the reporting system providing reliable, consistent and useful information?	Document review Key informant interviews
Success
9. To what extent have the Management of the Documentary Heritage achieved the outcomes indicated in the logic model?	Document review Key informant interviews Focus groups Expert panel International benchmarking
10. Are the acquired items representative of Canada? Is the publication collection process comprehensive? Are key unpublished items collected?	Document review Key informant interviews Focus groups Expert panel
Cost-effectiveness and alternatives
11. To what extent are LAC resources for Management of the Documentary Heritage Activities used cost-effectively? Is LAC fully exploiting its partnerships with other stakeholders, such as publishers and other libraries to reduce description effort?	Document review Key informant interviews Focus groups
12. What could improve the Management of the Documentary Heritage Activities?	Document review Key informant interviews Focus groups

The methodologies are further explained below.

4.2.2 Methodologies

The methodologies are basically the research mechanisms and sources of information that the evaluation team will used to gather the necessary evidence to address the evaluation issues above. They are briefly described below. This type of evaluation would cost in the range of $ 75,000 to and require 6 months to complete.

4.2.2.1 Document Reviews
Many documents and previous studies would provide useful information for evaluation purposes. These include reports on plans and priorities, program and policy documents, Departmental Performance Reports, Quarterly and Annual Reports, previous studies, audits and evaluations and other documents required.

In particular, the reports derived from the monitoring mechanisms described in Section 4.1 would provide crucial output and outcome information, as well as information related to program delivery.

4.2.2.2 Key Informant Interviews
Key informant interviews are in-depth, one-on-one interviews based on open-ended questions. Such interviews are ideally conducted in person and can last up to one hour. The evaluation would rely on key informant interviews to obtain factual information, examples and explanations to address every evaluation issue. About 45 interviews would be conducted with senior LAC staff and external stakeholders, including publishers, library representatives and international stakeholders.

4.2.2.3 Focus Groups with LAC staff
Focus groups with staff will be used to gather evidence from the perspective of LAC staff, including factual information, views, examples and suggestions for improvement. A focus group is a discussion moderated by a professional facilitator. Typically, focus groups involve 8 to 10 participants in a two hour discussion format. Three focus groups would be organized (one per sub-sub-activity).

4.2.2.4 Expert Panel
To assess LAC priorities, workplans for unpublished materials and scope of the collection, an expert panel would be organized to obtain expert views on these.

4.2.2.6 International Benchmarking
To further assess the performance of the activity, an international benchmarking exercise could be conducted with comparable countries, such as Australia and New Zealand. The purpose of the exercise would be to compare resources invested, management approaches and the scope of the collections.

Previous | Table of Contents