ARCHIVED - About Us

Audits and Evaluations

Evaluation Plan 2008-2009

As Presented to:
The Evaluation Committee of Library and Archives Canada

April 03, 2008

Corporate Performance and Information Division
Corporate Management Branch

Table of Contents

• Synopsis
• Introduction
  • Responsibilities of the Evaluation Function
  • Evaluation policy
  • The Revised Evaluation planning process
• Proposed Evaluation Work
  • New engagements
  • Ongoing engagements
• Annex 1: Results-Based Management Accountability Framework  [PDF 20 KB]

---

Synopsis

This document presents the evaluation plan of LAC for the fiscal year 2008/09. It has been developed with the view to address organizational/corporate risk areas and key activities of LAC's PAA architecture.

The plan addresses Treasury Board requirements in the areas of evaluation and risk management with a balance of projects in each of these areas taking into account the needs of Library and Archives in achieving its strategic objectives.

The plan was developed through an assessment of the risk environment currently facing LAC, the PAA architecture and the new strategic priorities of the institution.

Introduction

Responsibilities of the Evaluation Function

The Corporate Performance and Information Division is responsible for providing independent evaluation and performance management/measurement services to Library and Archives Canada and for fulfilling the requirements of Treasury Board policies with respect to this function. Evaluation provides key inputs into management decision-making by assisting in the development of management and accountability frameworks and by performing professional assessments of the performance of policies and programs. In addition to these the Division is involved in a broad range of activities including:

• Providing advice and assistance to managers
• Training for managers
• Development of Risk Management Frameworks
• Supporting the Evaluation Committee
• Liaison with Treasury Board

Evaluation
The role of evaluation is:

• Help managers design or improve the design of policies, programs and initiatives
• Provide were appropriate periodic assessment of policy/program effectiveness and relevance, intended and unintended impacts, alternative ways of achieving desired results

Evaluation services:

• Summative Evaluation- an objective examination of evidence to provide an independent assessment of the continued relevance and effectiveness (including outcomes) of a policy, program, activity, or management initiative. Normally involves consideration of alternative ways of achieving expected results.
• Formative Evaluation- an objective examination of evidence for the purpose of providing an independent assessment of the likelihood of success of a policy, program, activity or management initiative in its early stages. It is often focused on the structural framework and process aspects of delivery or implementation
• RMAF- a framework for on-going measurement and reporting on the performance of a program, policy or activity. Normally includes a results and performance measurement strategy by they will be identified, measured and reported along with related accountabilities. Usually completed early in the development of a new program.

Modern Management and Performance Information

The role of performance measurement is:

• To provide integrated departmental performance reporting, service standards, operating, evaluative, financial and cost management information
• To measure client satisfaction

Performance measurement activities:

• Development and implementation of key measures to monitor overall organizational performance and best-value results
• Development and implementation of mechanisms to monitor service quality and efficiency of program delivery
• Design and conduct surveys to assess levels of client satisfaction and the importance of service
• Monitoring of service standards
• Ensuring the availability of timely and useful financial information

Evaluation policy

The 2001 TBS Evaluation Policy is currently undergoing renewal. The aim of the renewed policy being to give the evaluation function more neutrality and independence thus enabling it to provide credible information on the relevance, success and cost-effectiveness of individual programs, policies or horizontal areas to Ministers, departments and central agencies.

Furthermore the new Evaluation policy would be expanded to cover and support the Federal Accountability Act which requires that all Transfer Payment Programs are evaluated in terms of relevance and effectiveness over a 5-year cycle. In addition the 2006 Budget commits the Evaluation function to using results and value-for-money to inform priority setting and decision-making through a renewed Expenditure Management System.

Under the renewed policy Heads of Evaluation would be required to provide input into the performance measurement strategies embedded in the MRRS, to approve the performance measurement strategies of all new and proposed program spending and to ensure that program managers develop and implement ongoing performance measurement strategies. In addition they would be required to report annually on the state of performance measurement of programs.

The Revised Evaluation planning process

A new Evaluation Policy is expected be implemented by Treasury Board sometime in the next fiscal year and like the new Audit Policy it will undoubtedly have new requirements. Some of these which are anticipated are in addition to a new policy, a directive, standards and guidelines. It is also anticipated that the new approach will involve evaluation engagements that will cover 100% of direct program spending over a five year cycle using a variety of evaluation tools which would be applied based on an assessment of risk. Since both the policy requirements and the new tools are expected shortly any long-term planning at this time would likely be overtaken by events, it is proposed that a one year plan be developed and a more comprehensive 3 or 5 year plan be submitted to the Evaluation Committee for the period beginning with fiscal year 2009/10. This approach also takes into account the splitting of the Audit and Evaluation functions at LAC and the formation of separate Committees to oversee their work.

What follows is a one year Evaluation Plan which will prepare the institution for the future by examining key program activities and in doing so establish a baseline for continuous and systematic assessment of the performance, efficiency and effectiveness (based on the PAA). The emphasis of the work will be the development of five Results-based Management and Accountability Frameworks (RMAFs) for each of the LAC's three program activities and two of its key enablers. The rationale for the plan was derived from the Program Evaluation planning cycle. The latter involves 4 phases:

1. Program (program activity) initiation
2. Results- based Management Accountability Framework
3. Progress assessment/ Formative evaluation
4. Summative evaluation

This is supported by an environmental scan and consultation with the management teams of all LAC sectors at which point it became apparent that the institution is often lacking a starting point from which the effectiveness of programs can be determined.

The projects proposed are outlined in the table below and further descriptions are contained in the appendix that follows.

Furthermore it is proposed that the Evaluation Committee meet on a more systematic basis twice per year- in April and in October respectively. The April meeting would cover the report from the past fiscal year and the plan for the current fiscal year. The October meeting would involve a progress report on the evaluation projects approved by the committee for the current fiscal year, including a discussion of adjustments if required, as well as a follow-up of progress in implementation of approved management responses from previously completed engagements.

Using this approach, the long-term plan for 2009/10 and years beyond will be build on a strong foundation. It will also be possible to engage managers across the institution and build support for performance measurement as part of the development of the RMAFs which will not only support the strategic objectives of the institution but assist in fulfilling present and future Treasury Board requirements.

Proposed Evaluation Work Plan 2008-2009

New Engagements**

**CPID has currently available funding to cover only 3 of the above projects for the remaining two additional/alternative funding is requested

Engagement Name: Results-Based management and Accountability Framework (RMAF) - Managing the disposition of the Government of Canada records and recordkeeping

Administrative Area: Government Records Sector

Description: For detailed description see Annex 1

Objectives: To develop a framework for on-going measurement and reporting for the program activity "Managing the disposition of the Government of Canada Records and recordkeeping."

Fiscal Year Start: 2008/09 Q1

Engagement Name: Results-Based management and Accountability Framework- Managing the Documentary Heritage of Interest to Canada

Administrative Area: Documentary Heritage Collection

Activity Description: For detailed description see Annex 1.

Objectives: To develop a framework for on-going measurement and reporting for the program activity "Managing the Documentary Heritage of Interest to Canada."

Fiscal Year Start: 2008/09 Q3

Engagement Name: Results-Based management and Accountability Framework - Making the Documentary Heritage known and accessible for use

Administrative Area: Programs and Services

Activity Description: For detailed description see Annex 1.

Objectives: To develop a framework for on-going measurement and reporting for the program activity "Making the Documentary Heritage known and accessible for us."

Fiscal Year Start: 2008/09 Q2

Engagement Name: Results-Based management and Accountability Framework - Public Policy Services

Administrative Area: Corporate Management Sector

Activity Description: For detailed description see Annex 1.

Objectives: To develop a framework for on-going measurement and reporting for the program activity "Public Policy Services."

Fiscal Year Start: 2008/09 Q2

Engagement Name: Results-Based management and Accountability Framework - Human Resources

Administrative Area: Corporate Management Sector

Activity Description: For detailed description see Annex 1.

Objectives: To develop a framework for on-going measurement and reporting for the program activity "Human Resources."

Fiscal Year Start: 2008/09 Q1

Proposed Evaluation Plan 2008-2009

Ongoing engagements

Engagement Name: Development of LAC Enterprise Architecture

Administrative Area: Information Technology Branch (ITB)

Activity Description: The development of an Enterprise Architecture (EA) was a key objective of the ITB strategic plan. The aim of the initiative was to align LAC business priorities and its IT resources/services. In 2007-2008 based on the recommendations of CPID ITB conducted an environmental scan and assessed the needs of the institution to develop an Enterprise Architecture framework that is relevant to LAC and able to support its mandate. From that analysis an Action Plan was developed for the implementation of an EA model.

Engagement type and objectives: This is a consultation type of engagement. Focus: to work with ITB on developing an Evaluation Framework to assess the performance of the Enterprise Architecture model.

Engagement Name: Development of MRRS

Administrative Area: Strategic Office, Strategic Planning and Corporate Secretariat

Activity Description: The Management, Resource and Results Structure Policy of TBS requires the development of common government-wide approach to the collection, management, and reporting of financial and non-financial performance information. It provides the standard basis for reporting on the alignment of resources, program activities and results consistent with the Management Accountability Framework. The MRRS has 3 main components: 1) Clearly defined and measurable Strategic Outcomes 2) A Program Activity Architecture and 3) Description of current governance structure of departments outlining the decision-making mechanisms, responsibilities and accountabilities. The Strategic Planning and Corporate Secretariat has, as part of its mandate, the responsibility for providing leadership in the development, monitoring, evaluation and revision of the strategic and operational plans and associated resource allocations; as well as for developing and managing LAC strategic planning agenda, frameworks and processes, including objectives, accountability frameworks and indicators of success.
Work on the development of MRRS for LAC has started during 2004-05 and will continue in the current fiscal year.

Engagement type and objectives: This is a Modern Management/Performance information type of engagement. Focus: to provide support to management in the development of MRRS (performance indicators, baseline data and targets for the future) for LAC.

Engagement Name: Support to Programs and Services branch

Administrative Area: Programs and Services Branch

Activity Description: This will involve the completed implementation of a performance measurement framework for Programs and Services Branch through the provision of methodological and analytical support to the working group responsible for implementation of the framework. The focus for 2008/09 will be to build on the success to date with an emphasis on developing performance measurement capacity in the sector, target setting and analytics.

Engagement type and objectives: This is a modern management/performance information type of an engagement. Focus: to assist Programs and Services branch in developing performance objectives and performance indicators.

Engagement Name: Genealogy Strategy

Administrative Area: Programs and Services

Activity Description: The Canadian Genealogy Center is a joined initiative of LAC and the Department of Canadian Heritage developed in the context of the Canadian Culture Online strategy. It offers a single window access for researchers and family historians to genealogical resources nationwide. The latter are offered by a network of partners to provide a wide range of products and services. The following goals have been established for the Center:

• Contribute to the federal government strategy of providing opportunities for citizens to create online communities and to work together to develop Canadian cultural content, making it available on the Web;
• Provide the technical infrastructure within the resources of the Center to allow genealogical communities that have not yet been able to place their products and resources on the Web to do so, allowing them to share their information with all Canadians;
• Provide a site on the web where genealogical products created by Canadian genealogical communities will remain under Canadian ownership and under the control of Canadian cultural institutions, while being shared by all at no cost.

Engagement type and objectives: This is a modern management/performance measurement consultative engagement. As such it could assist the Canadian Genealogy Center in: the development of key measures to monitor its overall performance; the development and implementation of mechanisms to monitor service quality and efficiency of service delivery; the design and conduct of surveys to assess client satisfaction; and with the monitoring of service standards established by the Center.