This archived Web page remains online for reference, research or recordkeeping purposes. This page will not be altered or updated. Web pages that are archived on the Internet are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats of this page on the Contact Us page.
August 2012
| DPR | Departmental Performance Report |
|---|---|
| GC | Government of Canada |
| IM | Information Management |
| IRBV | Information Resources of Business Value |
| IREV | Information Resources of Enduring Value |
| ISDA | Institution-Specific Disposition Authority |
| LAC | Library and Archives Canada |
| MIDA | Multi-Institutional Disposition Authority |
| OAG | Office of the Auditor General |
| PAA | Program Activity Architecture |
| RDA | Records Disposition Authority |
| RDACS | Records Disposition Authorities Control System |
| RKAI | Recordkeeping Accountability Instrument |
| RPP | Report on Plans and Priorities |
| RSC | Regional Service Centres |
| SGB | Society and Governance Branch |
| TBS | Treasury Board Secretariat |
The formative evaluation of strategic outcome (program) 1.0, “current government information is managed to support government accountability,” which is part of Library and Archives Canada’s (LAC) Program Activity Architecture (PAA), was included in the evaluation plan for 2011‑2012, approved by the Deputy Head and Librarian and Archivist of Canada.
Responsibility for PAA strategic outcome 1.0 and its two support activities, i.e., 1.1 “Development of regulatory instruments and recordkeeping tools” and 1.2 “Collaboration in the management of government records,” is entrusted to the Society and Governance Branch (SGB) and to the Chief Operating Officer. One of SGB’s key activities is to facilitate implementation of the Directive on Recordkeeping (the Directive) of the Treasury Board Secretariat (TBS), adopted pursuant to the Policy on Information Management (the Policy). The work includes development and implementation of the recordkeeping methodology to assist federal departments and agencies in managing their own information resources and to enable LAC to issue disposition authorities more efficiently. Ultimately, it is expected that activities 1.1 and 1.2 will lead to the following two outcomes:
The objective of the formative evaluation was to present a neutral opinion on the program’s chances of success, focusing on governance and delivery. The main evaluation questions were as follows:
We find and recommend the following pursuant to the objective of this formative evaluation.
Governance
Governance and organizational structures are generally clear, notably in LAC. The management structure of LAC is effective. Regarding the clarity of roles and responsibilities of the different actors, there are varied perceptions. However, numerous stakeholders found LAC roles and expectations with respect to implementing the recordkeeping methodology to be unclear.
RECOMMENDATION 1: It is recommended that LAC continue working towards clarifying its roles and responsibilities. If every group fully assumes its responsibilities, the result will be improved management and monitoring, and more specifically, improved outcomes.
Implementation
The evaluation focused on the content of the memoranda of understanding and RDAs. Although notable progress has been made since the 1990s, there are still a number of RDAs needing to be put in place or updated. More than 200 institutions are subject to the Library and Archives of Canada Act (the Act), and many of them are not covered by any agreement at this time.
The recordkeeping methodology, which includes a comprehensive disposition methodology, is specifically intended to simplify and accelerate the disposition analysis and approval process. The recordkeeping methodology early adopters are pleased with the tools, training and support offered by LAC. Nevertheless, the implementation process is requiring more time and resources than planned.
Since 2007, the resources dedicated to the Directive and the recordkeeping methodology have steadily increased, reaching 85% of total program resources in 2011‑2012. Yet it appears unlikely in the short term that the objectives of the recordkeeping methodology can be achieved. The extent of resources required for implementing this methodology was underestimated.
RECOMMENDATION 2: In light of the fiscal climate and results achieved, it is recommended that work plans be reviewed and that provisions be made for an accurate timeframe for estimated resources, contingency plans for adapting to contexts and organizations, and an in-depth examination of the impact of the digital transition.
In order to limit accumulation of information resources in federal institutions, a new service model has been developed and gradually implemented. Federal organizations and LAC appear to be successfully managing the new model, the objective of which is to reduce legacy records and storage costs.
One of the key aspects of this evaluation is performance monitoring. Monitoring mechanisms include surveys, institutional performance measurement and operational data collection. There is no tool allowing easy verification of the effectiveness of memoranda of understanding on the transfer of information resources, transfer schedules or the scope of application of disposition measures within departments, or the percentage of institutions subject to the Act that have memoranda of understanding.
RECOMMENDATION 3: It is recommended that information systems be adapted to function as actual performance monitoring dashboards and to provide an overview of the scope of application of disposition measures in the departments and agencies subject to the Act.
Successes to date
As the early adopters testing is not yet complete, it is too early to know the final impact of the recordkeeping methodology. No LAC document describes in detail (by sub-activity) how the amounts allocated to programs are spent. The lack of financial data on costs makes analysis of program efficiency impossible.
RECOMMENDATION 4: It is recommended that financial reporting be established to clearly indicate the cost of each of the components of the program activities for management and accountability purposes.
We wish to express our gratitude to the management and staff at LAC, as well as, our partners and respondents from the departments who took part in this evaluation. They have shown cooperation and patience in supporting the work of the evaluation team throughout our mandate.
As indicated in the report, the recordkeeping methodology has evolved as lessons were learned during its implementation by the 13 early adopters. Therefore, since the preparation of this report, important changes were made to the Disposition and Recordkeeping Program using the recommendations of this evaluation.
Overall, the findings of the evaluation are positive and the recommendations are constructive, thus contributing to improved program delivery. Consequently, management will pursue the implementation of the action plan in response to the recommendations made in this report. Increased effort is necessary to improve the methods used in order to remain accountable for the success of the activities and to achieve results.
This report presents the formative evaluation of strategic outcome 1.0, “Current government information is managed to support government accountability” (hereinafter referred to as the program) and Library and Archives Canada’s (LAC) Program Activity Architecture (PAA) activities 1.1 “Development of regulatory instruments and recordkeeping tools” and 1.2 “Collaboration in the management of government records.” This evaluation was provided for in the 2011‑2012 evaluation plan, recommended by the evaluation committee, and approved by the Deputy Head and Librarian and Archivist of Canada. It was carried out by the evaluation team of LAC’s Audit and Evaluation Directorate, with contributions from the consultant firm Goss Gilroy Inc.
This report identifies the purpose of the evaluation, as well as the recordkeeping methodology. Section 4.0 presents the evaluation findings, and section 5.0 sets out conclusions and recommendations.
The Library and Archives of Canada Act (the Act) came into force in 2004 and created LAC with a mandate to:
With its mandate to serve as the continuing memory of the federal government and its institutions, LAC has exclusive responsibility for the disposition of records and the acquisition of government records of enduring value (archival) (see definition in Appendix C). It also provides assistance and guidance to federal institutions on matters of recordkeeping.1 Indeed, LAC plays a key support role with federal institutions in the area of information management (IM) to ensure that all Canadians have access to information resources. LAC offers its services to more than 200 departments and agencies subject to the Act and 75 other institutions not subject to the Act.
LAC’s Chief Operating Officer and the Society and Governance Branch (SGB) are responsible for program delivery. SGB personnel are concerned mainly with the following: development of instruments and guidelines on the disposition of information resources; disposition and recordkeeping awareness and training; advice to departments; management of legacy records; and storage of records in the Regional Service Centres (RSC). These employees also perform certain duties relating to the acquisition, description and resource discovery subsumed under LAC Strategic Outcome 2.0 “Canada’s continuing memory is documented and made accessible to current and future generations.”
In November 2003, the Auditor General of Canada tabled a report focusing on the protection of cultural heritage in the federal government. The report revealed that “departments have given little attention to information management in recent years” and that the National Archives of Canada “has not yet developed the tools it needs to acquire efficiently those government documents that may be of historic interest and archival importance.” In response to these observations by the Office of the Auditor General (OAG), LAC organized a series of Deputy Minister Roundtables on IM and recordkeeping.
In 2006, LAC, in cooperation with departments and agencies reporting to the Treasury Board Portfolio Advisory Committee, was mandated to develop a Change Agenda aimed at supporting senior officials in their efforts around effective public administration and making recordkeeping a core public service value. A working group of assistant deputy ministers provided insights and guidance on the deputy ministers’ recordkeeping recommendations. In response to certain observations in the OAG report, in 2008, LAC, working in collaboration with TBS, prepared the Directive on Recordkeeping (the Directive) and the recordkeeping methodology, including the comprehensive disposition methodology.
In 2009, LAC undertook a large-scale modernization effort taking into account the new ways in which information is created and accessed, including digital records, Internet and social media. Key to that modernization effort is a more collaborative relationship between LAC and its various partners and the review of LAC work processes. The SGB at that time was entrusted with a modernization innovation initiative aimed at applying “[...] a modernized framework and tools in [LAC’s] work with all Government of Canada institutions to ensure the effective management of government information.” 2 As well, since 2010, TBS, in cooperation with LAC, has been piloting the federal Digital Office Initiative, the goal of which is to move toward the concept of paperless government. LAC has announced that, as of 2017, it intends to accept only digital records in its archives.
In 2011‑2012, the resources dedicated to the program 1.0 reached approximately $10.7 million and 211 full-time equivalents, or about 9.5% of LAC’s total expenditures.
In 2011‑2012, LAC established a new PAA (see Appendix E) defining strategic outcome 1.0, “Current government information is managed to support government accountability,” as well as activities 1.1 “Development of regulatory instruments and recordkeeping tools” and 1.2 “Collaboration in the management of government records.” The objective of these two activities is to allow and facilitate the management of information within Government of Canada departments and agencies to ensure that government records of archival and historical interest are appropriately identified and preserved and to manage the records disposition process. These program activities involve the development of standards, IM strategies and tools, including training, in order to help federal institutions comply with the Directive, among other things.
To measure and assess program performance, the evaluation uses logic models that define key activities and the main outcomes expected from the programs. In January 2011, LAC developed the following logic model (see Figure 1) for strategic outcome 1.0, “Current government information is managed to support government accountability.”
Figure 1: Program 1.0 Logic Model, Government Information
The objective of the formative evaluation was to present a neutral opinion on the program’s chances of success, focusing on governance and implementation.
The main evaluation questions were as follows:
The detailed evaluation questions are presented in Appendix B.
This formative evaluation was carried out from November 2011 to March 2012, and covers the period from 2008 to 2012. It is based on data from several sources collected by the usual methods, namely, document review and key informant interviews.
Documents were reviewed to tease out a profile of the activities and obtain information to answer most of the evaluation questions (see Appendix F for references). These documents included:
A total of 27 individual and group interviews were conducted with 39 key informants, among them, LAC personnel and senior management, representatives from various departments and agencies (e.g., early adopters, Canada School of Public Service, TBS, etc.) and experts. Respondents comprised representatives from within and outside the program.
Document review. The recordkeeping methodology, including comprehensive disposition methodology, has been evolving constantly since 2008; as a result, information necessary for the evaluation was not always available or sufficiently reliable. The part devoted to comprehensive disposition is based on the whole-of-society approach; this aspect of the recordkeeping methodology is still being developed. It was not possible, for example, to measure systematically to what extent immediate objectives have been achieved among early adopters. It also proved difficult to find information on progress made and achievement of objectives for the activities in general, because such data were not collected systematically and regularly, or were not recent enough. In addition, there were no specific timeframes for achieving certain objectives, no set budgets for the use of resources, and no performance/progress measures other than regular verbal reports.
Key informant interviews. Although the interviews enabled collection of detailed information on activities, issues and challenges, we were unable to speak to all stakeholders, including the On Second Thought Advisory consultants, who worked closely on developing the recordkeeping methodology and whole-of-society approach. We communicated with the stakeholders identified on the list provided by the SGB. Some of them were reluctant to participate in the process with LAC and also refused to take part in the consultant interviews.
We were able to compensate to some extent for these challenges and limitations through the triangulation of various data. The interview questionnaires were designed to make up for the gaps identified in the document review process or clarify the ambiguities they contained. In certain cases, the questionnaires even led us to additional documentation. The close relationship between the LAC evaluation group and the consultants was helpful in terms of understanding the program and the issues that emerged during the course of the evaluation.
One major challenge that should be kept in mind has been the frequent changes to the management of LAC’s program 1.0. Since 2008, the branch and sector have seen several different leaders come and go, and this has had a significant impact on communication, both within LAC and interdepartmentally. It was therefore difficult to get a clear picture of LAC’s program management history.
Analysis of efficiency and economy. The evaluation team came up against challenges performing an in-depth analysis of program costs for the purposes of the analysis of efficiency and economy. The financial information relating to the current management of the program does not indicate total expenses. The data appearing in LAC departmental reports were prepared by financial services, as the SGB is not able to present a separate accounting for the expenses of its various activities. The evaluation team had to rely on qualitative methods to analyze resource use (i.e., document review, staff interviews).
Within LAC itself, governance and organizational structures are generally clear. However, comprehension of the institutional framework, particularly stakeholder responsibilities under the Directive, is affected by a high turnover of staff in departments and agencies, and for LAC as well. Moreover, LAC roles and expectations around recordkeeping methodology implementation do not seem to be clear.
Under the Policy on Information Management, IM in the federal government is the responsibility of three main entities: TBS, federal departments and agencies, and LAC. Our evaluation took into account the contributions of those entities to the government IM program and focused on the clarity and effectiveness of governance of the activities under consideration here, namely, 1.1 “Development of regulatory instruments and recordkeeping tools” and 1.2 “Collaboration in the management of government records.”
4.1.1 Roles and responsibilities
In TBS, the Information Management Division has responsibility for the Policy and the Directive, and for monitoring their application by federal departments and agencies. To this end, the division uses Area of Management No. 12, “Effectiveness of Information Management,” of the Management Accountability Framework, as well as assessments/reviews of various departmental documents.
In each department, the senior official responsible for IM takes on the central role of coordinating, promoting and directing IM within the department. And according to the Directive, the IM senior official is responsible for identifying and protecting information resources of business value (IRBV) and for establishing and implementing the key methodologies, mechanisms and tools to support the departmental recordkeeping requirements throughout the information life cycle.
Furthermore, federal employees must ensure that they responsibly manage the information they collect, create and use to support program outcomes and departmental operational needs and accountabilities. Managers at all levels in a department have the role of managing resources, tools and processes. Thus, they are also responsible for IM as an integral part of their program and service delivery and as a strategic business resource.
Under the Act, LAC “[…] provides direction and assistance on recordkeeping for the Government of Canada”; and […] identifies, selects, acquires and preserves government records […] considered to be of enduring value to Canada [...].”3
See Figure 2 for a summary of the various roles and responsibilities.
Figure 2: Roles and responsibilities of partners in information management
| Employees and Managers | IM Senior Official | Deputy Head | LAC | TBS | |
|---|---|---|---|---|---|
Management of information resources of non-enduring value Regulatory Framework |
Information resources are created, acquired and managed directly by employees at their workstations or in a shared protected work area, often informally. |
Information resources with no business value are not declared, managed or placed in a repository. |
Issues the associated policy, directives and manuals. Implements measures to ensure compliance with the Directive. |
||
Information resources of business value (IRBV) Regulatory Framework |
Program managers are responsible for identifying, declaring and managing IRBV, with a view to meeting operational needs and service requirements. |
Responsible for coordinating, promoting and directing IM. |
Responsible for identifying and protecting IRBV. Implements recordkeeping mechanisms throughout the information life cycle. |
Facilitates and supports delivery of recordkeeping; creates recordkeeping instruments and tools; communicates recordkeeping standards. |
Issues and communicates the Directive and manuals and instruments to determine business value. Implements measures to ensure compliance with the Directive. |
Management of information resources of enduring value (IREV) Regulatory Framework |
|
|
|
Identifies, selects, acquires and preserves IREV; issues RDAs; authorizes plans for the retention and disposal of information resources; issues digital transfer standards. |
|
Governance in LAC
Within LAC employees indicated that governance and organizational structures are generally clear. The Framework of Government Records Disposition and recordkeeping methodology were developed and implemented in consultation with the departments.
Participants in the early-adopter initiative, a group of 13 departments and agencies chosen to be the first to use recordkeeping methodology (see Appendix D), generally agree that LAC management structure is effective. For example, there is regular reporting and follow-up for each of the three stages of recordkeeping methodology, described below in section 4.2.2, and each of the 13 projects/adopters has an assigned project leader (archivist).
Perception of roles among federal organizations
In 2010, LAC conducted a survey on the recordkeeping initiative among the IM leads in federal organizations to determine their readiness and ability to manage information resources. The survey indicated that 88% of the respondents knew that LAC was responsible for supporting implementation of the Directive; according to the 2012 version of the survey, that recognition had increased to 93%.
However, in interviews conducted as part of the formative evaluation, it became clear that certain respondents are still not sure who is in the best position to answer questions about the Directive and about recordkeeping methodology: is it TBS or LAC? Indeed, LAC staff, especially in the regions, often have to remind others about recordkeeping roles and responsibilities and the regulatory framework governing IREV management.
Valuation and acquisition services and consulting services in the SGB, traditionally offered by an archivist dedicated to a department, have, since July 2011, been offered by horizontal functional area specialists who act as portfolio managers. This organizational change has still not been entirely accepted by certain respondents, who have significant reservations about this service approach and say they no longer know who their contacts are at LAC.
LAC senior management and staff indicate that it is difficult to communicate LAC roles and expectations with respect to recordkeeping methodology implementation. As a result, LAC employees have sometimes performed duties not within their area of responsibility. Having worked on the preparation of the Directive, recordkeeping methodology, tools and training, LAC employees have now become recordkeeping specialists. LAC senior management acknowledges that employees have carried out duties beyond their responsibilities in order to facilitate and support departments in recordkeeping matters. The employees say that the direction provided by LAC managers has not always been entirely clear and that having to work simultaneously on modernizing both internal and external work processes has frequently delayed them in their work.
Finally, there has been high staff turnover in departments and organizations, such that responsibility for managing recordkeeping in many departments and agencies is problematic. Comprehension of the institutional framework and the Directive suffers as well. The same is true at LAC, where there has been significant turnover of management personnel.
4.2.1 Memoranda of understanding and records disposition authorities
About half of valid RDAs resulted in IREV transfers to LAC. The process takes seven years on average, and the average age of records received is about 30 years. It is widely felt that current disposition agreements do not adequately meet the needs of several organizations, and consequently, not all the activities of a department’s program architecture are shown in the RDAs. The recordkeeping methodology, which includes a comprehensive disposition methodology, should simplify and accelerate the analysis and approval process for the disposition of information resources.
Section 12(1) of the Act states the following: “No government or ministerial record, whether or not it is surplus property of a government institution, shall be disposed of, including by being destroyed, without the written consent of the Librarian and Archivist or of a person to whom the Librarian and Archivist has, in writing, delegated the power to give such consents.”
Following the 2003 OAG report and the work done by the deputy ministers’ committees, LAC has undertaken to review the process for delegating disposition by establishing a methodology for comprehensive disposition that can be applied to all departments in order to simplify and accelerate the process.
The purpose of the memoranda of understanding and the RDAs is to ensure that LAC acquires the documentary heritage of each department or agency (i.e., IREV). The memoranda of understanding determine which IREV will be transferred to LAC and are used to establish the terms and conditions for destroying ephemeral information resources or IRBV. LAC has been signing memoranda of understanding with federal organizations for decades (about 450 memoranda of understanding are registered in LAC systems). RDAs are instruments that authorize federal institutions to dispose of information resources by permitting their destruction, by requiring their transfer to LAC, or by agreeing to their removal from the control of the Government of Canada. Currently, there are two categories of RDAs: Institution-Specific Disposition Authority (ISDA) and Multi-Institutional Disposition Authority (MIDA). MIDAs are used more or less regularly. For example, with regard to human resources management, MIDAs were used by 77% of organizations, while 10% of organizations reported using MIDAs for the management of institutional records in ministers’ offices.4
The SGB also reports that an important exercise was conducted in 2002 to rationalize the number of RDAs and to ensure that RDAs registered in the Records Disposition Authorities Control System (RDACS) were valid and useable by the institutions. Subsequently, over 1,200 of the 2,200 RDAs in the RDACS were revoked. Almost simultaneously, the OAG noted that several RDAs had not been updated or used.5 Currently, nearly 850 RDAs are active, and according to annual performance measurement reports, the SGB issues five to ten memoranda of understanding or RDAs and examines 20 to 45 RDAs each year.
In late 2011, the SGB conducted an internal review of valid RDAs and related records transfers. An initial assessment showed that transfers of records to LAC occurred for about half the RDAs issued and that the average age of records received is about 30 years. The importance of renewing agreements within a shorter timeframe is relative given the ensuing transfer practices. The data in the 2011 review also revealed that transfers to LAC take an average of seven years between the date of signing an agreement and the physical transfer of the records.
According to departmental respondents, issuing a memorandum of understanding requires a large investment of time by all parties involved, especially by LAC. The multiple levels of approval are also perceived as one of the key obstacles for issuing memoranda of understanding. Finally, LAC employees generally find that many organizations are not adequately covered by existing agreements on disposition in the sense that not all department activities are included in the RDAs. Yet, LAC employees have noticed a substantial increase in the coverage of each department since the 1990s when the macro-appraisal methodology was introduced. This approach has been described in the recordkeeping methodology in relation to comprehensive disposition within organizations. More specifically, its purpose is to simplify and accelerate the analysis and approval process for the disposition of information resources, including digital transfer. It therefore focuses on increasing organization involvement in decision-making about the status of information resources.
4.2.2 Information management tools
The recordkeeping methodology has been used since September 2011 under the early-adopter initiative. Participants appear to be pleased with the recordkeeping methodology, the tools and training offered by LAC, and the high degree of LAC support and employee commitment. However, the implementation process is taking more time and resources than planned, which is forcing LAC to modify the recordkeeping methodology in order to simplify its application.
In 2008, LAC and its consultants, On Second Thought Advisory, in cooperation with TBS developed the recordkeeping methodology in order to facilitate the process that would allow departments and agencies to comply with the Directive. The recordkeeping methodology requires greater participation by every federal department and agency, so that eventually they will be better able to manage their information resources themselves “by identifying information resources of business value (IRBV) and ensuring the application of identified recordkeeping controls.”6
Recordkeeping methodology applies a risk-based approach and allows federal organizations to obtain RDAs for information resources in all formats, including digital.
As shown in Figure 3 below, the recordkeeping methodology has three phases. The purpose of the first planning and prioritization phase is to assess the department’s recordkeeping situation, to determine the scope of the recordkeeping implementation project and, by the end of this phase, to establish an action plan, which is an integral part of the memorandum of understanding. The department receives the authority to proceed with the disposition of certain information resources through valid RDAs.
The second implementation or evaluation/“One-time Effort” phase, defines the Recordkeeping Accountability Instrument (RKAI) and the comprehensive disposition authority. Specifically, the purpose of this phase is to determine what the department must do to achieve and maintain its recordkeeping and disposition capacity.
The third phase “Ongoing Recordkeeping and Disposition” enables organizations on a daily basis to ensure “ongoing, accountable, and documented recordkeeping by designated Recordkeepers and IM Specialists.” Appendix D provides more details on the Recordkeeping Methodology.
Figure 3: Summary of the Recordkeeping methodology7
LAC has set up work teams focused on: implementing the recordkeeping methodology, developing generic evaluation tools, preparing comprehensive disposition tools, recordkeeping advice and communication, and recordkeeping training programs for consultants and specialists.
This evaluation reviewed the implementation of the recordkeeping methodology to determine the degree to which the methodology is meeting expectations so far, based on the results of the early-adopters initiative.
Early-adopters initiative
The early-adopters initiative brought together 13 departments and agencies that between September 2011 and March 2012 implemented the recordkeeping methodology in their respective organizations. The summary of the feedback session that took place in January 2012 indicates that most of the early adopters are satisfied with the degree of LAC employee involvement in the initiative framework, the support provided by LAC, and the tools made available to them. More specifically, the participants stated that the archivists played an essential support role. LAC was also able to adapt the recordkeeping methodology implementation process to the organizational reality and adapt tools to meet their needs. LAC also managed to ensure that the project remains a priority for the departments and agencies.
Some department and agency respondents reported that the terminology used and the terms of use are not always clear and should be simplified. More links should be made between item 12 of the Management Accountability Framework, which is the effectiveness of information management, and the recordkeeping methodology tools.
The initiative showed that not all organizations will need to complete the three recordkeeping methodology phases. Some early adopters already have recordkeeping practices in place that are meeting the Directive requirements. This has allowed them to move directly to phase 3 (Ongoing Recordkeeping and Disposition) where recordkeeping is done independently on a daily basis.
The early adopters also indicated that the process is taking more time than expected and suggested reviewing the recordkeeping methodology implementation schedules to make them more realistic. Some respondents reported challenges of a technical nature, such as difficulties in listing IRBV (i.e., the substance of the Directive). However, since each organization and its PAA are unique, the guides are difficult to interpret, and the retrieval process could take up to six months.
Some early adopters are convinced that less time will be needed to apply the recordkeeping methodology in the future, because LAC staff will have more experience, and the recordkeeping methodology and tools will be simplified and adapted as the work progresses.
As for the departments, the recordkeeping methodology tools developed by LAC were generally useful and well designed. According to the survey conducted in 2012, 97% of respondents stated that during the previous 24 months their organization had increased its capacity and its level of preparation to manage information effectively. However, with respect to their organization’s readiness and capacity to manage information effectively, on average the respondents gave a score of 3 out of 5 (with 5 indicating a state of readiness or full performance).
Many respondents, including representatives of LAC senior management, stated that due to a lack of resources it will be difficult for certain institutions to meet Directive requirements by 2014. Since some departments are opting to comply with the Directive independently, LAC is not responsible for ensuring that the departments comply with the Directive, but rather for facilitating its implementation by developing the recordkeeping methodology and tools, and by providing training and support to institutions.
4.2.3 Recordkeeping methodology capacity
The objectives of the recordkeeping methodology are unlikely to be met in the short term. The extent of resources needed to implement this methodology has been underestimated. The complexity of recordkeeping methodology and the budgetary context are affecting the resources required by LAC and making objectives more difficult to reach.
Numerous sources confirmed that in recent years, a very large share of the resources dedicated to activities 1.1 and 1.2 had been allocated to the development of the Directive and the recordkeeping methodology, to recordkeeping methodology implementation within the framework of the early-adopter initiative, as well as to designing tools and the disposition process, and to offering comprehensive disposition to each department.
The objectives of the recordkeeping methodology are to facilitate recordkeeping and disposition, and to make the recordkeeping and disposition processes more effective.
One of the major findings of the early-adopter initiative has been the amount—initially underestimated—of resources needed to implement the recordkeeping methodology, since due to its complexity and the level of detail involved, the recordkeeping methodology is difficult to apply. In the current fiscal environment, this is of critical importance since in 2012‑2013 recordkeeping methodology is expected to be implemented in 40 other organizations. In the medium term, it is estimated that about 150 institutions will require support to comply with the Directive. Some LAC employees have also pointed out that archivists will have less time for other issues affecting organizations. Many respondents also mentioned that more training will be needed for LAC staff to implement the recordkeeping methodology on a wider scale.
In LAC’s 2010 survey, institutions were asked to identify the obstacles to implementing the Directive. According to the results, the obstacles most frequently mentioned by the managers interviewed were the lack of resources (59%), follow-up on the low level of familiarity with the Directive (39%), and a lack of attention paid to IM in general (31%).
From LAC’s perspective, numerous employees also report a lack of resources for all activities related to the recordkeeping methodology and the disposition process, and that the timeframes often shift. Another concern is employee health, which could suffer due to burnout. It seems unlikely that all organizations will achieve the recordkeeping methodology objectives in the short term.
4.2.4 Storage model and legacy records
In order to limit the excessive accumulation of information resources within federal institutions, including RSCs, a new service model has been developed and gradually implemented to provide secure, accessible and cost-effective storage of the Canadian government’s IREV. Consequently, at LAC the number of RSCs will be cut from eight to three. These closures represent substantial immediate costs for federal organizations that will have to sort and segregate the records. However, both the organizations and LAC appear to be managing the situation well; with their objective being to reduce legacy records and storage costs.
The November 2003 OAG report noted that many departments hesitate to transfer their records to LAC: “they claim that they need immediate access to meet their needs. When the federal departments and agencies finally transfer records to the National Archives, it is often storage needs that motivate them to do so.”8
The LAC survey conducted in 2010 showed that 61% of organizations have set up a classification structure (completed or almost completed) in order to facilitate the storage, search and extraction of IRBV, while 53% of organizations have set up repositories to store these IRBV. The survey also revealed that three quarters of the organizations interviewed used the RSCs to store records.
LAC has developed a new storage system that makes it possible to save the Canadian government’s IREV in a way that is secure, accessible and cost-effective. The objective of LAC is to have only three RSCs instead of the current eight to store IREV only, and to no longer store IRBV or records of ephemeral value.
The respondents interviewed on the new storage model pointed out the substantial immediate costs and effort required following the closure of some RSCs, since they must examine their files stored at the RSCs and retrieve or destroy those that are no longer of enduring value. At the time of the formative evaluation, approximately 50% of records examined at the RSCs had been destroyed.
Broadly speaking, both the organizations and LAC appear to be managing the situation well. However, some cases remain difficult, such as the Department of Fisheries and Oceans, which does not have the authorities required to handle the records stored at the RSCs. In the short term, the closures will persuade many organizations to proceed with ongoing records disposition using the recordkeeping methodology and tools.
4.2.5 Monitoring mechanisms
Federal organizations have a few mechanisms for monitoring recordkeeping methodology implementation. These mechanisms enable some monitoring of activities and consist of surveys of information managers, institutional performance measurement, and operational data collection. There is no tool allowing easy verification of the effectiveness of memoranda of understanding on the transfer of information resources, transfer schedules or the scope of disposition actions applied within departments, or the percentage of institutions subject to the Act that have memoranda of understanding.
LAC has three ways to monitor its operations, including the planned operations described in the SGB annual plan, and the operations outlined in the SGB performance report that have been completed:
The SGB produces a plan and an annual report that set out the expected strategic results and the results achieved. These reports provide qualitative data, but also some quantitative data, for example, data on institutional performance measurement, awareness activities and training undertaken, the storage model, legacy records management, digital records management, etc.
RDACS lists the authorities by federal organization with a brief description of the content. It cannot perform analyses of all the RDAs or of summary performance reports. The database MIKAN also was not designed to produce reports. Neither system can show the scope of application of RDAs within departments or the percentage of institutions subject to the Act that have memoranda of understanding.
In short, LAC management has no tools allowing easy verification of the effectiveness of the memoranda of understanding with respect to the transfer of information resources received, transfer schedules, or the scope of application of disposition measures within departments, or the percentage of institutions subject to the Act that have memoranda of understanding. In practical terms, surveys and performance measurements are the only tools that can be used to monitor performance at a global scale.
4.3.1 Program 1.0 outcome achievements
The survey conducted by LAC in 2010 indicates that only 20% of departments believe that their organization has the capacity (knowledge and staff expertise) required to manage information effectively. The early-adopter initiative garnered a high level of satisfaction regarding the recordkeeping methodology and other tools used. This suggests that federal organizations using this methodology will find it equally useful, and that this will increase the departments’ readiness and capacity to manage government information.
According to the program 1.0 logic model, the immediate outcome is: “Increased capacity and readiness to manage Government of Canada information effectively.” In the LAC survey of 2010, almost half (49%) of the organizations stated that they followed recordkeeping policies and practices. Fewer organizations stated that recordkeeping policies and practices had been implemented at all levels of the organization. The 2012 survey did not indicate if the organizations have increased their capacity to manage recordkeeping.
Even though not complete, only the early-adopter initiative has allowed us to measure the effectiveness of the recordkeeping methodology. No final conclusions will be known until a new survey of the departments and agencies subject to the Act is conducted. Meanwhile, the satisfaction expressed about the recordkeeping methodology and other tools suggests that these instruments will likely be of use to federal organizations and will increase their degree of readiness and capacity. Moreover, many LAC employees have also observed recent progress within departments and agencies with the departments being more disciplined, doing the clean up, and returning records. However, many respondents stated that it is too early to know the impact of the recordkeeping methodology. In short, the immediate outcome of the program is within reach of many federal organizations.
The intermediate outcome of program 1.0 is: “a government-wide regulatory regime is implemented and government information is managed and disposed of appropriately.” Of the more than 200 organizations that are subject to the Act, 13 have begun to implement the recordkeeping methodology. Approximately 40 organizations will be receiving support from LAC in 2012‑2013. This is not sufficient to achieve the intermediate outcome.
Phase 2 of the recordkeeping methodology, including comprehensive disposition for departments, has not been deployed in its final form. The whole-of-government approach to records disposition (which includes: the generic methodology deployment for the disposition of government information resources, the review of the macro-appraisal framework and the use of generic tools for internal services) sets the stage for harmonization with LAC’s whole-of-society approach. However, the whole-of-society concepts and approach are still under development and there are significant challenges to their integration in the appraisal process. LAC’s institutional performance measurement calls for future surveys to measure this outcome achievement.
4.3.2 Use of resources
No LAC document describes in detail how the funds allocated to programs are spent. Only the overall expenditures for program 1.0 are available. The lack of financial data on costs makes analysis of program efficiency impossible.
The following figure shows the amounts spent within the framework of LAC’s government information program. In 2007‑2008, expenditures were $13.0 million, while expenditures were estimated at $10.7 million in 2011‑2012. Since 2007, resources allocated to the Directive and to the recordkeeping methodology have steadily increased from 33% in 2009‑2010 to 85% in 2011‑2012. See resources summary in Figure 4.
Figure 4: Actual resources allocated to the program9
| 2007‑2008 | 2008‑2009 | 2009‑2010 | 2010‑2011 | 2011‑2012 estimate (*) |
|
|---|---|---|---|---|---|
| Actual spending | $13.0M | $13.9M | $11.2M | $10.5M | $10.7M |
| Full-time Equivalents | 167 | 173 | 177 | 189 | 211 |
* Estimate provided in March 2012 by LAC Financial Services.
No LAC document provides more detail on how the amounts are spent. With responsibilities covering several PAA activities, it is difficult to attribute expenditures within the SGB.
The lack of reliable financial data on the costs of producing memoranda of understanding or RDAs precludes analysis of program efficiency. The comparison of costs before and after the adoption of the recordkeeping methodology and the new processes will be uncertain in practical terms. Data are missing, while recordkeeping methodology and comprehensive disposition are focused on the effectiveness and efficiency of the new process.
On the other hand, the shift to digital will allow the federal government to generate savings since, according to many respondents, storing digital records will be much less expensive. In addition, in view of the results of L’Étude des résultats du bureau numérique de BAC [LAC study of digital office outcomes] “the increased level of collaboration, more systematic and direct access to pertinent information, the simplification of the archiving process, and the significant reduction in paper use” 10 [translation] are important benefits for organizations that could partly offset the high costs of implementing the digital office. The data on operating costs before and after the digital office is set up should be made available in order to enable reporting successes in this area.
Based on the findings in the previous section, we conclude and recommend the following in relation to the objective of this formative evaluation.
The results show that the governance and organizational structures are generally clear, especially within LAC. Participants in the early-adopter initiative, which consisted of 13 departments and agencies chosen to be the first to use the recordkeeping methodology, by and large agree that LAC management structure is effective. Regarding the clarity of roles and responsibilities of the different actors, there are varied perceptions: the turnover rate within federal organizations (including LAC) has resulted in an uneven understanding of responsibilities under the institutional framework and the Directive. In addition, many stakeholders find LAC roles and expectations regarding the recordkeeping methodology implementation to be unclear.
RECOMMENDATION 1
It is recommended that LAC continue working towards clarifying its roles and responsibilities. If every group fully assumes its responsibilities, the result will be improved management and monitoring, and more specifically, improved outcomes.
5.2.1 Memoranda of understanding and records disposition authorities
The evaluation focused on the content of the memoranda of understanding and RDAs. While considerable progress has been made since the 1990s, numerous authorities have yet to be formulated or updated, which explains the need for the recordkeeping methodology and for integrating a comprehensive disposition approach into the recordkeeping methodology in order to accelerate this renewal. LAC continues to face major challenges as over 200 institutions are subject to the Act and many do not yet have an agreement.
5.2.2 Information management tools
The recordkeeping methodology, which includes a disposition methodology, aims to simplify and accelerate the disposition analysis and approval process. All indications are that the early adopters are satisfied with the recordkeeping methodology, the tools, and the training and support provided by LAC. However, the implementation process is taking more time and resources than expected. This has led to modifications being made to the recordkeeping methodology to simplify its application.
5.2.3 Recordkeeping methodology capacity
Since 2007, the resources dedicated to the Directive and the recordkeeping methodology have steadily increased to 85% of resources allocated to the program in 2011‑2012. It seems unlikely that the recordkeeping methodology objectives will be achieved in the short term. The extent of resources necessary to implement the recordkeeping methodology has been underestimated. The complexity of the methodology and the current fiscal climate are affecting the resources needed by LAC and are making it more difficult to achieve objectives.
RECOMMENDATION 2
In light of the fiscal climate and results achieved, it is recommended that work plans be reviewed and that provisions be made for an accurate timeframe for estimated resources, contingency plans for adapting to contexts and organizations, and an in-depth examination of the impact of the digital transition.
5.2.4 Storage model and legacy records
In order to limit the excessive accumulation of information resources within federal institutions, a new service model has been developed and gradually implemented. This model will provide secure, accessible and cost-effective storage of the Canadian government’s IREV. At LAC, the number of RSCs will be cut from eight to three. These closures represent substantial immediate costs for federal organizations that will have to sort and segregate the records. However, both the organizations and LAC appear to be managing the situation well; with their objective being to reduce legacy records and storage costs.
5.2.5 Monitoring mechanisms
One of the key aspects of this evaluation is performance monitoring. Monitoring mechanisms include surveys of information managers, institutional performance measurement and operational data collection. However, no tool can easily verify the effectiveness of memoranda of understanding with respect to the transfer of information resources, transfer schedules or the scope of disposition actions applied within departments, or the percentage of institutions subject to the Act that have memoranda of understanding. Therefore, it is impossible to assess the progress made by all the institutions to meet LAC’s mandate. Without requiring great effort, the system or a completely separate tool should be able to indicate:
RECOMMENDATION 3
It is recommended that information systems be adapted to function as actual performance monitoring dashboards and to provide an overview of the scope of application of disposition measures in the departments and agencies subject to the Act.
5.3.1 Program 1.0 outcome achievements
Many representatives of federal departments and agencies were satisfied with the recordkeeping methodology and LAC tools, especially with regard to the recordkeeping methodology’s effectiveness. Since the early-adopters initiative is not fully complete, it is too early to know the recordkeeping methodology’s final impact.
5.3.2 Use of resources
No LAC document describes in detail (by sub-activities) how the funds allocated to programs are spent. The lack of financial data on costs makes analysis of program efficiency impossible.
RECOMMENDATION 4
It is recommended that financial reporting be established to clearly indicate the costs of each of the components of the program activities for management and accountability purposes.
| Recommendations | Management Response | Planned Actions | Expected Completion Date | Responsibility |
|---|---|---|---|---|
| Recommendations, as described in the report. These recommendations cannot be modified. | State whether or not you accept the recommendation (if not, please write a brief explanation). | Briefly and clearly state the action that will be taken to remedy the situation. | Enter an approximate date (e.g., one quarter). | State the level of the authority responsible for implementing action. |
RECOMMENDATION 1 |
Accepted. Management recognizes the importance of clarity in the roles and responsibilities of the parties involved in the recordkeeping initiative. |
LAC will continue to renew its methods and ensure that its activities and priorities are aligned with its mandate. |
|
Director General, Documentary Heritage Relevance Assessment |
|
Fall 2012 |
|||
|
September 2012 |
|||
|
Ongoing |
|||
|
July – August 2012 |
|||
RECOMMENDATION 2 |
Accepted. Management recognizes the need to incorporate a more detailed human resources plan into its business plans. |
Management will improve its decision making by regularly itemizing and reviewing the human resources component of its work plans. |
|
Director General, Documentary Heritage Relevance Assessment |
|
June 2012 |
|||
|
Fall 2012 (once new structure is in place) |
|||
|
September 2012 |
|||
|
July 2012 |
|||
RECOMMENDATION 3 |
Accepted. Management recognizes the limits of current systems to analyze data and produce reports. |
Management will acquire tools and mechanisms to facilitate decision making. |
|
Director General, Documentary Heritage Relevance Assessment |
|
June 2012 |
|||
|
Fall 2012 (once new structure is in place) |
|||
RECOMMENDATION 4 |
Accepted. The proposed recommendation exceeds the responsibility of the “Documentary Heritage Relevance Assessment” branch. The findings apply to all areas of the organization. Management must work with the limitations of the financial systems to analyze data and produce reports. |
Management commits to initiate a discussion to find a corporate solution. |
|
Director General, Documentary Heritage Relevance Assessment Senior Director General, Corporate Services |
|
Summer 2012 |
|||
|
June 2012 |
(Excerpt from Preserving the Archival and Historical Memory of Government. Prepared by Richard Brown. Reviewed and Edited by Candace Loewen, Judith Roberts-Moore and Paul Sabourin. Appraisal and Special Projects, Summer 2001).
To fulfill its mission to preserve government's historical memory, Library and Archives Canada (LAC) must make decisions about the archival or historical value of the information created and maintained by government institutions. The records are voluminous and multimedia, relating to every conceivable element of civil and social life in Canada, and the mass of information continues to grow as government makes increasing use of technology. It is not possible for LAC to preserve every government record, nor would any such endeavour serve the best interests of Canadians. Like any large organization, the Government of Canada produces an enormous number of records which become superfluous after a time—sometimes after a very short time—to the extent that there can be no legitimate administrative reason for keeping them, even if it were feasible to do so. Alternatively, there are many essential records of government activity which provide critical information that must be retained permanently or for long periods of time.
To respond to the challenge of preserving government's historical memory, LAC has adopted an appraisal strategy to identify and protect a comprehensive archival record of the administration of the national state, the machinery of the federal government, and the interaction between government and its citizens. Broadly speaking, the archival acquisition or protection of government records by LAC is related to their national significance as determined through an exhaustive appraisal process in support of the following general objectives:
During the appraisal process, LAC undertakes a rigorous analysis of the functions and activities of federal departments and agencies, seeking to understand their policy, program and service delivery environments and their importance to government and Canadian society at large, and to identify the substance and context of the documentation being created and managed by institutions in support of public business enterprise. Using the knowledge gained through this business systems analysis, LAC acquires selected records from offices and responsibility centres in departments and agencies which best illustrate the operations of government and the broad framework of national governance as expressed in the criteria outlined above. Decisions about the archival status of government records, and permission to dispose of records without archival or historical value, are conveyed to government institutions in the form of Records Disposition Authorities.
LAC is responsible for preserving the Government of Canada's archival and historical memory, and consequently, it bears an obligation to explain to Canadians—and to others who may wish to use its archival holdings—how and why decisions are made regarding the archival preservation of government records.
LAC maintains comprehensive documentation of the government records appraisal process, including the rationale supporting preservation decisions for all Records Disposition Authorities. Through its records description systems and associated holdings management processes, LAC also maintains the attributes of archival government records as evidence; ensures that these records remain in continuous official custody; and keeps with the official archival holdings any other documented analysis leading to the selection or disposal of related records to provide the full context for any records disposition decision making. All of this information is available to the public upon request.
Recordkeeping methodology must ultimately allow federal departments and agencies to manage their information resources themselves “by identifying information resources of business value (IRBV) and ensuring the application of identified recordkeeping Controls”11. The creation of a Recordkeeping Accountability Instrument (RKAI) is one of the crucial stages in the recordkeeping methodology. The purpose of the RKAI is to bring together “the institution’s information resources of business and enduring value, recordkeeping Controls, and associated roles”12. In collaboration with LAC, institutions must also undertake the activities necessary to adopt comprehensive disposition authorities that will ensure that all their information resources are covered by disposition measures.
Recordkeeping methodology is being used within the framework of an initiative set up in 2011 in collaboration with the following 13 departments and agencies:
The 13 organizations were selected to provide a certain degree of representativeness for all federal organizations, especially with respect to size and number of agreements finalized at the start of the process. The early adopters implemented the recordkeeping methodology in three phases beginning in September 2011.
Based on the information gleaned from interviews, it is expected that the main departments, which represent about 20 organizations, as well as 50 to 60 smaller organizations, will be the next to implement the recordkeeping methodology. In the case of the smaller organizations, based on essential institutional documents, such as RPPs and DPRs, LAC expects that these organizations could benefit from comprehensive disposition authorities. Officially, the early-adopter initiative was scheduled to end on March 31, 2012 at the latest.
The early-adopter initiative is being delivered in three phases: the planning and prioritization phase, the implementation or “One-Time Effort” phase, and the monitoring or “Ongoing Recordkeeping and Disposition” phase.
The objective of the first phase is to evaluate the institution’s recordkeeping situation, determine the scope of the project, and establish an action plan. It also provides institutions with the option to proceed with the disposition of selected information resources before a comprehensive disposition authority has been set up. There are six stages in the first phase:
A memorandum of understanding detailing the terms for implementing the action plan is signed at the end of phase 1 in order to allow the department or agency to begin phase 2.
After the RKAI and comprehensive disposition authority have been set up in phase 2, LAC expects that in phase 3 the department or agency will be able on a daily basis to ensure “ongoing, accountable, and documented recordkeeping by designated Recordkeepers and IM Specialists” (information management).
The early-adopter initiative revealed that not all the organizations will need to complete all three phases. Some early adopters do not have to complete all of phase 2 (One‑Time Effort). After the initial implementation phase (Planning and Prioritization), and before the process began, certain organizations had already set up recordkeeping practices and could therefore proceed to phase 3 (Ongoing Recordkeeping and Disposition), during which recordkeeping is done independently on a daily basis.
1 TBS Policy on Information Management.
2 Modernization Innovation Initiative No. 3, LAC, 2010.
3 Quoted at section 8.2 of the Policy on Information Management, TBS 2007.
4 These results are estimates taken from a LAC survey conducted in 2012.
5 Report of the Auditor General of Canada, November 2003.
6 Library and Archives Canada, Recordkeeping methodology Handbook, 2012, pp. 1-3.
7 Library and Archives Canada, Managing Information as an Asset: A Methodology for Effective Recordkeeping, PowerPoint document, n.d.
8 Report of the Auditor General of Canada, November 2003, Chapter 6–Protection of Cultural Heritage in the Federal Government.
9 LAC Departmental Performance Report, 2007‑2011. Under the former PAA, the program corresponded to Activity 1.1: “Managing the disposition of the Government of Canada records of continuing value.”
10 L’Étude des résultats du bureau numérique de BAC, 2011.
11 Library and Archives Canada, Recordkeeping methodology Handbook, 2012, pp. 1‑3.
12 Ibid.