Canadian Digital Information Strategy

4. Trends and Issues

Previous | Table of Contents | Next

The trends and issues described in this chapter are based on survey findings, including discussions held during the survey with key contacts, and the personal knowledge of the authors of this report. While the major focus of the survey was on 'key organizations' involved in some aspect of digital information in Canada (see section 2.1), certain trends and issues became apparent as the survey information was collected and analyzed. The chapter contains a discussion of some of the major trends and issues in the creation, access, and preservation of digital information in Canada as well as the design, development and implementation/maintenance of the infrastructure required to enable these activities. This sets the stage for the next chapter of the report, in which suggestions are offered concerning how these trends and issues can inform the development of a 'way forward' strategy for the organization of a national summit on digital information.

4.1 Digital Information Activities

Increasingly the information generated to support every aspect of Canadian life is being recorded in digital form. Digital information is no longer restricted to data in large applications systems. It can be found everywhere - from the email messages connecting Canadians with each other and others around the world, to the photos documenting everyday life, to the valuables documents that protect rights and entitlements, to the published and unpublished forms of information that contribute to a knowledge-based society, to the computer games, films, and music that serve as sources for Canadian entertainment.

Twenty to thirty years ago, Canadians were less concerned about the management of digital information simply because they were rarely exposed to it. Today, many segments of Canadian society have wholeheartedly embraced the digital environment and the implications of this are profound. Digital information, in its many forms, has become a major and important asset to Canada and Canadians and its significance is growing exponentially. From farmers accessing agricultural information, to seniors securing their benefits through online application to government benefits programs - from the purchase of digital music over the Internet, to the cyber-laboratories used by Canadian researchers, digital information is transforming every aspect of our lives as Canadians.

Creators and publishers have discovered the power of the Internet for the delivery of content to users. A growing service industry has emerged to assist providers in offering their content in digital formats, while other providers are developing the underlying technological and management infrastructure required to enable the creation, access and preservation of digital information.

However, the stewardship of digital information produced in Canada is disparate and uncoordinated. There is a growing awareness of the challenges of managing this growing body of digital information, which comprises more and more of Canada's social interactions, knowledge base, and cultural artifacts. Planning for what is created and used is specific to the organization and its interests. The sharing of experiences in managing digital information (including identifying common interests and issues) is rare, and expertise in digital information is scattered. Digital information is extremely transient and acquisition strategies are dispersed and uncoordinated. In the area of digital preservation, which involves extremely complex processes at both the organizational and technical levels, comprehensive strategies are not yet being employed. Many feel that much of the digital information being created today will be lost forever.

These and other issues will shape the objectives and program for the proposed National Summit and inform the nature of a National Strategy on Digital Information which should emerge as a result of the Summit. Given their breadth and complexity, these issues must be clearly identified, defined and understood if the objectives and program for the Summit are to be relevant. The following sections list key issues and trends affecting the creation, access and preservation of digital information as well as its underlying infrastructure.

4.1.1 Creation of and Access to Digital Content

The actions and transactions that are carried out by Canadians every minute of every hour of every day are increasingly electronic, creating important digital artifacts of Canadian life. The volume and the diversity of digital information being generated is growing exponentially and the digital environment is on its way to becoming the principal medium for the creation and distribution of documents, articles, newspapers, music, websites, moving images, recorded sound, games, images, books and much more. Digital objects are becoming more complex and may contain a wide and mixed-range of formats and be composed of dynamic or distributed elements.

The availability of digital content varies significantly depending on the type of creator/publisher (ie. commercial, cultural, private sector, etc.), the type of access model employed (subscription based, pay per view/download, free access, etc.), and the content management system in use. A whole range of services is present to assist content providers with the creation, access, and preservation of their content. Examples of the types of services being offered are digital hosting, metadata, digitization, preservation, etc. Content aggregators, whose role is to collect digital content and provide an access point for these resources, are plentiful and can be found in virtually all sectors. There has also been a growth and improvement in search and retrieval services, portals, and registries to aid users in finding the information they are looking for.

The growing number and complexity of digital objects, along with the variety of economic and access models applied to these digital resources presents a significant challenge for those who wish to ensure their long-term preservation. Content creators, for the most part, are concerned with issues of immediate accessibility, rather than permanence, and thus have not undertaken to employ the technologies and standards and practices that will better ensure a long-life span for their content.

4.1.2 Preservation of Digital Content

From the findings of the survey and discussion with contacts in the various sectors described in this report it appears that there is a growing sense of urgency with respect to the preservation of digital information. Many organizations are beginning to recognize that preservation involves more than simply placing the information on durable media. Preservation in the digital environment is an active management issue, as much as a technological one. Indeed, media storage and the storage of digital information in environmentally controlled facilities are important. Equally important, however, is the need to store the information in formats that account for changes in technology; the need to ensure that sufficient metadata is associated with the digital information to enable its continued access and understandability; and, above all, the need for an accountability framework to ensure that roles and responsibilities for digital information preservation are clearly assigned. In most Canadian organizations such active management frameworks have yet to be designed and implemented.

While it is difficult to generalize across sectors, the survey found that no specific digital content type was being preserved in a comprehensive way. Regardless of the sector or the progress made towards preserving digital information, no organization is in a position to claim that it has solved the digital preservation problem. Internal documents and records are particularly neglected, while "published content" such as academic articles, newspapers, monographs, websites, music, and government documents are being preserved on an ad hoc basis. The Electronic Collection of Library and Archives Canada probably maintains and actively preserves the most comprehensive cross-sector collections of Canadian digital "published content". There are also large collections of digital information in some Government of Canada databases, such as those maintained by Statistics Canada and natural resources agencies such as Fisheries and Oceans Canada and Environment Canada that are also being preserved. Individual libraries (academic, public and private) are digitizing and collecting digital material, but have avoided anything that is not out of copyright.

Preservation issues are of particular significance in the modern office where there are few controls over the creation, transmission and filing of electronic documents and records. The absence of such controls coupled with the general lack of assigned accountability, effective policy direction, and relevant and effective standards, practices, and technologies are leading to an information management crisis in many organizations. Within this environment, there are rising fears that the continued relevancy, currency and trustworthiness of critical electronic documents, records and published materials may be at risk.

On another front, there appears to be a general lack of clarity about the legality of copying, collecting, providing access to, and preserving digital objects that are covered by Canadian copyright legislation. For example, libraries that license digital materials are unsure about whether they have the contractual right to preserve those materials. Preservation agencies may need to copy publications repeatedly over time, yet the law specifies that only one copy may be made for preservation purposes. And, publishers who protect their publications with technological measures may be preventing them from being copied at all. Numerous issues surrounding copyright in the digital environment need to be clearly articulated and resolved if responsibility for collecting and preserving digital information is to be assumed by organizations who are not the holder of copyright for that material. These and other issues seriously hamper any efforts made towards a comprehensive approach to digital preservation in Canada.

4.2 Digital Information Infrastructure

4.2.1 Laws and Policies

Laws and policies are finally catching up to the requirements of the digital environment. The best examples are the "Personal Information Protection and Electronic Documents Act" (PIPEDA) and the federal government's "Management of Government Information" (MGI) policy, both of which focus on information in digital form. The shift to media neutral laws and policies that are oriented to the digital environment is having an impact on the status accorded to this form of information. No longer is digital information viewed as an ephemeral output from paper-based processes where the 'official record' is in paper form. Digital information carries the same status under law and policy as its paper counterparts.

Questions remain, however, about the ability of organizations to respect all of the provisions of these laws and policies. There are two critical issues: One relates to the challenges of establishing the attributes of authenticity for digital information objects subject to a given law or policy; and the second relates to the ability of organizations to preserve the authenticity of digital information objects for as long as they are required to be retained. While research is underway through initiatives such as InterPARES and standards are beginning to emerge such as those made available through ISO, generally accepted criteria, standards and practices have yet to be established to enable organizations to know with confidence that they are capable of preserving an authentic object in an electronic environment. This can have significant implications for organizations subject to laws such as PIPEDA (where the authenticity and integrity of personal information must be retained according to formal retention standards that may reflect lengthy retention periods).

Other laws are also having an impact on the management of digital information. For instance the extension of the legal deposit legislation in Canada to include digital publications will presumably assign greater responsibility for the collection of digital content to LAC. Similarly the legal requirement for government organizations to establish retention and disposition authorities for digital information will have an impact on the ability of archives to secure digital records of archival value.

Preservation policies are in their infancy at the organizational or inter-organizational level in Canada. The survey found that few digital initiatives had comprehensive preservation policies in place, with the notable exception of the federal government, Library and Archives Canada, and a few other organizations (ie. Government of Alberta: Digital Preservation Strategy; Stewardship of Scholarly Resources in a Digital World - A CARL Position Statement). If the preservation of Canada's digital heritage is to occur, the creators and publishers of digital information will have to begin to implement preservation policies as well.

4.2.2 Standards and Best Practices

In general, standards and best practices for digital preservation have been developed outside of Canada. Examples include the OAIS model, standards set out in the Research Libraries Group-OCLC Report "Trusted Digital Repositories: Attributes and Responsibilities", and migration, emulation, and other approaches to digital preservation that are being tested by national archives in several countries (e.g. The UK National Archives, the National Archives of Australia, etc.).

The survey found a few instances in which Canadian organizations have published standards or best practices for sector specific content. For example, the Canadian Heritage Information Network has published a paper setting out best practices for the digital preservation activities for museums. Library and Archives Canada has published a best practice aid for the management of digital information in government, and standards for describing and managing geomatics information have been developed and published.

The InterPARES Project has contributed usefully to the current body of standards and practices surrounding digital preservation. Results of the first phase of InterPARES have been published in the book: "The Long-term Preservation of Authentic Electronic Records: Findings of the InterPARES Project". The second phase of the project, which was launched in 2002, focuses on the development of model policies, strategies and standards aimed at ensuring the longevity of a wide variety of content types. So far, the project team has released "The Chain of Preservation Model", which illustrates the relationship among activities of records creators and records preservers. Work is also continuing on case studies that are examining the preservation of records produced in complex digital environments ranging from artistic and scientific to e-government and e-commerce.

Despite the existence of standards and best practices, their level of application in the Canadian context has not been comprehensive. This is especially true with respect to the preservation of digital information. In general, the library, archives and museum communities have taken the lead in addressing digital preservation issues. However, they have not taken on the kind of comprehensive approach outlined in models such as the Open Archival Information System (OAIS). The survey found that preservation activities in Canada are restricted to the use of standardized formats, the creation of back-up copies, and the use of durable physical formats. There is no certification process to establish that a digital repository can be 'trusted' which means that there is always the risk that institutional repositories may not be capable of preserving digital information even though their specifications may suggest that they can. Preservation metadata is rarely being assigned and even specialized preservation services do not provide the full range of management activities required to ensure the long-term preservation of digital objects.

Meanwhile, existing models for digital preservation may not even be sufficient to address the preservation of all types of digital material. For instance, the OAIS model addresses the management of objects as individual entities and does not account easily for the management of the relationships among objects (i.e. records) that document decision-making. Nor does it account for the governance and management frameworks that need to be in place to ensure that the underlying infrastructure (the focus of the OAIS model) continues to be relevant and effective.

4.2.2.1 Metadata and Interoperability

The issue of metadata deserves special attention because of its importance to all stages of the life cycle of digital information from creation and collection, to access and use, to retention and preservation. The effective organization of digital information to support both access and preservation depends on the management and organization of metadata. Metadata interoperability, in particular, takes on an important significance when building large collections, with diverse content types and metadata schemas. Metadata can make it possible to search across multiple collections or to create virtual collections from materials that are distributed across several repositories. Metadata also enables access to and understanding of the relationships among digital objects, such as the records that document important decisions and actions, and the functional and organizational context within which they were generated and used. Incompatible or incomplete metadata, on the other hand, impedes resource discovery and access. Although differences between metadata records can be reconciled using metadata crosswalks that "translate" between different metadata element sets, the use of disparate standards exacerbates the silo effect and is likely to hamper effective searching or database management. Further exacerbating the silo effect are the varies metadata models, standards, and tools employed by various information management disciplines - file classification schemes for records managers, cataloguing for librarians, data models and data architectures for data managers, etc. - each reflecting distinct (but ultimately highly relatable) principles, requirements, and techniques.

The rapid growth of digital information has been accompanied by a proliferation of metadata schemas, each of which has been designed to fulfill the requirements of a particular user community and content sector. A plethora of custom-designed metadata standards and applications are currently available for use by content managers, such as MARC, NISO Z39.87-2002 Technical Metadata for Digital Still Images, the DIG35 Specification for digital images, the MPEG-7 standard for audio and video content, CanCore Application Profile for Learning Objects, and many more. That being said, the use of Dublin Core metadata seems to be on the rise. It has been adopted by a number of sectors including by the Government of Canada for its web resources as well as many in the scholarly and cultural communities. The Open Archives Initiative Protocol for Metadata Harvesting, which uses Dublin Core metadata, and has a fairly high profile has likely had the affect of increasing the use of the Dublin Core metadata in many instances.

The survey identified various initiatives aimed at improving interoperability within certain sectors (e.g. EduSource and the CARL Institutional Repositories Project that aim to improve the interoperability of content and repositories), but interoperability across sectors is still quite rare. Records management classification schemes for digital documents are incompatible with data architectures and standards established for data in applications systems. These in turn are incompatible with those developed by the geomatics or learning objects communities, which have developed their own highly sophisticated approaches to defining metadata standards and architectures. As organizations move towards greater information sharing and collaboration, there will be an increased need to break down the silos and develop enterprise-wide, cross sector, inter-operable metadata architectures and standards.

4.2.3 Human Resources Capacity

There was a growing recognition among the organizations surveyed that human resource capacity in the areas of digital information management and digital preservation is lacking. The capacity of content providers, in particular, appears to be quite low. In many sectors, information managers lack training in the processes of digital preservation and awareness of the responsibilities required for managing digital information.

The issues connected with awareness and training relate very strongly to the broader issue of human resources management. A shared perspective on the nature of the work required to manage digital information and to build and maintain the comprehensive infrastructure required for its effective management has yet to be established. It is only after establishing a clear understanding about the nature of the work, that steps can be taken to develop job descriptions, define accountability relationships, and identify competencies. Unlike human and financial resources, the roles and responsibilities for managing the digital information resources have yet to be reflected in job descriptions. Aside from the work of ALARM (Alliance of Libraries, Archives and Records Management), a competency framework for the management of digital information has yet to be defined and there is no mechanism in place for ensuring that whatever competency profiles are developed have authority and can be maintained. These factors inhibit the development of comprehensive, relevant and effective education and training programs (not to mention recruitment programs) to fill identified gaps, rewards and recognition programs that are consistent in their design, and performance measures that are applicable and measurable across information management disciplines and domains.

An issue of particular concern is the capacity required to address the preservation of digital information. Currently education and training for digital preservation in Canada is targeted at libraries and archives. Library and information studies schools across Canada have begun to incorporate courses that touch on the issue of digital preservation into their curriculum (ie. the Electronic Records stream at the University of Toronto's Faculty of Information Studies). Some community colleges are also addressing digital information but their courses are few and poorly coordinated. Professional associations such as ARMA and the ACA offer training but these are often restricted to specific workshops and training sessions offered at conferences or as special events. Overall, current university and community level education and training in the preservation of digital information appears to be inadequate. An inter-disciplinary approach to education and training is needed, as is the establishment of educational standards in this area.

4.3 Leadership and Governance

As with other activities surrounding the creation, access, and preservation of digital information, leadership and governance efforts are, for the most part, sector specific. For example, Canadian Heritage has a strategy to support and develop Canadian cultural resources on the Internet. Through the Canadian Culture Online program and the Canadian Heritage Information Network, Canadian Heritage administers funding programs, policy initiatives, and R&D activities aimed at developing a critical mass Canadian cultural content on the Internet. Likewise, in the education sector, Industry Canada has a played a strong leadership role in its support of the development of standards and best practices, as well as R&D. Other sectors, especially private industry, lack any discernable coordinated leadership.

The survey found no existing mechanisms (e.g. committees, etc.) in place to address digital information issues at the national, multi-sector level. While inter-jurisdictional partnerships are growing, through pan-Canadian, international initiatives, and cross-sector initiatives such as InterPARES, there are still large gulfs between sectors regarding the preservation of digital information. The silo analogy probably most accurately portrays the current environment. This is especially evident in the case of digital preservation. Content creators are often not aware or interested in developing their resources so that they can be preserved in the long-term and digital content is being acquired and preserved in an ad hoc manner. There is a need for a governance structure to coordinate the digital information environment in order to ensure that digital information will be available and accessible in the future.

Exacerbating the situation is the fact that each discipline (records management, library services, etc.) and sector (newspaper, music, broadcasting, publishing, etc.) supports its own approach to the management of digital information. There is little sharing of standards and practices and experiences across sectors. With some exceptions, disciplines such as data management, records management, library services, and web content management tend to work in isolation from one another. This isolation has led to distinct cultures, philosophies, professional values, as well as policies, standards and practices and even technologies. It has even led to distinct vocabularies as each discipline builds its own definitions of key concepts (e.g. 'record', 'publication', 'digital object', digital content', etc.). The lack of clarity and a shared understanding of key concepts are huge inhibitors to cross-disciplinary and cross-sector communication.

The lack of inter-sector approaches can be extrapolated to individual organizations where the management of digital information is often scattered, discipline-specific and viewed from the perspective of the individual program or business line rather than as a valued corporate asset that could benefit the organization as a whole. Often issues concerning the creation, access and preservation of digital information are subsumed under the information technology area of a given organization. Appropriate roles and responsibilities for the management of digital information (especially the collection and preservation of digital content) have yet to be defined and accountability is unclear, poorly assigned or non-existent. Any inter-disciplinary and enterprise-wide initiatives that might be established are often hampered by the lack of leadership, poorly assigned accountability and the absence of effective governance and management frameworks.

Digital information must be managed throughout its lifecycle and this requires the involvement of multiple stakeholders. This means that decisions regarding access and preservation of digital content are distributed across a number of different stakeholders with diverging interests. Some are concerned with the immediate commercial value of their resources, others are interested in their preservation, and others with access to and re-use of material. It is little wonder that a national vision or strategy for digital information has yet to be established in Canada. However, if Canada is to maintain and preserve the digital information that contributes to its documentary heritage, then an inclusive and comprehensive approach to managing digital information is crucial. Strong leadership by key stakeholders is needed to navigate Canada in the ever increasingly digital universe.

Previous | Table of Contents | Next