SESSION#02: UNIQUE CONTENT

80 Thousand Pages On Street Art : Exploring Techniques To Build Thematic Collections

Ricardo Basílio
Arquivo.pt, Portugal

Street art, especially graffiti, has a strong presence in the streets of Lisbon. The city authorities have a service dedicated to urban art and relations with artists, the GAU: Gabinete de Arte Urbana (Urban Art Office) (http://gau.cm-lisboa.pt). On its website, more than 500 artists have been registered with their creations since the 1990s. Every year, GAU organises a festival dedicated to street art, the MURO(Wall) Festival (https://www.festivalmuro.pt/).

In 2023, Arquivo.pt (https://arquivo.pt) began a collaboration with GAU with the aim of promoting Arquivo.pt, offering training to the street art community and creating a special collection of web content of their interest.

This presentation describes the process of creating the web collection about street art in Portugal, namely 1) the techniques used to identify thousands of URLs; 2) the type of recording; 3) the difficulties and the lessons learned.

First, we describe several techniques that have been tried. We highlight the technique that simply uses a search engine and a link extractor. This allowed us to obtain around 80 thousand unique URLs from searches for "graffiti" and artist names. Anyone can use this technique, even without being an IT expert to quickly identify large amounts about of web content about a given topic.

We also show how we used automatic search tools to add new URLs (seeds) to our collection. We experimented with Bing Search API to obtain results from Bing and with SearXNG to get results from multiple search engines. For digital curators capable of running scripts, automatic search through APIs is a powerful technique to be explored.

Finally, we consider the identification of content about street art by experts, through the collaboration with the GAU team. In this case, the contents were chosen individually, recommended for their relevance and accompanied by additional information, ready to be included in collections such as the "Street Art" collection promoted by IIPC.

Second, we explain how we proceeded with the recording. To record more than 80 thousand seeds, we chose to limit recording to the page level (single page), aggregating the recording into batches of up to 10,000 URLs and treating some types of pages separately, such as social networks. We used the Webrecorder's Browsertrix-cloud service (beta) and Browsertrix-crawler. The generated WARC files were integrated into Arquivo.pt.

In the third point, we share the difficulties we found in crawling the list of selected URLs, we explain how we broadened the criteria to include pages about street art from other countries and why we tolerated some off-target pages.

In conclusion, the use of various techniques to speed up the identification of existing content on the Web about a given topic is useful for building collections. However, it raises questions that curators should debate. This is an ongoing project and in April 2024 we will publish the results of the interaction between the street art community in Portugal and Arquivo.pt.

Saving Ads: Assessing and Improving Web Archives’ Holdings of Online Advertisements

Christopher Rauch¹, Mat Kelly¹, Alexander Poole¹, Michele C Weigle², Michael L Nelson², Travis Reid^2
1Drexel Univesity, United States of America; ²Old Dominion University, United States of America

Advertisements provide foundational source material for social, cultural, ethnographic, and commercial historical studies. The existence of research library collections for printed advertisements, such as Penn State University Libraries’ “Advertising: History and Archives” and The Smithsonian National Museum of American History’s Advertising collection, suggest the importance of preserving evidence of the mores and norms of a time and place represented by advertisements and their context. Online advertisements have similar—if not greater—cultural significance and impact. Just as physical ephemera in libraries, archives, and museums fuels compelling research, so too do online ads illuminate the contemporary objectives of advertisers, social norms, viewpoints, and ideals in ways that carefully curated news stories cannot.
As the scale and scope of data sources on the web have grown, the creators of web archival tools, such as Ilya Kreymer’s Webrecorder, have added methods to exclude content from capture for both practical reasons : the expense of storing the massive volume of data, and legal or social restrictions: the owners assert copyright, or the content is deemed not appropriate to archive. The maintainers of popular web archives, such as the Internet Archive, have employed these techniques to focus collection activities on certain categories of content, excluding most ad-related resources. Even when the ad-related content is captured, it is often not possible to experience replay in context because of the dynamic components of rich media ads and other restrictions that discourage the rendering of ads outside of their original intended advertising channels. Exclusive curatorial decisions come at a cost. Libraries and web archivists have a strong interest in being able to provide these types of resources for scholars, as scholars cannot study what has not been preserved.

In this presentation, we describe the current state of archival practice as it relates to the preservation of web-based advertising in context, highlight the large gap in the historical record created by the failure to archive such advertisements, and suggest approaches archivists can use to ensure that the temporally and culturally relevant information sources represented by web advertisements are preserved. Attendees will become familiar with the settings in popular tools that control the scope of archival crawling, techniques to broaden the scope of capture to include advertising materials with a reasonable tradeoff to additional storage requirements, and understand how already archived data can be coaxed into replaying web advertisements that are not rendered by default.

Working Together to Capture, Preserve and Provide Access to Digital Artworks

Claire Newing¹, Tom Storrar¹, Patricia Falcao², Sarah Haylett², Jane Kennedy^2
1The National Archives, United Kingdom; ²Tate, United Kingdom

It is well known that interactive, multimedia content is difficult to capture and present using traditional web archiving techniques. We will seek to explain how two organistions with a long standing relationship, developed a new way of working together to improve the quality of complex content held in a web archive.

The first organisation is a specialist art institution, which is also a public body. The second is a national memory organisation which hosts a national web archive. The specialist art institution recognised that some interactive, multimedia content was missing from archived websites held in the national web archive. It undertook a project to capture the content using Webrecorder as part of a focused effort to preserve web artworks. Although it had experience of digital preservation it did not have any experience hosting web archives. The national memory organisation was capturing their websites regularly as part of large scale crawls but did not have resources available to manually capture interactive artworks. It was agreed that it was most appropriate to provide access to the additional content through the national web archive where it could be presented within the wider context of archived versions of its website.

As it was the first time either institution had embarked on a WARC transfer project, all workflows and documentation were either built from scratch or by adapting resources developed for different purposes. This included: composing an agreement between the two organisations, creating a safe and secure workflow for transferring the WARC content, establishing a system for quality assurance checks and developing a methodology for cataloguing. It was a truly collaborative effort across the two organisations which had a good outcome for both.

We will discuss the process followed to achieve the successful transfer and the various legal, technical and practical considerations involved, from the perspective of both the transferring and receiving organisations. We hope to provide the audience with some knowledge of the factors they would need to consider when undertaking similar projects.

Put it Back! Archived Memes in Context

Valérie Schafer
University of Luxembourg

Memes constitute a significant aspect of online digital cultures (see Shifman, 2014; Milner, 2018; Denisova, 2016). Their role in reactions to events like the Ukrainian war highlights their broad influence. However, preserving memes, especially in their original context, proves challenging (Pailler & Schafer, 2022). Initiatives like SUCHO's Meme Wall and the one by the Library of Congress (which archived Know Your Meme but also Meme generator, when creating a particular collection in 2017 “Remix, Slang and Memes: A New Collection Documents Web Culture”) reflect attempts at preserving these cultural artifacts. Furthermore, institutional web archives, as exemplified by projects I led like Hivi (A history of online virality) and Buzz-F in collaboration with the BnF Datalab, have significant content related to memes and viral online phenomena such as the Harlem Shake. Nevertheless, the process of archiving memes remains underdeveloped, resulting also in challenges related to retrievability, searchability, and contextualization.

This presentation delves first into the unique challenges posed by memes concerning contextualization. It explores their circulation across platforms, notably on social media networks, their diverse and ephemeral nature, and the cognitive context intricacies tied to content and usage. Memes can for example vary drastically in meaning based on where they appear – be it a tweet, a post within an activist Facebook group, or other contexts. Understanding these multilayered intricacies is vital for comprehending the broader significance of memes and for developing suitable archiving processes.

Following a brief presentation of the various contextual layers involved into meme analysis, we will delve into the extent to which meme archiving addresses – or could address – these diverse dimensions. It becomes crucial to identify elements that must be preserved to facilitate proper contextualization and potential new approaches to these challenges. One proposal involves for example creating shared ontologies to enrich metadata. Additionally, the idea of specialized meme collections within archives could be explored.

This presentation draws upon the experience gained from two research projects, Hivi and Buzz-F, with the aim of initiating a discussion on context preservation and the challenges posed by ephemeral online phenomena such as memes. It offers insights relevant to the Research & Access theme, touching upon topics such as Unrealized business and technical requirements for web archives research; Creating & providing researchers with datasets from web archives collections; Collaborations between researchers and web archivists.

References:

Denisova A (2016) Political Memes as Tools of Dissent and Alternative Digital Activism in the Russian-language Twitter. PhD thesis, Westminster University.

Milner R (2018) The world made meme: Public conversations and participatory media. Cambridge, MA: The MIT Press.

Pailler F & Schafer V (2022). « Never gonna give you up ». Historiciser la viralité numérique. Revue d’histoire culturelle 5, http://journals.openedition.org/rhc/3314

Shifman L (2014). Memes in digital culture. Cambridge, MA: The MIT Press.

SESSION#03: CONTEXTUAL

Averting the “Digital Dark Age”: The Digital Preservation Moment and the Birth of Modern Web Archiving, 1994-1996

Ian Milligan
University of Waterloo, Canada

The Call for Papers for WAC 2024 was announced on the 20th anniversary of the forming of the International Internet Preservation Consortium (IIPC), a conscious nod to the importance of history in understanding the context of contemporary web archiving. One cannot understand web archives without understanding not only the broader context of the time in which they were created, but the broader historical context of why and how web archiving organizations developed and how particular approaches to web archiving were adopted. Yet histories of web archiving are often quick and somewhat rote. A nod towards the Internet Archive and a few other national libraries launching web archiving projects in 1996, perhaps a glimpse to the Wayback Machine in 2001, and occasionally the IIPC’s 2003 founding.

In my presentation, I provide discussion around the origin moment of web archiving. Why did web archiving begin at the Internet Archive, the National Library of Canada, the Swedish National Library, the Dutch Royal Library, and the Australian National Library all roughly at the same time? I argue that 1994-1996 witnessed a process that built a social and cultural consensus about web archiving.

My presentation argues that the 1990s transformed a debate which had been largely happening within record management and within the archival profession throughout the 1960s and 1970s, and propelled it into public conciseness through the conscious framing of a “digital dark age.” This concept made concerns around digital obsolescence seem like a problem that was not just for the Fortune 500 and governments, but all of society. Ideas spread outwards from academic venues and fora such as the 1994-1996 Task Force on Digital Archiving and research libraries to have broader social and cultural impact.

Between 1995 and 1998, a series of individuals – including science fiction author Bruce Sterling, Microsoft Chief Technology Officer Nathan Myhrvold, information scholar Margaret Hedstrom, technologist Brewster Kahle, documentarian Terry Sanders, and Long Now Foundation founder Stewart Brand – reshaped the cultural conversation to broaden digital preservation from an academic field to one understood as having wide-ranging implications. This would not just be the preservation of technical or corporate documents, but rather the collective digital memory of our society.

Why does this fit with this conference? Curators and users of web archives need to understand the broader historical context which gave rise to their programs, and this can help understand ways in which to motivate it today. When we speak of a “digital dark age” in 2023, we are drawing on an intellectual conversation stretching back 30 years. My goal is that practitioners and researchers, with a better understanding of their history, will be well positioned to explain the value of their programs today.

The Form Of Websites: Studying The Formal Development Of Websites, The Case Of Professional Danish Football Clubs 1996-2021

Niels Brügger
Aarhus University, Denmark

This presentation discuss how we can analyse the historical development of the form of websites based on the holdings in web archives. In other words: what can we say about a website's form, no matter which concrete words, images, graphics, videos and sounds it conveys?

Focus is on the following topics that can help us understand the formal characteristics of a website:

• delimiting the website in space and time by striking the balance between having a website with most web pages, and keeping it temporally consistent
• including the web pages that were not archived, but linked to
• size (number of web pages), and structure (broad/deep website)
• word-heaviness (number of words in total and on average per web page)
• image-heaviness (number of images in total and on average per web page)
• menu items (number of main-/submenu items, vertical/horizontal)
• length of web pages

The formal characteristics of websites are important to study historically since they constitute the changing framings of the content and the use.

Inspiration: Theoretically the presentation is informed by the monograph The Form of News (Barnhurst & Nerone, 2001), in which the authors analyse the various elements that have historically played into making the news look as they have done, and how these forms have changed from 1750 to 2000, including analyses of the role of pictures, the front page, the overall format and design, etc. The presentation investigates how such an analysis would look like if the web is the object of study, keeping the specificities of the archived web in mind (Brügger, 2018). Also, the presentation is inspired by the seminal work of Byrne about the use of the archived web as a source to study football history (Byrne, 2020).

Context: The presentation takes its point of departure in the ongoing research project "Becoming professional: The web presence of Danish Superliga clubs 1996-2021". Part of this analysis is a detailed analysis of the formal development af the soccer club's official websites.

Data: The study is based on the holdings of the national Danish web archive Netarkivet. As for the study's first years Netarkivet's material is constituted by an ingest of relevant Danish websites acquired from the Internet Archive.

Methods: Quantitative methods, using the software Orange, and manual coding.

Contribution: The presentation contributes to the fields of researcher use of web archives as well as to web history by demonstrating how web archives can be used in the study of the development of the web as such and of cultural phenomena in the context of web archives such as the communicative use of the web by sports clubs. As a side effect the presentation also indicates how one can study the completeness of what is archived.

References:

Barnhurst, K.G., Nerone, J. (2001). The Form of News: A history. New York/London: The Guildford Press.

Brügger, N. (2018). The archived web: Doing history in the digital age. Cambridge, MA: MIT Press.

Byrne, H. (2020). Reviewing football history through the UK Web Archive, Soccer & Society, 21(4), 461-474, DOI: 10.1080/14660970.2020.1751474

Challenges of Putting Web Archives in a Comprehensive Context: the Case of Vdl.lu

Carmen Noguera
University of Luxembourg

This presentation aims to showcase the importance of context when analyzing the archived web, may it be for a research on a specific topic using web archives as a primary source, for a particular period – for example, the web of the 1990s– or for the analysis of a diachronic study of a particular website.

The presentation will be based on my experience using web archives for my PhD research on Digital Cultures and their development in Luxembourg from the 1990s to the present day. One of my case studies aims to analyze the origins of e-government and e-citizens in the country. In order to do that, I mapped the communes that had a website in the 1990s and early 2000s, identified their main objectives, and analyzed how they evolved over the years towards greater participation of citizens.

In concrete, this presentation will focus on the analysis of the website of the Ville de Luxembourg (www.vdl.lu) and what additional context is needed to conduct this research through web archives. Based on the challenges I faced during the diachronic analysis of this website using different web archives, such as the Internet Archive, arquivo.pt and the National Library of Luxembourg web archives, I will discuss the needs, strategies, and requests in terms of contextualization, documentation, and metadata, as well as discoverability and incompleteness of data. I will also underline what would be needed in terms of interoperability between web archives.

The need for greater contextualization of the data context and more documentation and descriptive metadata (Milligan, 2020; Webster, 2017; Brügger, Schafer, Geeraert, Isbergue & Chambers, 2020; Venlet et al., 2018; Vlassenroot et al., 2019, among others) is a recurrent request of web archives researchers. The case study aims to demonstrate the practical challenges when analyzing a website and the importance of contextualization and documentation, especially when facing information loss and working with several web archives.

References:

Milligan, I. (2020). You Shouldn't Need to Be a Web Historian to Use Web Archives: Lowering Barriers to Access Through Community and Infrastructure. Newark New Jersey USA: WARCnet. https://doi.org/10.1145/2910896.2910913

Webster, P. (2017). Users, technologies, organisations: Towards a cultural history of world web archiving. In N. Brügger & N. (Eds.), Web 25. Histories from 25 years of the World Wide Web (pp. 175–190). New York: Peter Lang.

Brügger, N., Schafer, V., Geeraert, F., Isbergue, N., & Chambers, S. (2020). Exploring the 20-year evolution of a research community: Web-archives as essential sources for historical research. Bladen voor documentatie/Cahiers de la documentation, 2, 62-72. http://hdl.handle.net/10993/43903

Vlassenroot, E., Chambers S., Di Pretoro E., Geeraert, F., Haesendonck G., Michel A., & Mechant, P. (April 1, 2019). Web Archives as a Data Resource for Digital Scholars. International Journal of Digital Humanities 1, no. 1: 85–111. https://doi.org/10.1007/s42803-019-00007-7

Venlet, J., Farrell, K. S., Kim, T., O'Dell, A. J., & Dooley, J. (2018). Descriptive Metadata for Web Archiving: Literature Review of User Needs. https://doi.org/10.25333/C33P7Z

SESSION#04: Delivery & Access

Renascer Project Brings Back Old Websites at Arquivo.pt

Ricardo Basílio, Daniel Gomes
Arquivo.pt, Portugal

Organisations keep domains that referenced websites which are no longer being used, to prevent them from being bought or because they were just forgotten. In this presentation we share how Arquivo.pt developed the Renascer project to recover the content of historical websites and make it widely accessible once again. The aim of the Renascer (Reborn) project is to bring back historical websites whose content is no longer available online but whose domain continues to be held by their owners.

“Forgotten” domains can cause cybersecurity problems. For example, in May 2023, the domain hmsportugal.pt of the Harvard Medical School-Portugal project referenced just one default web page hosted on an active server but the domain continued to be owned by its owner. In this situation, the original content of the website was inaccessible despite that the domain continued to be owned by the author of the website. Furthermore, since the domain was still pointing to an active web server, cybersecurity incidents could occur if this server was not being properly maintained. However, the website hmsportugal.pt could be safely reference its historical content because it was preserved by Arquivo.pt (https://arquivo.pt/wayback/20131105224548/http://hmsportugal.pt/).

How are websites Reborn? The domain owner just needs to redirect it to Arquivo.pt, through the Memorial service (https://arquivo.pt/memorial). For example, the mctes.pt domain owned by the Portuguese Government resumed to reference its content which was preserved by Arquivo.pt. Thus, making this website content widely accessible online once again. Notice that all the links from other websites to the web pages of the reborn website are also reactivated.

During 2023, we demonstrated the potential of the Renascer project by applying it internally to the historical websites of our organisation FCT (Foundation for Science and Technology). The Renascer project identified 11 active domains managed by FCT which were not referencing valid content, and then brought their historical content back to life. From this initial project, we are now promoting that organisations should recover the content of their historical websites. We have received positive feedback. Reborning historical websites is very interesting for organisations to commemorate anniversaries or easily keeping track of past activities. On the other hand, it is a simple yet powerful service that demonstrates the “real-world” utility of web archives in modern societies.

Preserving the Uncrawlable: Serving the Server

Andrew McDonnell
University of Kentucky, United States of America

Many of the challenges inherent in archiving dynamic web content revolve around the capture and playback of ever-evolving social media sites. However, older dynamic web content that has ceased to evolve continues to elude the tools and conventions most widely available to web archivists. This presentation will share work conducted to preserve an online digital humanities project that defied preservation via Archive-It and other crawler-based web archiving tools. The presentation will, moreover, offer an exploration of alternative options for web archivists struggling to preserve sites dependent on server-side processes for their essential functionality.

Suda On Line is a collaborative digital translation of a 10th-century encyclopedia. The interactive project was initially developed in 1998 and constructed as a website using CGI scripts, a bespoke database suite, and server rules that, taken as a whole, required user input via search fields and clicks to dynamically generate pages. With over 30,000 entries compiled by more than 200 scholars from 20 countries across five continents, the last entry in the Suda was translated on the site 16 years after it launched. As the primary collaborators responsible for the site retired from academic work, they found an entity willing to temporarily host their server, but that arrangement is winding down. It has no apparent long-term home willing to maintain and update the site in perpetuity. As a result, one of the project’s original authors, with the permission of his collaborators, asked their university archives to archive this collective work of scholarship, and the archives accepted the task.

The university’s archives discovered that tools such as Archive-It and WebRecorder were capable of mimicking the look of the Suda interface, but none of its core functionality. The site's dependence on users' interactions with a database via server processes meant that even though 70,000+ files were captured during crawls, the end result could not playback as intended. As a secondary means of preserving the work, the archives were granted a copy of the site’s full Ubuntu server image, packaged as two virtual machines within an OVF file. This presentation will unpack the various paths and cul-de-sacs explored in the process of preserving and providing access to the Suda On Line, including tools such as ReproZip-Web and Oracle VM VirtualBox. It will also reflect on the feasibility and merit of providing offline access to tools, research, and scholarship that in their active lives of creation and purpose were so fundamentally online. In the broader context, is digital preservation of the offline surrogate of this site (and others like it) fundamentally transforming it from tool to artifact in a manner that is irreversible?

Lost and Found in Cyberspace: Reconstructing MultiTorg

Jon Carlstedt Tønnessen
National Library of Norway

In May 1993, a group of computer scientists published the very first website on the Norwegian top-level domain: MultiTorg. Probably among the first hundred websites in the world, MultiTorg pioneered web publishing by offering instant news from the National News Agency. Visitors could also view recent satellite images from outer space, short clips from upcoming video games, and download Paul McCartney’s latest hit “Hope of Deliverance”. In its time, MultiTorg was regarded as ground-breaking, creating “a door from people’s homes into cyberspace."

Since MultiTorg was published at the very dawn of the web, long before anyone thought of archiving the web, it escaped archiving in a standardised way. However, one of the creators made a backup of the server in September 1993. This was recently acquired by the Norwegian Web Archive, and provides a valuable glimpse into the birth of the web. The crucial question is how these remains of the early web can be preserved and examined in a reliable manner.

The presentation will unfold in three parts:

Taking inspiration from the innovative approach of Dutch computer scientists to perform web archaeology, I will present a systematic method to excavate, register, refit, and reconstruct the remains of MultiTorg, based on the server backup.

Delving into MultiTorg’s content, I will present valuable insights into web media in its infancy. We will explore the visions of the early pioneers, the revolutionary simplicity of HTML markup, and understand how MultiTorg tried to conceptualise cyberspace as a “marketplace”.

Finally, I will address some unsolved questions, related to the preservability of old web content that has escaped harvesting. This includes the need for international cooperation in rescuing remains of the lost, pre-archived web that may still be found and preserved.

Towards Multi-Layered Access with Automatic Classification

Jon Carlstedt Tønnessen, Thomas Langvann
National Library of Norway

Providing access remains a challenge for most web archives. In Norway, legislative preparations have suggested a multi-layered model for access, related to the material’s copyright status and the degree of personal data. However, information about these matters is rarely declared in the content we harvest, and never in a uniform or standardised way.

This paper will present an effort to automatically classify web archive records into different categories of access. The objective is to make as much of the material available to as many as possible, satisfying the needs for documentation and research, within the limits of copyright and data protection laws.

First, the different categories for access will be presented briefly:

• Governmental sites (open access to anyone online)
• Publications with a responsible editor-in-chief (accessible for anyone in University Libraries)
• Content without a responsible editor-in-chief, or with personal data not regarded as widely known (accessible for researchers with a relevant purpose)

Second, we will share the process of establishing computational acceptance criteria for the different categories and a confidence score, necessary to perform automatic classification. And third, we will review our experiences with different approaches to automatic classification:

• Linking data with external registers and authorities
• Rule-based classification, based on regular expressions and link targets
• Machine-learning, using labelled datasets with certain extracted features to train a model, and then test the model’s ability to predict the category

The presentation will focus on the advantages and disadvantages with the different approaches. Furthermore, we will touch upon dilemmas related to precision level (domain, page or resource), maintenance over time, tracking domain ownership history, and classifying older-generation web content.

SESSION#05: Collaborations

LGBT+ and Religion: Queering Web Archive Research

Jesper Verhoef
Erasmus University Rotterdam, Netherlands

It is often believed that religion is inimical to LGBT+ people in general (e.g. Wilcox, 2006). However, some scholars have rightly questioned the alleged disconnect between religion and non-normative gender and sexuality (e.g. Klinken, 2019; O’Brien, 2014). In fact, religion plays a critical role in the lives of many queer individuals and ‘queer-positive religious movements and spaces have mushroomed globally during the last few decades’ (Jansen, 2022, p. 5; cf. Talvacchia et al., 2014).

Websites are a prime example of these (safe) spaces. However, extant research fails to do these justice. In my talk, I will show how web archives can be used to study queer religious web sites. I will share results of the project Mapping the Dutch queer web sphere that I recently conducted at the National Library (KB) of the Netherlands, in conjunction with KB web archivists and a software engineer. As part of its special LGBT+ web collection, the KB has annually harvested sixteen websites catered to Christian queers and a couple geared towards LGBT+ Muslims. By means of hyperlink analyses, I analyzed the networks that these religious queer websites formed, and whether these changed over time (2009-2022, but because of available data, the focus will be recent years).

I will demonstrate that there was a distinct queer religious web sphere, where religious websites predominantly referred to one another. I will interpret these findings – which I will also render as insightful Gephi visualizations – and briefly discuss intriguing findings. For instance, the relation between catholic and protestant websites was strong, which suggests that the ‘common cause’ of being both Christian and LGBT+ trumps intra-religious differences. I will also go into inter-religious differences, i.e. between the links of queer Muslim and Chistian websites.

Crucially, this talk should have an impact on all conference attendees, because it does not merely highlight – in a hands-on fashion – how collaboration between researchers and web archivists can further our understanding of key societal and historical questions. Moreover, it discusses the workflows we developed to create and analyze datasets. These should be conducive to future researchers, thus finally leading to more researcher engagement with the invaluable but notoriously underused KB web archive - or any web archive, really.

Enter The Trading Zone: When Web Archivists And Researchers Meet To Explore Transnational Events In Archived Web Collections

Susan Aasman¹, Anat Ben-David², Niels Brügger^3
1University of Groningen, Netherlands; ²Open University of Israel; ³Aarhus University, Denmark

The aim of this presentation is to discuss the ways archived web collections can be suitable for transnational studies of past and present web communication. By sharing lessons learned from the Web Archive Studies (WARCnet) Network it will become clear what the opportunities and constraints are for both web archivists and researchers.

From 2020 until 2023, WARCnet has been an active group comprising web archiving practitioners and researchers, dedicated to researching web domains and events. The primary goal was to facilitate high-quality research about the history of both national and transnational web domains, as well as to open up ways for studying transnational events across web domains. During these years, the network actively encouraged interdisciplinary research among researchers and engaged with web archivists and IT developers as crucial partners in this emerging cultural heritage field.

From this community grew a tremendously rich output of papers, as well as the forthcoming edited volume The Routledge companion to transnational web archive studies. The book covers research by scholars and web archivists about the challenges of analysing entire national web domains from a transnational perspective; reflections on the opportunities of archiving and researching transnational events such as the COVID-19 pandemic the challenges of using digital methods in web archive studies; and assessments of the politics of web archives as collections.

During the presentation, we will share the main lessons learned from this research network and take our book’s output as a starting point for discussing the added value of close collaborations between scholars, archivists, IT developers, and data scientists in this field. We have come to appreciate both the challenges and opportunities to develop what has been called a ‘trading zone’(Collins et.al 2007) with interdisciplinary interactions that enable participants to have meaningful collaborations.

Collins, H, Evans, R., Gorman, M., Trading zones and interactional expertise, Studies in History and Philosophy of Science Part A, Volume 38, Issue 4, 2007.

Web Archiving, Open Access, & Multi-Custodialism

Monica Westin¹, Jefferson Bailey^2
1Internet Archive, United Kingdom, ²Internet Archive, United States of America

Starting in 2018, the Internet Archive pursued a large-scale project to build a complete collection of open-access scholarly outputs that are published on the web, the IA Scholar project (scholar.archive.org). Additionally, the project has worked to improve the discoverability of scholarly works archived through data processing and AI/ML work, partnership development, technical integrations, and related products that make this material available in myriad ways.

While the IA scholar program originated to solve the challenges of a lack of preservation infrastructure for many scholarly publications and related outputs (data, code, etc), especially from under resourced, “long-tail,” publishers, the project also has illuminated the benefits of multi-custodialism, i.e. that web archives have affordances that enable their preservation and accessibility at a variety of complementary stewardship organizations. The idea of “more copies in more places” is a core tenet of digital preservation; however, in practice this effort usually implies a geo-distributed replication of content across multiple technical locations administered by one institution or through cooperative agreements sharing technical infrastructure, and not necessarily long-term content stewardship by multiple independent, but collaborating, organizations.

This talk will describe how the web archiving work of the IA Scholar project, as well as related efforts on web data sharing at Internet Archive, have advanced a multi-custodial approach to ensuring both preservation and access to essential scholarly knowledge and how these archives can be shared and stewarded by a diverse set of organizations. Initiatives that will be discussed in this talk:

The IA Scholar Selfless Deposit Service: Using IAS and its underlying catalog, the IAS team has developed tools and techniques to automatically determine how many articles were published by authors affiliated with an institution, then compare these lists with an institution’s holdings in its own institutional repository (IR). Content in IAS that is not in an institution’s IR can then be shared with the institution, helping fulfill many of their local mandates on data stewardship, open access, and digital preservation.

Preservation Equity Program: The Internet Archive's Archiving & Data Services group is has run a variety of sponsored services, especially around web archiving, that ensure the preservation of at-risk content in multiple repositories to help ensure equal access to digital preservation tools for everyone, with a focus on empowering mission-driven organizations. Similarly, in collaboration with CLOCKSS, DOAJ, Keepers Registry and PKP, Internet Archive is a partner in Project JASPER, an initiative to preserve open access journals in multiple preservation systems in response to research that shows that smaller online journals are highly ephemeral and lack long-term digital preservation resources.

Web Data Sharing: The same IA team has been working with a number of national libraries and other organizations to provide extractions of early web archive data to partners to help complete their collections and ensure the preservation of this data in multiple repositories.

This presentation will provide an overview of these projects to illustrate and advocate for the larger idea of web archives being uniquely suited for a multi-custodial approach to preservation.

SESSION#06: Legal & Ethical

Intellectual Property & Privacy Concerns of Web Harvesting in the EU

Anastasia Nefeli Vidaki
Vrije Universiteit Brussel, Belgium

With data being confronted as the “oil of the 21st century” special attention on a global scale has been drawn to its more technologically efficient, fast and reliable accumulation methods. At the same time, new disruptive technologies such as, but not limited to Artificial Intelligence (AI) and machine learning have appeared and gradually dominated the data scene, assisting and facilitating even more data collection. Often based on them web harvesting is playing a leading role in obtaining digital data.

Web harvesting tools utilise software in many cases built upon AI so as to methodically browse the internet and extract the data carrying desired information. The process is rather easy to follow. A set of webpages is made available to the web harvester, which fetches further the other pages made accessible via this initial set. Through this procedure some parts of the webpage are stored and the content, possibly along with metadata in it, are downloaded and stored as well. The use of the data obtained afterwards can vary from archiving to data analytics and distribution to third-parties.

The paper focuses on the legal issues to which the aforementioned re-use of data gives birth. The most pertinent, at least for the European Union (EU) sphere, are considered the constraints imposed by the intellectual property and data protection legislation. On the one hand, the crawling of websites that include copyright-protected content and its extraction and reproduction without authorization violates EU copyright law and leads to the imposition of serious sanctions. However, the EU legislator has catered for offering some statutory exceptions, namely the one for private use, the one for temporary copies and the most recent one for Text and Data Mining (TDM) purposes. They will be explored theoretically and practically in order to figure out whether they provide a solution to the problem or if they demand deeper interpretation in light of the jurisprudence and the continuous technological development. The same observations can be made regarding the sui generis database right.

On the other hand, as long as the crawled content might consist of personal data, the matters of privacy and data protection come into play. With its strict regulatory framework EU has struggled to combat the unlawful processing of personal data. Therefore, data-intensive technologies like web scrapers should be designed and operated taking into account the principles prescribed in General Data Protection Regulation (GDPR). Nonetheless, compliance with data minimization, storage limitation, anonymization and lawfulness of the processing might be needed on behalf of data controller and processor along with complex organisational and technical measures. Questions arise concerning the set of rules for data access and scrutiny imposed by the Digital Services Act (DSA), which has not yet completely entered into force.

Finally, by bearing in mind the costs and burdens caused, a balancing between the obligations enshrined by the EU law and an aspiration for a more technological deterministic open data policy, powered by practices like web harvesting is suggested. There is need for transnational and interdisciplinary debate and cooperation.

DSM to the Rescue? Implications of the new EU Copyright Directive for Social Media Archiving: the Case of the Belgian Transposition and the Cultural Heritage Archives in Flanders

Ellen Van Keer, Rony Vissers
meemoo, Belgium

Main topic: legal context
Keywords: social media archiving, copyright, reproduction, text- and data mining

Social media archiving presents various legal obstacles for cultural heritage institutions (CHI’s). The aim of this contribution is to clarify how recent developments in EU copyright legislation can lower the barriers for social media archiving projects in the heritage sector.

Much content on social media is protected by copyright. Rightsholders hold exclusive rights over the use of their works and users need to gain their prior permission for using them. However, due to the large scale and wide diversity of social media content it is not realistic for heritage institutions to get permissions of all potentially involved rightsholders before engaging with social media archiving practices. Of course, this is not a completely new problem. Many items in heritage collections are protected by copyright, which generally lasts until 70 years after the author's death - this term is harmonised across the EU.

In order to keep a fair balance with competing fundamental rights such as the right of information and access to culture, copyright systems foresee a number of exceptions allowing certain uses in the public interest without the burden of prior authorization. CHI’s fulfil public tasks and are an important category of beneficiaries. While copyright remains a national competence, exceptions have been harmonised at European level. A first milestone in this development was the so-called InfoSoc-directive from 2001 (1), which included a closed list of 20 facultative exceptions (Art. 5 InfoSoc). A significant update came with the so-called DSM-directive in 2019 (2) which has been transposed into (most) national law systems over the last few years. In particular two provisions in the DSM are relevant here. First, an exception for preservation of cultural heritage has become mandatory (Art. 6 DSM). It provides a legal solution for digitising and preserving digital cultural content. Secondly, a new exception for text and data mining has been introduced (Art. 3 DSM). This creates a legal framework for large scale digital data collection and the application of AI.

This contribution will discuss the relevant provisions in more detail and clarify how they apply to social media archiving practices and projects in CHI’s, both in view of capture and preservation as well as access and valorization of social media content. As a case in point we will be looking at the Belgian transposition and its implications for the cultural heritage archives in Flanders, but the legislative framework and archival questions addressed bear relevance to the broader international web archiving community.

(1) https://eur-lex.europa.eu/legal-content/NL/TXT/?uri=CELEX%3A32001L0029

(2) https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32019L0790&qid=1695379375092

Digital Legal Deposit beyond the web

Vladimir Tybin
National Library of France

Under the law on copyright and related rights in the information society passed in France in 2006, digital legal deposit was introduced at the Bibliothèque nationale de France along with web legal deposit. At the time, BnF was responsible for collecting all "signs, signals, writings, images, sounds and messages communicated to the public by electronic means in France". In reality, the library had begun archiving the web and building up collections of French websites long before, since our web archive collections have a historical depth dating back to 1996 and to date represent more than 48 billion URL for 2 petabytes of data. In this way, the heritage mission of collecting everything that is disseminated on the French web in order to build up a national digital memory has gradually developed and been strengthened to the point where it is now an essential component of BnF's historical legal deposit mission.

However, it soon became apparent that many digital objects distributed electronically were escaping the automatic harvesting carried out by our robots for technical reasons or simply because of the commercial barriers behind which they were hiding. This is the case for digital books and scores on the market; for digital periodicals, journals, magazines and newspapers; for digital maps and plans; for digital photographs distributed by agencies and authors and videos distributed on streaming and VOD platforms; for applications, software and egames on the market, but also for all the production of born-digital audio documents, music production distributed on platforms. A solution had to be found to guarantee continuity from the physical to the digital world, and to avoid any disruption to BnF's legal deposit collections and heritage gaps.

We therefore decided to set up a new system for collecting all born digital documents in the same way as the historical legal deposit system, under which publishers, producers, authors and distributors would deposit their digital files along with their metadata to guarantee long-term preservation, record in our general catalogue and access for consultation in the reading rooms. From digital books to digital sound, not forgetting all other types of digital document, all these channels for entry, cataloguing, preservation and access are gradually being put in place, and are part of a major strategic challenge for the Bibliothèque nationale de France.

The aim of this presentation is to describe the various projects that have gone into setting up these systems: changes to the legal framework, technical development of specific tools and workflows, organisational work required to integrate these new processes and the implementation of a scientific policy for legal deposit.

SESSION#07: Communities I

Wikipedia Articles Related To Switzerland For Eternity

Barbara Signori
Swiss National Library

To ensure that the Wikipedia source of knowledge is securely preserved and accessible for future generations, the Swiss National Library – a federal memory institution with the legal mandate to collect, preserve and disseminate information with a bearing on Switzerland – is building a digital collection in which it compiles Wikipedia content related to Switzerland, archives it permanently, and makes it freely available online. The Wikipedia collection is listed in the library catalog Helveticat and can be searched and consulted in e-Helvetica Access, the entry portal to the Swiss National Library digital collections (containing also books, dissertations, journals, standards and websites). Wikimedia CH – the Swiss Chapter of the Wikimedia Foundation and promoting free knowledge in Switzerland – welcomes and supports this new digital collection. The two organizations have worked closely together to realize it.

The collection was officially initialised in June 2023. The first overall crawl includes about 130,000 identified articles. Identifying the relevant articles with a reference to Switzerland is definitely the biggest challenge: What is considered to be related to Switzerland and what is not? Where to draw the line? We analysed whether AI could help us with this – however, the initial effort would have been very large and the quality of the result uncertain. That's why we opted for a simpler solution. The articles are identified with the help of the query tool PET-scan and the categories “Switzerland” including their subcategories. The URLs (the contents) are obtained via script, copied, downloaded, fed into the long-term archive system and processed. The individual articles are “frozen” and remain unchanged in the archive. The live Wikipedia article, on the other hand, can evolve at any time thanks to the collaboration of volunteer contributors. New and modified articles are planned to be added and archived in a regular cycle (at most once a year). Content is collected in the four national languages, i.e. from Wikipedia in German, French, Italian and Romansh. The collection of content in other languages is not excluded in the long term. However, the representative compilation does not claim to be exhaustive. In addition to the text, we include all images, graphics, audios, videos contained in the Wikipedia articles, the list of contributors and the applicable license terms, etc., including their metadata. We also have internal organizational procedures and measures in place to ensure that reports of potentially infringing content can be processed and removed quickly if necessary.

The contribution tells about how we, together with representatives from the Wikipedia community, have built a new kind of web collection. The collection is not integrated into Web Archive Switzerland – it is an independent web collection. The first one that we are allowed to make freely available. Conference attendees can benefit from our experience should they plan similar projects.

A Conceptual Model of Decentralized Storage for Community-Based Archives

Zakiya Collier, Jon Voss, Bergis Jules
Shift Collective, United States of America

Community-based archives hold some of the most valuable materials documenting the lives of marginalized people and they mostly exist independently of other traditional academic or government-run cultural heritage institutions. But while these archives continue to collect and preserve these histories, many of them face difficulties growing their operations, keeping their doors open, and enhancing their programming and collections activities because of a lack of funding opportunities and other ongoing resources common in larger institutions.

Beginning in 2023, the presenters began a three-year collaborative research and development project to develop use cases for a decentralized storage solution, and an ethical framework for engaging in this quickly developing technology.

With an eight-person research faculty and technical advisors, the first year of the project was dedicated to exploring the ethical, cultural, and technical needs of small cultural memory organizations that steward the most important historical collections for diverse communities around the world—including but not limited to precarious and hyperlocalized web archived content. In the first year, we focused on exploring affordable and sustainable digital storage and how a decentralized storage network might address particular stated needs for the tens of thousands of these organizations.

In the second year, the team is focused on designing a conceptual model for community-centered, non-extractive, affordable, and accessible long-term storage, using the Historypin platform as the front-end interface. The model suggests ways for larger, better-resourced cultural memory institutions to provide low to no-cost digital storage solutions for small organizations without extracting or removing digital assets, transferring ownership, dictating access terms, or even requiring access to community-owned collections at all.

This presentation will share our research to date and highlight the broader implications for the field.

Recommendations For Content Advisories In The UK Web Archive

Nicola Bingham
British Library, United Kingdom

The UK Web Archive plays a vital role in preserving digital content for societal benefit. However, it faces the challenge of addressing distressing and harmful material encountered by users. The need for content warnings was initially raised in March 2020, leading to ongoing discussions and assessments.

This conference paper outlines the strategic approach, ratified in September 2023, to address users' encounters with harmful or distressing content within the UK Web Archive. The ‘Content Advisory’ project’s primary goals were to mitigate the risks associated with offensive and controversial material in the archive while enhancing the overall user experience and maintaining appropriate content presentation. Rather than censorship, the focus is on contextualization.

The Content Advisory Group's recommendations focus on the use of content warnings and advisories, understanding associated risks, reviewing best practices from other web archives, and engaging with related projects.

The conference presentation will focus on the following areas:

1) Access Routes and Content Advisory Levels.

The UK Web Archive offers diverse access routes, making it challenging to ensure uniform content warnings. The majority of content comes from automated domain crawls, making precise warnings difficult. The presentation will outline the group’s exploration of various advisory placement options, including MARC records, Terms & Conditions pages, the UK Wayback Banner, calendar views, individual website advisories, and collection-level warnings.

2) Recommendations: Next Steps

Given limited resources, the Content Advisory paper recommended a comprehensive and continuous review of collection descriptions, the creation of collection scoping documents, a focus on Equality, Diversity, and Inclusion (EDI) goals, improved user education, and the introduction of a high-level content warning on the home page. It also suggested monitoring automated language-checking developments. This conference presentation will report on the results of a pilot project to review and revise a smaller number of our collection descriptions and will outline how this was implemented.

3) Related Groups and Projects

The Content Advisory Group drew insights from other relevant projects and institutions, emphasizing collaboration and alignment with institutional policies. The conference presentation will outline the work of related groups, projects and roles at the [host institution] including the Content Guidance Steering Group, Race Equality Action Plan, Collections Access Review and Approval Group (CARAG), new Black Studies Lead Curator, Metadata Lead for Equity and Inclusion, and the Legal Deposit User Forum.

Contribution and impact of paper:

This conference paper outlines the strategy to handle harmful content in the UK Web Archive. It presents practical recommendations and highlights the importance of collaboration with related projects and groups. The approach prioritizes user experience and contextualization, providing a valuable framework for managing challenging content in digital archives. The paper's impact lies in its potential to influence policy and practice within the web archive community, ensuring a more inclusive and user-friendly web archive experience.

Archiving The Black Web: Research & Access In Context - An Update

Zakiya Collier¹, Bergis Jules², Makiba Foster^3
1Shift Collective, United States of America; ²College of Wooster, United States of America; ³Archiving the Black Web, United States of America

Archiving the Black Web (ATBW) WARC School is a web archiving training program for memory workers affiliated with Black collecting community archives, public libraries, and HBCUs (Historically Black Colleges and Universities). The WARC School is set to launch in fall 2024 with the intention of educating and training current LIS professionals and community-based memory workers to not only become builders but also future users of web archives that focus on Black culture. Placing our training program within the constellation of efforts to understand what exactly is “The Black Web,” we, as web archivists and memory workers, are collaborating with humanists and social scientists in researching and mapping The Black Web from experience and theory to web archive practice. In this talk, ATBW will share information about our strategy to develop web archiving training curriculum within the context of building web archive collections that center Black digital content. In our discussion of the research project, we will share how the scholarship contextualizes Black people and their experiences within the larger world wide web and thereby informs our work in training people to create the web archives related to Black history and culture.

SESSION#08: Tools

The BnL’s Migration From OpenWayback To A Hybrid PyWb-SolrWayback Engine Powered By S3 Storage

László Tóth
National Library of Luxembourg

During the year 2023, the National Library of Luxembourg has undertaken the task of migrating its existing web archives, totaling around 300 TB compressed WARC files served by OpenWayback using a static CDX index, to a high-performance hybrid infrastructure consisting of PyWb and SolrWayback using an OutbackCDX index server and state-or-the-art S3 object storage. The goal of this migration was threefold: to modernize the BnL’s offer to users in terms of accessibility, search speed and overall end-user experience, to use the latest web archiving tools and workflows available to date, and to provide a highly efficient and responsive storage solution for our web archives. Thus, we improve the three main pillars of our web archives: user experience, software and hardware. Our final solution sits atop 4 high-performance servers, two of which were initially used for indexing our collections. These machines, having more than 2.5 TB RAM and 192 cores in total, are geographically distributed and host the Solr cluster, SolrWayback, PyWb, OutbackCDX applications and are connected to our governmental S3 object storage network. In order to efficiently retrieve data from this storage, we developed custom modules for SolrWayback and PyWb that are able to stream WARC records directly from an S3 storage starting at any given offset up to any given length, without any additional I/O delay related to stream skipping. Thus, playback is as fast as it would be if the archives were read from local storage. To conclude, these features, though not available by default within the aforementioned applications, have been developed in an open-source spirit and are made available freely online for anyone to use.

WARC-GPT: An Open-Source Tool for Exploring Web Archives Using AI

Matteo Cargnelutti, Kristi Mukk, Clare Stanton
Library Innovation Lab, United States of America

This year the team at the Library Innovation Lab (home of Perma.cc) has been exploring how artificial intelligence changes our relationship to knowledge. Partially inspired by colleagues at last year’s IIPC conference, one of our initial questions was: “Can the techniques used to ground and augment the responses provided by Large Language Models be used to help explore web archive collections?”

That question led us to develop and release WARC-GPT: an experimental open-source Retrieval Augmented Generation tool for exploring collections of WARC files using AI.

WARC-GPT functions as a highly-customizable boilerplate the web archiving community can use to explore the intersection between web archiving and AI. Specifically, WARC-GPT is a RAG pipeline, which allows for the creation of a knowledge base out of a set of WARC files, which is later used to help answer questions asked to a Large Language Model (LLM) of the user’s choosing.

What would it mean for your team’s process if it could interact with a chatbot that had insight into what you’ve captured from the web? Would a chatbot with knowledge of your collection be of use for description work? While still an experimental tool, WARC-GPT is a step towards understanding questions like that. Our team will share our experience so far testing things out, decisions we've made around tools, and share how other organizations can do that same.

The Perma team believes the expansion of our project beyond the technology and service we’ve offered for many years is of interest to the IIPC community. All of our work is still rooted in our original service, focusing on authenticity, fidelity, and provenance - but built to be more expansive.

R You Validating These WARCs? Automating Our Validation- And Policy-Checking Processes With R

Lotte Wijsman, Jacob Takema
National Archives of the Netherlands

As the National Archives of the Netherlands we receive web archives from one or more producers instead of harvesting these ourselves. Subsequently, the quality of the web archives can differ. To be able to ensure a consistent quality and the long-term preservation of the web archives, we must ask ourselves if the archive is complete, technically sound, and conforms to our guidelines. Since 2021, national government agencies need to comply to the guideline on archiving governmental websites (2018) when harvesting websites. Amongst other requirements, the web archives should be daily harvested, conform to the ISO-28500 standard, contain full- and incremental harvest, and have a maximum size of 1GB. To ensure conformance to the WARC standard, we validate the WARC files with e.g. JHOVE.

Previously, we have presented our work on validation and the web archiving guideline at the WAC. In 2023, we had a poster on WARC validation that also included our future ambitions, because there is still much to improve. The output of WARC validation tools is not always easy to work with, especially when working with a lot of files, and the tools don’t always check everything we want checked (such as the size of the WARC). Furthermore, we considered conformance checking to be the next step beyond validation. This is why we went searching for a way to not only find something that fits our every need concerning validation and conformance checking, but also to automate these processes.

To accomplish this work, we have started to use R, a programming language suited for data analysis. Using R, we have worked on building an automated conformance checker, which not only validates mandatory properties of the WARC standard, but also optional yet (for us) important properties (e.g. payload digests and file size), and important aspects for long-term preservation (e.g. embedded file formats). Furthermore, we have implemented an automated conformance check to see if web archives conform to our guideline for archiving Dutch governmental websites (e.g. harvest frequency, full- and incremental harvests).

In our presentation, we will first provide the attendees with some background information on our previous work on the web archiving guideline and WARC-validation. Subsequently, we will share how we came to automated conformance checking and we will give a demo using R to show our prototype conformance checker.

Machine-Assisted Quality Assurance (QA) in Browsertrix Cloud

Tessa Walsh, Ilya Kreymer
Webrecorder, Canada

Manual and software-assisted workflows for quality assurance (QA) of crawls are an important and underdeveloped area of work for web archives (Reyes Ayala, 2014; Taylor, 2016). Presentations at previous Web Archiving Conferences such as last year’s talks by The National Archives (UK) and Library of Congress have focused on institutions’ internal practices and tools to facilitate understanding and assuring the quality of captures of websites and social media (Feissali & Bickford, 2023; Bicho, et al, 2023). A similar conversation was facilitated by the Digital Preservation Coalition in their 2023 online event “Web Archiving: How to Achieve Effective Quality Assurance.” These presentations and discussions show that there is a great deal of interest in and perceived need for tools to assist with performing quality assurance of crawls in the web archiving community.

This talk will discuss machine-assisted quality assurance features built in the open source Browsertrix Cloud project that have the potential to help a wide range of web archivists across different institutions gain an understanding of the quality and content of their crawls. We will discuss the goals of automated QA as an iterative process, first, to help users understand which pages may require user intervention, and then, how those might be fixed automatically. The talk will outline assisted QA features as they exist in Browsertrix Cloud at the time of the presentation, such as indicators of capture completeness, whether failed page captures resulted from issues with the website or crawler, and how/if they could be automatically fixed. The talk will provide examples of the types of issues in crawling that may be discovered and how they are surfaced to the user for possible intervention, discuss lessons learned in collecting user stories for and implementing the QA features, and point to possible next steps for further improvement.

As the presentation will discuss the assistive possibilities of software in aiding traditionally manual processes in web archiving, lessons learned are likely to apply widely to all such assistive uses of technology, including other conference themes such as the use of artificial intelligence and machine learning technologies in web archives.

References:

Bicho, Grace; Lyon, Meghan & Lehman, Amanda. The Human in the Machine: Sustaining a Quality Assurance Lifecycle at the Library of Congress, presentation, May 11, 2023; (https://digital.library.unt.edu/ark:/67531/metadc2143888/: accessed September 21, 2023), University of North Texas Libraries, UNT Digital Library, https://digital.library.unt.edu; crediting International Internet Preservation Consortium.

Feissali, Kourosh & Bickford, Jake. Open Auto QA at UK Government Web Archive, presentation, May 11, 2023; (https://digital.library.unt.edu/ark:/67531/metadc2143893/: accessed September 21, 2023), University of North Texas Libraries, UNT Digital Library, https://digital.library.unt.edu; crediting International Internet Preservation Consortium.

Reyes Ayala, Brenda; Phillips, Mark Edward & Ko, Lauren. Current Quality Assurance Practices in Web Archiving, paper, August 19, 2014; (https://digital.library.unt.edu/ark:/67531/metadc333026/: accessed September 21, 2023), University of North Texas Libraries, UNT Digital Library, https://digital.library.unt.edu; .

Taylor, Nicholas. Rethinking Web Archiving Quality Assurance for Impact, Scalability, and Sustainability, presentation, August 4, 2016; https://nullhandle.org/pdf/2016-08-04_rethinking_web_archiving_quality_assurance.pdf: accessed September 21, 2023).

SESSION#09: Communities II

Building and Promoting A National Web Archive Through Regional Cooperation: A French Experience

Anaïs Crinière-Boizet¹, Elise Girold^2
1National Library of France; ²National University Library of Strasbourg, France

For the past 20 years, as part of the cooperation with associated regional centres, the Bibliothèque nationale de France (BnF) has been able to call on a network of external contributors throughout France to complement national collections with regional and local websites. This collaboration began with the 2004 regional elections and has been strengthened since then.

This network of 26 printing legal deposit libraries is called upon for every electoral collection with a local dimension (municipal, departmental, regional and legislative elections). These libraries also have the opportunity, thanks to an organisation of regional collections and a collaborative curation tool, to select websites related to the political, economic, cultural and social life of their region. They were able to offer local selections during the Covid-19 pandemic. This organisation allows them to participate in the creation of tomorrow's heritage as part of the shared challenge of preserving regional heritage online. In return, they can promote local content, related to their region, through « guided tours » into the BnF search application : « Archives de l’internet ».

In addition, since 2014, French law has authorised remote access to BnF's web archives in these partner libraries. This remote access is currently available in 21 of those and allows them to highlight web archives to the general public as well as to students and researchers during events such as the European Heritage Days or workshops and training courses.

This two voices presentation will include a feedback from one of those libraries, the Bibliothèque nationale et universitaire de Strasbourg (Bnu). The Bnu has been involved in the electoral crawls since 2007, is in charge of a regional collection for the region Alsace since 2013 and has developped many services to accompany the work of researchers on web archives.

Web Archiving for Preserving the Digital Memory of Brazil

Jonas Ferrigolo Melo¹, Moises Rockembach^2
1University of Porto, Portugal; ²University of Coimbra, Portugal

The evolution of the web presents challenges in preserving Brazil's internet history and protecting relevant digital content. This study underscores the importance of web archiving for preserving this history, enabling access to historical information, and preventing data loss. The "arquivo.ong.br" project, an essential initiative in this context, exemplifies the significance of these efforts.The growth of digital documents, including text, databases, images, audio, video, and websites, has been recognized since the early 2000s.

In 2010, the National Archives of Brazil initiated the AN Digital program to preserve and provide access to digital archival documents, emphasizing the importance of preserving digital content. In 2021, the National Archives Council (CONARQ) established the Technical Advisory Chamber (CTC) for the Preservation of Websites and Social Media, aiming to preserve dynamic digital documents. Additionally, legislation seeks to criminalize the misappropriation of digital assets from official websites. Early research by Dantas (2014) highlighted the absence of web page collections in Brazilian institutions, leading to the creation of the Buscas.br collection.

Projects like "arquivo.ong.br" are critical for preserving Brazil's digital memory, highlighting the ongoing need for their development and enhancement. Other recent projects like ARQWEB and Graúna also focus on preserving web content. The Arqweb project, initiated in 2022 by the Brazilian Institute of Information in Science and Technology (IBICT), aims to archive partner institution websites and government sites. Similarly, the Graúna Project, developed by the NUPEF Institute, focuses on preserving the memory of the internet by archiving websites related to human rights, the environment, culture, and health.

The "arquivo.ong.br" project, affiliated with the Federal University of Rio Grande do Sul, serves as a important resource for preserving web archives, including presidential election websites and COVID-related content. It has accumulated around 50GB of diverse digital data. The project uses tools like Heritrix for automated web data collection and Pywb for user-friendly access and exploration of preserved files. Pywb allows users to search, browse past website versions, and interactively explore the archive, making it an essential source for researchers, academics, and anyone interested in Brazil's web history.

Digital Diaspora: Preserving the Jewish Internet

Hana Cooper
National Library of Israel

Since its establishment in 1892, the National Library of Israel has been dedicated to the collection and preservation of the historical, cultural and intellectual story of Israel and the Jewish people. This mission has undergone various transformations over the past centuries in tandem with the changes that the Jewish people and the state of Israel have experienced. The advent of the internet as an information store has expanded our understanding of community, nationality and culture at large. In particular, the diasporic quality of the Jewish identity is reflected well in the wide-ranging and early use of online spaces to the benefit of individuals and communities.

Over the past decades, the internet has enabled Jews in Israel and the diaspora to preserve and share their cultural heritage, practices, and stories. It is a diverse and dynamic space where stories, memories, values and identities are shared and preserved across languages and borders. From personal blogs and social media accounts to digital archives and educational resources, the Jewish Internet showcases the richness and complexity of a heritage that spans centuries and continents. Since the early 2000s, The National Library of Israel has recognized that the preservation of contemporary Jewish identity hinges on the preservation of this digital heritage. Consequently, a large assortment of methods and tools have been implemented over the last decades—which we are still attempting to refine and restructure in order to meet the everchanging digital landscape.

Alongside our in-house efforts, we are actively working on ways in which we might be able to foster collaborative partnerships with other cultural institutions, academic researchers, and Jewish communal organizations. Some of these collaborations will engage in curatorial work (by region/language and by topic), while others will address legal and ethical concerns. In the IIPC conference we would like to present the progress made so far by the National Library of Israel, and raise a number of strategic, ethical and practical dilemmas that we face going forward.