The WARC file format: preparing next steps 16:30-17:30

Sara Aubry, Bibliothèque nationale de France (BnF)

For more than 20 years, memory storage organizations have been collecting and keeping track of World Wide Web material using web-scale tools such as web crawlers. At the same time, these same organizations archive large numbers of born-digital and digitized files. The WARC (Web ARChive) file format was defined to support these activities: it is a container format that permits one file simply and safely to carry a very large number of constituent data objects of unrestricted type for the purpose of storage, management, and exchange. Today, the WARC file format is extensively used within the web archiving community to support applications for harvesting web resources and accessing web archives in a variety of ways.

The WARC file format was initially released as an ISO international standard in May 2009 named 28500:2009 (also known as WARC 1.0). As with all ISO standards, the WARC standard is periodically reviewed to ensure that it continues to meet the changing needs that emerge from practice. The first revision, supported by an IIPC task force and the subcommittee in charge of technical interoperability within ISO information and documentation technical committee (ISO/TC46/SC4), was published in August 2017 as ISO 28500:2017 (also known as WARC 1.1). The next regular ISO vote to start another revision process is currently scheduled for 2020.

This discussion aims at gathering IIPC members interested in and working with the WARC format to inventory needs for either small or larger evolutions, share them within the group to identify common interests and start shaping the scope of the upcoming revision. Exchanges on IIPC Github and Slack  channels will be used to prepare and structure the discussion before the face-to-face meeting.