{"id":1213,"date":"2018-11-05T01:54:19","date_gmt":"2018-11-05T01:54:19","guid":{"rendered":"http:\/\/netpreserve.org\/ga2018\/?post_type=conf_workshop&p=1213"},"modified":"2018-11-07T00:21:42","modified_gmt":"2018-11-07T00:21:42","slug":"expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive","status":"publish","type":"conf_workshop","link":"https:\/\/netpreserve.org\/ga2018\/workshops\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\/","title":{"rendered":"Expansion and exploration in 2018: processing the Library of Congress web archive 14:00-14:30"},"content":{"rendered":"

Abbie Grotke & Grace Thomas<\/strong>, Library of Congress<\/em><\/span><\/p>\n

The Library of Congress Web Archive selects, preserves, and provides access to archived web content selected by subject experts from across the Library, so that it will be available for researchers today and in the future. The Library\u2019s Digital Collecting Plan, produced in February 2017, calls for an expansion of the use of web archiving to acquire digital content. Our talk will focus on how the Library of Congress plans to accomplish this by expanding selective crawling practices and simultaneously scaling up technical infrastructure to better support program operations.<\/p>\n

Web archiving efforts at the Library of Congress have, up to now, focused on highly selective collecting of specific, thematic collections proposed and described by subject specialists. An expansion of web archiving will require both enlisting additional subject specialists to engage in web archive collection development, and for those already engaged to broaden their web archiving selection to additional themes and subjects. The Library is also currently tackling the backlog of web archives in need of descriptive records for presentation on the Library\u2019s website.<\/p>\n

Expanding web archiving at the Library will also require finding solutions to analyze, process, and manage huge quantities of content. With over 1.3 PB of content and a current growth rate of more than 300 TB per year, the sheer size of the archive has begun to present technical challenges, particularly with rendering content on the public Wayback Machine and delivering research ready data to scholars. In 2018, new cloud infrastructure was made available to the Web Archiving Team for processing the archive. With this new capability, the team is exploring a variety of projects, including experimenting with alternate index models, generating multiple types of derivative files to gauge research engagement with the web archives content, and running analyses over the entire archive to gain deeper understanding of the content held within the collections.<\/p>\n

In the coming months, the Library of Congress will ingest the web archive into the cloud and test new processes for managing the web archive at scale, and will be able to share stories of triumphs and challenges from this crucial transition with the greater web archiving community.<\/p>\n","protected":false},"featured_media":0,"template":"","conf_sessions":[68],"conf_streams":[],"conf_areas":[],"class_list":["post-1213","conf_workshop","type-conf_workshop","status-publish","hentry","conf_sessions-institutional-program-transitions-chair-nicholas-taylor"],"yoast_head":"\nExpansion and exploration in 2018: processing the Library of Congress web archive 14:00-14:30 -<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/netpreserve.org\/ga2018\/workshops\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Expansion and exploration in 2018: processing the Library of Congress web archive 14:00-14:30 -\" \/>\n<meta property=\"og:description\" content=\"Abbie Grotke & Grace Thomas, Library of Congress The Library of Congress Web Archive selects, preserves, and provides access to archived web content selected by subject experts from across the Library, so that it will be available for researchers today\" \/>\n<meta property=\"og:url\" content=\"https:\/\/netpreserve.org\/ga2018\/workshops\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\/\" \/>\n<meta property=\"article:modified_time\" content=\"2018-11-07T00:21:42+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@netpreserve\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/netpreserve.org\\\/ga2018\\\/workshops\\\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\\\/\",\"url\":\"https:\\\/\\\/netpreserve.org\\\/ga2018\\\/workshops\\\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\\\/\",\"name\":\"Expansion and exploration in 2018: processing the Library of Congress web archive 14:00-14:30 -\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/netpreserve.org\\\/ga2018\\\/#website\"},\"datePublished\":\"2018-11-05T01:54:19+00:00\",\"dateModified\":\"2018-11-07T00:21:42+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/netpreserve.org\\\/ga2018\\\/workshops\\\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\\\/#breadcrumb\"},\"inLanguage\":\"en\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/netpreserve.org\\\/ga2018\\\/workshops\\\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/netpreserve.org\\\/ga2018\\\/workshops\\\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/netpreserve.org\\\/ga2018\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Expansion and exploration in 2018: processing the Library of Congress web archive 14:00-14:30\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/netpreserve.org\\\/ga2018\\\/#website\",\"url\":\"https:\\\/\\\/netpreserve.org\\\/ga2018\\\/\",\"name\":\"\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/netpreserve.org\\\/ga2018\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Expansion and exploration in 2018: processing the Library of Congress web archive 14:00-14:30 -","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/netpreserve.org\/ga2018\/workshops\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\/","og_locale":"en_US","og_type":"article","og_title":"Expansion and exploration in 2018: processing the Library of Congress web archive 14:00-14:30 -","og_description":"Abbie Grotke & Grace Thomas, Library of Congress The Library of Congress Web Archive selects, preserves, and provides access to archived web content selected by subject experts from across the Library, so that it will be available for researchers today","og_url":"https:\/\/netpreserve.org\/ga2018\/workshops\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\/","article_modified_time":"2018-11-07T00:21:42+00:00","twitter_card":"summary_large_image","twitter_site":"@netpreserve","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/netpreserve.org\/ga2018\/workshops\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\/","url":"https:\/\/netpreserve.org\/ga2018\/workshops\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\/","name":"Expansion and exploration in 2018: processing the Library of Congress web archive 14:00-14:30 -","isPartOf":{"@id":"https:\/\/netpreserve.org\/ga2018\/#website"},"datePublished":"2018-11-05T01:54:19+00:00","dateModified":"2018-11-07T00:21:42+00:00","breadcrumb":{"@id":"https:\/\/netpreserve.org\/ga2018\/workshops\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\/#breadcrumb"},"inLanguage":"en","potentialAction":[{"@type":"ReadAction","target":["https:\/\/netpreserve.org\/ga2018\/workshops\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/netpreserve.org\/ga2018\/workshops\/expansion-and-exploration-in-2018-processing-the-library-of-congress-web-archive\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/netpreserve.org\/ga2018\/"},{"@type":"ListItem","position":2,"name":"Expansion and exploration in 2018: processing the Library of Congress web archive 14:00-14:30"}]},{"@type":"WebSite","@id":"https:\/\/netpreserve.org\/ga2018\/#website","url":"https:\/\/netpreserve.org\/ga2018\/","name":"","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/netpreserve.org\/ga2018\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en"}]}},"_links":{"self":[{"href":"https:\/\/netpreserve.org\/ga2018\/wp-json\/wp\/v2\/workshops\/1213","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/netpreserve.org\/ga2018\/wp-json\/wp\/v2\/workshops"}],"about":[{"href":"https:\/\/netpreserve.org\/ga2018\/wp-json\/wp\/v2\/types\/conf_workshop"}],"version-history":[{"count":0,"href":"https:\/\/netpreserve.org\/ga2018\/wp-json\/wp\/v2\/workshops\/1213\/revisions"}],"wp:attachment":[{"href":"https:\/\/netpreserve.org\/ga2018\/wp-json\/wp\/v2\/media?parent=1213"}],"wp:term":[{"taxonomy":"conf_sessions","embeddable":true,"href":"https:\/\/netpreserve.org\/ga2018\/wp-json\/wp\/v2\/conf_sessions?post=1213"},{"taxonomy":"conf_streams","embeddable":true,"href":"https:\/\/netpreserve.org\/ga2018\/wp-json\/wp\/v2\/conf_streams?post=1213"},{"taxonomy":"conf_areas","embeddable":true,"href":"https:\/\/netpreserve.org\/ga2018\/wp-json\/wp\/v2\/conf_areas?post=1213"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}