2026-10-04T01:19:51Zhttps://skemman.is/oai/request

oai:skemman.is:1946/20712018-10-15T15:55:14Zcom_1946_2064com_1946_2057col_1946_2070

Háskóli Íslands Kristinn Sigurðsson 1977- Helgi Þorbergsson 1957- The World Wide Web contains an increasingly significant amount of the world’s knowledge and heritage. Since the Web is also in a constant state of change significant efforts are now underway to capture and preserve its contents. These efforts extend the traditional legal deposit laws that have been aimed at preserving printed material over the last centuries. The first three chapters outline the fundamental challenges for collecting the Web and present the software, Heritrix, which has been designed to perform this task. The first chapter focuses on the reasons and history behind this endeavour, with chapters two and three focusing on more technical aspects. The goal of this project was to develop a new way of collecting parts of the Web that are believed to change very rapidly and are considered of significant interest. The later chapters focus on defining such an incremental strategy, which we call an ‘adaptive revisting strategy’ and how it was implemented as a part of Heritrix. A part of this discussion is how to detect change in documents. Finally we discuss initial impressions of the new software and highlight areas that require further work or attention. As the goal of the project was primarily to establish the foundation for such incremental crawling and provide a simple and sturdy implementation, this section contains many thoughts on issues that could be improved on in the future. 2009-03-16T10:49:06Z 2009-03-16T10:49:06Z 2009-03-16T10:49:06Z 2005-05 http://hdl.handle.net/1946/2071 en Hugbúnaðargerð Heritrix Internet Skylduskil Tölvunarfræði Adaptive Revisiting with Heritrix Thesis Master's