Site Update Notification

The Site Update Notification project aims to deliver a mechanism for web masters to automatically publish information about changes to their website (changes in content and in presentation and layout) directly to those interested in keeping up-to-date with their publications. This could be Search Engines, Page Monitors, Bookmark Managers, Offline Browsers, Website Mirroring, Web Archives or others.

This is an alternative to the traditional crawling of sites that download pages and following hyperlinks to related pages (crawling the web) and distributes the burden from a centralised (albeit distributed) web spider and increases efficiency by reducing bandwidth requirements across the internet.

Read more...

Keywords

crawler, spider, web crawler scalability, optimization, change notification, site update notification

Project Goals

The Site Update Notification System seeks to ...

• address the problem with crawler scalability
• reduce the crawler bandwidth and index load
• reduce the time that a web site’s change takes to be available in a search engine’s index
• address the problem crawlers currently face in accessing the Deep Web
• provide an infrastructure for agent data aggregation and delivery to subscribers that reduces the bandwidth of individual web sites

Read more...

News

• Update notifications by E-mail
• Managing the workload of the sunaweb spider
• Share your watchlist with your friends, family and colleagues
• SnapShots bring thumbnails to watchlist URLs
• sunaweb crawler activity and status now available

Subscribe to RSS feed for updates.