id	summary	reporter	owner	description	type	status	priority	milestone	component	resolution	keywords	cc	parents
802	sitemap.xml generation	user-A		"Hi, I'm partiticipating to the opensource distributed searchengine [http://yacy.net/ YACY]. Thus I like to encourage you, to offer webspiders an sitemap.xml file (and refer them in the robots.txt): https://en.wikipedia.org/wiki/Sitemaps

This allows the bot to just walk trough the flat list and he doesn't has to get the webstructure out of the parsed HTML pages. This avoids that the searchengine misses some content and speeds up the analysis process."	defect	closed	major		programming	wontfix	seo, sitemaps, search