I have no idea why feed publishers do not care to return Last-modified header so readers can use conditional GET and 304s instead of full thing over and over again in case when feed contents don’t change. I beleive they just don’t care about RSS traffic money they spend on these absolutely useless data transfers. Also they don’t care about speed of clients reader (which needs to reparse data every time without need for it).
Anyway, in addition to using conditional GET, fetcher now checks if content really changed (comparing hashed raw feed content to previous one) and skips parsing fase. Hopefully it’ll make it sagnificantly faster and will save us some CPU cycles.
I’m thinking about writing an article about popular blogs (e.g. Engadget, Joel on Software and some more) that don’t use conditional GET. Check out yellow pressish headline: “Bloggers are wasting investor’s money”.