Atom feed thumbnails

[maiki interi]

Not entirely on topic with `#658 </issues/658>`_ or
`#659 </issues/659>`_, but related.

My expectation was for the atom feed to contain the thumbnail of
the image (or equivalent of other media types). This is important
to me, as I want to widgetize/aggregate the feed on other sites.

I found two places where feeds are generated,
``mediagoblin/user_pages/views.py`` and
``mediagoblin/listings/views.py``:

::

        for entry in cursor:
            feed.add(entry.get('title'),
                entry.get('description_html'),
                content_type='html',
                author=request.matchdict['user'],
                updated=entry.get('created'),
                url=entry.url_for_self(request.urlgen))

I am not sure what the thumbnail is, but I am betting it is a
**summary** or maybe **content** (see the
`werkzeug atom docs <http://werkzeug.pocoo.org/docs/contrib/atom/>`_).
I think this is important, even if we are moving to make it a full
ActivityStream, since the federation I have in mind is something
akin to a dashboard view of new photos posted on my friends'
sites.

So, can we get a thumbnail in there? =]

[Christopher Allan Webber]

Yes we certainly need a way to share this...

[Inconexo ø]

I have done a try. Not very happy because I had to:


1) Write html code inside python code. I don't feel that very
   orthodox.
2) Img comes inside html content. I don't know if there is a better
   way to include this, but I haven't found it inside the atom library
   documentation.

Please, tell me any comment that would improve my implementation.

git:
[https://gitorious.org/\ :sub:`inconexo/mediagoblin/inconexos-mediagoblin](https://gitorious.org/`\ inconexo/mediagoblin/inconexos-mediagoblin)
branch: 676\_feed\_thumbs

[Christopher Allan Webber]

Hey Inconexo, thanks for the work on this. :)

I think the best route would be to actually shift over the work to
making our own atom feed generator... I think we'll need to for
`#659 </issues/659>`_ anyway. And this would avoid such grossness
as we're extending that code.

Inconexo: if you're interested in pushing us toward a beautiful
federated future, a good way to start might be to help write an
atom feed generator using lxml :)
`http://lxml.de/tutorial.html <http://lxml.de/tutorial.html>`_

If you're interested in this, let me know! If you are interested
but have no experience with lxml, read that tutorial and ping me on
irc... you'll find it's pretty easy though, I think.

[Inconexo ø]

Well, finally, feed library is written. Please notice I am a
beginner with Python. Please comment any problem and I will fix
it.

I have kept two goals:


1) Same interface as Werkzeurg's. This way, changing one feed
   generator to other is changing the import.
2) Extensibility. I have made a generator class. If we wanted to
   generate RSS or other feed systems, we could write another
   generator, reusing a lot of code.

I have also added thumbnails capabilities. First, I added
media:thumbnails elements, but it doesn't seem to work with
firefox' and other's agregators. Any help with that is appreciated.
So, I used the same old trick: including the thumbnail inside the
content.

Anyway, you can take a look:

git:
[https://gitorious.org/\ :sub:`inconexo/mediagoblin/inconexos-mediagoblin](https://gitorious.org/`\ inconexo/mediagoblin/inconexos-mediagoblin)
branch: feed\_library

[Elrond]

Okay, this one diverged somewhat into "we need a feed library". And
one was created.
This is actually bug `#711 </issues/711>`_.
So made this one block'd by 711.

When 711 is done, we should revisit this one.

[Will Kahn-Greene]

The original url for this bug was ​http://bugs.foocorp.net/issues/676 .
Relations:
#310: related, #311: related, #358: blocked

[spaetz]

To post media in the atom feed, we'll need to add something like:

<link rel="enclosure" type="image/jpeg" href="​http://farm9.staticflickr.com/8343/8187137313_2e211c431f_b.jpg" />

to the atom feed. At least this is what the flickr atom feed does.

[spaetz]

I pushed a Work-In-Progess branch 328_media_in_atom_feed to my repo at ​git://gitorious.org/~spaetz/mediagoblin/spaetz-mediagoblin.git

It does not implement a new Atom feed generator, but at least it embeds the posted media as a
<link href="....jpg" rel="enclosure"/> which is what the flickr atom feed also does to convey the posted media.

I stumbled upon 2 roadblocks while implementing: mg_globals.public_store.file_url(media_entry) returns a relativ URL without HOST and we need a real fully external URL there. How can we achieve that?

Second, how can I determine the media type of the media so that browsers detect it correctly? Right now I hardcoded 'type': 'image/jpeg' which is obviously bad.

[Will Kahn-Greene]

Given that this is about thumbnails and not enclosing the actual media item, it's not corrrect to use <link href="..." rel="enclosure"/>. That's for enclosing media items like for podcasts, picturecasts, etc. Flickr encloses the entire media item.

To have thumbnails, you should use the MRSS module:

​http://www.feedforall.com/mediarss.htm

Firefox feed preview shows enclosures, but not thumbnails (weirdly, I helped write that).

What YouTube does is create an HTML thingy with the thumbnail in that and put that in the description. I suggest you do something like that. That's very common with podcast-like feeds.

[Christopher Allan Webber]

Will says MRSS should work with Atom.

[mrb]

FYI: ​https://gist.github.com/4398058 is my local patch to have an enclosure link.

[Christopher Allan Webber]

Heya,

I've been delayed in looking at mrb's patch... however, I've finally reviewed it!

Stylistically and code-cleanliness wise this patch seems very nice. I'm afraid it makes some assumptions that we can't make though: it's assuming that all different media types provide an "original" file (not all do!) and that this is what we want to serve.

Maybe a good solution instead is to take advantage of the MediaManager class and add a "enclosure" method. The method could take a request as an object and return a list(?) of files to give in the enclosure? (I remember that Atom feeds can have multiple enclosures for different sizes / file types...)

Thoughts?

[Ben Sturmfels]

I've revised mrb's patch to include both a thumbnail image and enclosures for all available media files.

Diff:
​https://gitlab.com/BenSturmfels/mediagoblin/compare/master...328-atom-thumbnails

Check out:
git fetch ​https://gitlab.com/BenSturmfels/mediagoblin.git 328-atom-thumbnails
git checkout FETCH_HEAD

Will recommended MRSS's thumbnail element, but I couldn't find any examples of this in an Atom feed in the wild. Found some in RSS, but not Atom. Did some tests and wasn't able to get a thumbnail to show up in Firefox or Liferea using something like <media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="..." />. Also, we can't add a thumbnail element without subclassing and modifying werkzeug.contrib.atom.AtomFeed. Flickr uses an <img> in the Atom content, so I'll go with that for now, unless anyone can suggest an live Atom feed using thumbnail and an Atom client that renders thumbnail.

I've built on mrb's patch, but added fully-qualified URLs for enclosures, which I believe is required. I've also included all media files as enclosures, instead of just hard-coding to original.

I also investigated some more creative solutions such as specifying an image *as* the content, as per ​http://www.ibm.com/developerworks/library/x-atom10/index.html#N10111. This both doesn't play well with the interface provided by werkzeug.contrib.atom.AtomFeed, and I also couldn't get a client to render it.

[ayleph]

Hey sturm, thanks for revising the original patch. I tested this on one of my instances, and there are a couple of things that I'm not sure about.

1. On Firefox, I see the main image as well as the enclosure you described. However, when I open the atom link in Chromium, it seems to display a plaintext xml page instead of an atom feed. Did you happen to try this? Maybe this is just a known issue with Chromium and atom feeds.

2. I don't know if it makes sense to show all of the available media in the enclosure. Based on what I've seen on other sites, just showing the thumbnail image and providing a link back to the hosted media is what I'd prefer. I don't see much value in including links to the various media files in the atom feed. How do you feel about this?

edited for spelling

[ayleph]

After browsing to various sites, it appears that Chromium is a lot stricter about feed formats than Firefox. For example, ​Engadget's RSS feed works fine in Firefox but not Chromium, while ​Slashdot's RSS feed works in both. This appears to have nothing to do with the current patch, so it seems to be out of scope for this ticket.

[Ben Sturmfels]

Replying to ayleph:

2. I don't know if it makes sense to show all of the available media in the enclosure. Based on what I've seen on other sites, just showing the thumbnail image and providing a link back to the hosted media is what I'd prefer. I don't see much value in including links to the various media files in the atom feed. How do you feel about this?

Yep, I agree. It gets a bit messy if you start including all the attachments. That might be one for a separate ticket with regard to supporting podcasts (and videocasts). Will amend to only include the thumbnail.

[Ben Sturmfels]

As discussed I've whittled this patch down to include only the thumbnail image, not adding the enclosure links to all available media types. I've updated the following branch and will also attach a patch.

Diff:
​​https://gitlab.com/BenSturmfels/mediagoblin/compare/master...328-atom-thumbnails

Check out:
git fetch ​​https://gitlab.com/BenSturmfels/mediagoblin.git 328-atom-thumbnails
git checkout FETCH_HEAD

[ayleph]

Thanks, I like the look of that much better. I ran into an issue when testing that I didn't catch before though. If a site has the blog plugin enabled, this patch causes the atom feed to crash mediagoblin because thumb isn't a member of file_urls. I know that the blog plugin isn't production ready, but I don't want to introduce a regression that affects people who are using it (myself included). Here's my proposed work around.

diff --git a/mediagoblin/listings/views.py b/mediagoblin/listings/views.py
index 0b8006d..181486c 100644
--- a/mediagoblin/listings/views.py
+++ b/mediagoblin/listings/views.py
@@ -105,8 +105,11 @@ def atom_feed(request):
     for entry in cursor:
         # Include a thumbnail image in content.
         file_urls = get_media_file_paths(entry.media_files, request.urlgen)
-        content = '<img src="{thumb}" alt='' /> {desc}'.format(
-            thumb=file_urls['thumb'], desc=entry.description_html)
+        if 'thumb' in file_urls:
+            content = '<img src="{thumb}" alt='' /> {desc}'.format(
+                thumb=file_urls['thumb'], desc=entry.description_html)
+        else:
+            content = entry.description_html
 
         feed.add(entry.get('title'),
             content,

Does that look okay to you? If so, I'll add this small change to the patch and push to master. Otherwise, let me know if you have a better way to handle this.

[Ben Sturmfels]

Thanks Ayleph, looks good to me, please go ahead. :)

[ayleph]

Thanks! Pushed to ​65d8ca7.