I tried to inform myself about the past Meeting on May 3rd, but ​http://mediagoblin.org/irclogs/ gives a 504 :(

Hi, I'm partiticipating to the opensource distributed searchengine ​YACY. Thus I like to encourage you, to offer webspiders an sitemap.xml file (and refer them in the robots.txt): ​https://en.wikipedia.org/wiki/Sitemaps

This allows the bot to just walk trough the flat list and he doesn't has to get the webstructure out of the parsed HTML pages. This avoids that the searchengine misses some content and speeds up the analysis process.

As you know there are other open distributed networks, that share the same problem of federation, as goblin. There is for example the ​Diaspora community, that creates a distributed social network, where some users provide some servers (so ​called pods). As federation is even here very complex, they are working currently to isolate the code of the protocol into a seperate layer.

Instead of every community spends a lot of resources on developing/testing/tuning/mainting a own inter-server protocoll for federation, I suggest to join a working group that addresses that problem and maybe raises some money so they can put attention on the development for a longer time. Maybe it results in a standard protocol as Jabber is for IM.

The D* community seeks for such kind of contacts, so please join in: ​https://www.loomio.org/discussions/6233