Move CloudFiles storage backend over to pyrax

[joar]

python-cloudfiles is no longer being maintained, we should move to pyrax instead: ​https://github.com/rackspace/pyrax

[joar]

Some work has been done over at ​https://github.com/joar/mediagoblin/commit/8391dc3719a58edd71ee7fc9103d1589f023d9b9. I do not think I will manage to find time to finish it.

[Christopher Allan Webber]

​https://pypi.python.org/pypi/python-swiftclient/ also an option

[Christopher Allan Webber]

claiming and assigning to 0.8.0

[ayleph]

I've been playing around with pyrax and can so far upload images to and delete images from RackSpace CloudFiles. However, it's not compatible with current MediaGoblin due to restrictions on the python six package.

  File "/path/to/mediagoblin/mediagoblin/tools/common.py", line 34, in import_component
    __import__(module_name)
  File "/path/to/mediagoblin/mediagoblin/storage/pyrax.py", line 28, in <module>
    import pyrax
  File "/path/to/mediagoblin/lib/python2.7/site-packages/pyrax/__init__.py", line 59, in <module>
    from novaclient.shell import OpenStackComputeShell as _cs_shell
  File "/path/to/mediagoblin/lib/python2.7/site-packages/novaclient/shell.py", line 47, in <module>
    from novaclient import client
  File "/path/to/mediagoblin/lib/python2.7/site-packages/novaclient/client.py", line 34, in <module>
    from oslo.utils import netutils
  File "/path/to/mediagoblin/lib/python2.7/site-packages/oslo/utils/netutils.py", line 13, in <module>
    from oslo_utils.netutils import *  # noqa
  File "/path/to/mediagoblin/lib/python2.7/site-packages/oslo_utils/netutils.py", line 170, in <module>
    class _ModifiedSplitResult(parse.SplitResult):
AttributeError: 'Module_six_moves_urllib_parse' object has no attribute 'SplitResult'

GMG setup.py requires six==1.5.2. A cursory internet search reveals this is sufficient for pyrax itself, but python-novaclient seems to require six>=1.6.

To test pyrax, I upgraded six (1.9.0) and everything is working okay so far in my limited testing. What's the reason for holding six at 1.5.2 in setup.py?

[ayleph]

I used Joar's changes as a starting point and created a branch with some additional changes. His branch was so far behind master I decided to create a new repo instead of forking his. I moved the pyrax code into a separate pyrax.py file, but I kept the idea of moving the options into a separate cloudfiles section of config_spec.ini.

The pyrax.py file is a straight copy of the cloudfiles.py with some content changed to enable uploading and to allow setting CORS headers on new objects. I haven't tested all of the methods yet.

​https://gitorious.org/mediagoblin/aylephs-mediagoblin/commits/9f62f305bbfd5929a3c4c923a342931d5d3a72cc

[ayleph]

The requirement on six has been removed in #5011.

[ayleph]

My branch referenced above, besides being completely out of date, is no longer accessible on gitorious. I need to track down the changes on my local machine, rebase, and make a patch or branch available somewhere.

[ayleph]

Just to touch base here, I completely intend to make this work and provide code for it. I've got a version of it in production already. There are a lot of deprecation warnings though, so I need to spend some time cleaning it up first. But for now, I'm going to remove ownership since nobody's actively working on this.

[ayleph]

I have an implementation of pyrax that I've been using for months, but I've just realized that it doesn't work with python3. It looks like development on pyrax is stopping in favor of a different SDK.

Pyrax is in the process of being deprecated in favor of the OpenStack SDK

​https://pypi.python.org/pypi/pyrax

In light of this news, I don't think moving over to pyrax is going to be a viable option.

[Boris Bobrov]

​https://pypi.python.org/pypi/openstacksdk i wonder how hard it will be to use it

[ayleph]

I moved away from pyrax. I'm currently using ​apache-libcloud with RackSpace Cloud Files. However, there's ​an issue that prevents accessing Cloud Files via SSL. I've added the missing functions on my own server, and I'm planning to offer that as a fix for this issue.

I really like the idea of integrating apache-libcloud, as I think it would give us an easier pathway to ​other storage providers.

[ayleph]

I've opened a ​pull request for the CloudFiles SSL URI issue.

[ayleph]

My pull request was accepted. We should be able to migrate to apache-libcloud when they build a release containing the change.

[ayleph]

apache-libcloud 2.2.0 currently on pypi includes the necessary changes. I'll provide a patch to allow using this to access Rackspace CloudFiles.

[ayleph]

Oh gosh thanks for the poke. I'll try to dig this up and see if it still works.

[Ben Sturmfels]

No panic - I'll move this to 0.11.0 for now.