Opened 9 years ago

Closed 9 years ago

#5339 closed defect (fixed)

urlparse fails

Reported by: ayleph Owned by:
Priority: major Milestone:
Component: programming Keywords: api, feed, pump
Cc: Parent Tickets:

Description

Running latest master on python2.7, media uploads via pump clients fail when attempting to reach the feed URL.

From sapienTech's pypump logs:

In[2]: myimage = pump.Image(display_name='Test from PyPumpShell')
 
In [2]: myimage.from_file('/home/user/Desktop/MediaGoblinApp/goblinoid/images/newman.jpg')
---------------------------------------------------------------------------
PyPumpException                           Traceback (most recent call last)
/home/user/Documents/2015_summer/MediaGoblin/PyPump/pypump-shell in <module>()
----> 1 myimage.from_file('/home/user/Desktop/MediaGoblinApp/goblinoid/images/newman.jpg')
 
/home/user/Documents/2015_summer/MediaGoblin/PyPump/pypump/models/image.pyc in from_file(self, filename)
    119             self._post_activity(data)
    120         else:
--> 121             self._post_activity(data, unserialize=False)
    122
    123             # update image with display_name and content
 
/home/user/Documents/2015_summer/MediaGoblin/PyPump/pypump/models/__init__.pyc in _post_activity(self, activity, unserialize)
    105         )
    106
--> 107         data = self._pump.request(feed_url, method="POST", data=activity)
    108
    109         if not data:
 
/home/user/Documents/2015_summer/MediaGoblin/PyPump/pypump/pypump.pyc in request(self, endpoint, method, data, raw, params, retries, client, headers, timeout, **kwargs)
    339         )
    340
--> 341         raise PyPumpException(error)
    342
    343     def _requester(self, fnc, endpoint, raw=False, **kwargs):
 
PyPumpException: Request Failed to https://pump.goblinrefuge.com/api/user/sapientech/feed (response:
    <html>
    <head>
    <title>Server Error</title>
   
    </head>
    <body>
    <h1>Server Error</h1>
    <div style="text-align:center;font-family: monospace">
  <h1>YEOWCH... that's an error!</h1>
  <pre>
.-------------------------.
|     __            _     |
|    -, \_,------,_//     |
|     <\  ,--   --.\      |
|      / (x  ) ( X )      |
|      '  '--, ,--'\      |
|     / \ -v-v-u-v /      |
|     .  '.__.--__'.\     |
|    / ',___/ / \__/'     |
|    | |   ,'\_'/, ||     |
|    \_|   | | | | ||     |
|     W',_ ||| |||_''     |
|      |  '------'|       |
|      |__|     |_|_      |
|     ,,,-'     '-,,,     |
'-------------------------'
  </pre>
  <p>Sorry, our goblins had a little too much to drink.</p>
  <p>You may want to <a href="mailto:goblinpolice@goblinrefuge.com?Subject=Goblin%20Error" target="_top">alert the Goblin Police</a> and let them know what happened.</p>
</div>
   
    </body>
    </html> | status: 500)

From goblinrefuge.com server logs:

Error - <type 'exceptions.AttributeError'>: 'function' object has no attribute 'urlparse'
URL: https://pump.goblinrefuge.com/api/user/sapientech/feed
File '/path/to/mediagoblin/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in __call__
  app_iter = self.application(environ, sr_checker)
File '/path/to/mediagoblin/mediagoblin/app.py', line 342 in __call__
  return self.call_backend(environ, start_response)
File '/path/to/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.10.1-py2.7.egg/werkzeug/wsgi.py', line 591 in __call__
  return self.app(environ, start_response)
File '/path/to/mediagoblin/mediagoblin/app.py', line 276 in call_backend
  return self._finish_call_backend(request, environ, start_response)
File '/path/to/mediagoblin/mediagoblin/app.py', line 318 in _finish_call_backend
  response = controller(request)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 427 in wrapper
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/api/views.py', line 291 in feed_endpoint
  urlmap=request.app.url_map
File '/path/to/mediagoblin/mediagoblin/tools/routing.py', line 83 in extract_url_arguments
  parsed_url = urlparse.urlparse(url)
AttributeError: 'function' object has no attribute 'urlparse'
 
 
CGI Variables
-------------
  CONTENT_LENGTH: '1284'
  CONTENT_TYPE: 'application/json'
  CSRF_TOKEN: '3243560790715536161'
  DOCUMENT_ROOT: sanitized
  GATEWAY_INTERFACE: 'CGI/1.1'
  HTTPS: 'on'
  HTTP_ACCEPT: '*/*'
  HTTP_ACCEPT_ENCODING: 'gzip, deflate'
  HTTP_AUTHORIZATION: sanitized
  HTTP_CONNECTION: 'keep-alive'
  HTTP_CONTENT_LENGTH: '1284'
  HTTP_HOST: 'pump.goblinrefuge.com'
  HTTP_USER_AGENT: 'python-requests/2.7.0 CPython/2.7.6 Linux/3.13.0-53-lowlatency'
  PATH_INFO: '/api/user/sapientech/feed'
  PATH_TRANSLATED: sanitized
  REDIRECT_STATUS: '200'
  REMOTE_ADDR: '127.0.0.1'
  REMOTE_PORT: '48098'
  REQUEST_METHOD: 'POST'
  REQUEST_URI: '/api/user/sapientech/feed'
  SCRIPT_FILENAME: sanitized
  SERVER_ADDR: '127.0.0.1'
  SERVER_NAME: 'pump.goblinrefuge.com'
  SERVER_PORT: '8443'
  SERVER_PROTOCOL: 'HTTP/1.1'
  SERVER_SOFTWARE: 'lighttpd/1.4.35'
 
 
WSGI Variables
--------------
  application: <mediagoblin.app.MediaGoblinApp object at 0x7fcf00776190>
  paste.throw_errors: True
  werkzeug.request: <Request 'https://pump.goblinrefuge.com/api/user/sapientech/feed' [POST]>
  wsgi process: 'Multiprocess'
------------------------------------------------------------

Change History (5)

comment:1 by ayleph, 9 years ago

Cc: tsyesika added

comment:2 by ayleph, 9 years ago

Here are some notes from investigation.

$ git log -- mediagoblin/tools/routing.py
commit 39da9940581c8fea274c026df889069a60bf2cf5
Author: Berker Peksag <>
Date:   Fri Jun 5 19:45:00 2015 +0300

    Import urlparse from six.moves to work on both Python 2 and 3.
...
$ git show 39da994
commit 39da9940581c8fea274c026df889069a60bf2cf5
Author: Berker Peksag <>
Date:   Fri Jun 5 19:45:00 2015 +0300

    Import urlparse from six.moves to work on both Python 2 and 3.

diff --git a/mediagoblin/tools/routing.py b/mediagoblin/tools/routing.py
index ae7c715..8a30bab 100644
--- a/mediagoblin/tools/routing.py
+++ b/mediagoblin/tools/routing.py
@@ -15,10 +15,12 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import logging
-import urlparse
 
 import six
+
+from six.moves.urllib.parse import urlparse
 from werkzeug.routing import Map, Rule
+
 from mediagoblin.tools.common import import_component

I had six==1.9.0 installed on my system. I tried reverting back to six==1.5.2 (which used to be pinned in setup.py), but that didn't make a difference.

I tried reverting 39da994, but now I get a different error:

Error - <type 'exceptions.AttributeError'>: 'Activity' object has no attribute 'get_object'
URL: https://pumpdev.goblinrefuge.com/api/user/machalus/feed
File '/path/to/mediagoblin/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in __call__
  app_iter = self.application(environ, sr_checker)
File '/path/to/mediagoblin/mediagoblin/app.py', line 342 in __call__
  return self.call_backend(environ, start_response)
File '/path/to/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.9.6-py2.7.egg/werkzeug/wsgi.py', line 588 in __call__
  return self.app(environ, start_response)
File '/path/to/mediagoblin/mediagoblin/app.py', line 276 in call_backend
  return self._finish_call_backend(request, environ, start_response)
File '/path/to/mediagoblin/mediagoblin/app.py', line 318 in _finish_call_backend
  response = controller(request)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 427 in wrapper
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/api/views.py', line 424 in feed_endpoint
  return json_response(activity.serialize(request))
File '/path/to/mediagoblin/mediagoblin/db/mixin.py', line 490 in serialize
  "object": self.get_object.serialize(request),
AttributeError: 'Activity' object has no attribute 'get_object'


CGI Variables
-------------
  CONTENT_LENGTH: '2018'
  CONTENT_TYPE: 'application/json'
  CSRF_TOKEN: u'2753286736827930941'
  DOCUMENT_ROOT: sanitized
  GATEWAY_INTERFACE: 'CGI/1.1'
  HTTPS: 'on' 
  HTTP_ACCEPT_ENCODING: 'gzip, deflate'
  HTTP_ACCEPT_LANGUAGE: 'en-US,*'
  HTTP_AUTHORIZATION: sanitized
  HTTP_CONNECTION: 'Keep-Alive'
  HTTP_CONTENT_LENGTH: '2018'
  HTTP_COOKIE: 'mediagoblin_csrftoken=2753286736827930941'
  HTTP_HOST: 'pumpdev.goblinrefuge.com'
  HTTP_USER_AGENT: 'Mozilla/5.0'
  PATH_INFO: '/api/user/machalus/feed'
  PATH_TRANSLATED: sanitized
  REDIRECT_STATUS: '200'
  REMOTE_ADDR: '127.0.0.1'
  REMOTE_PORT: '51757'
  REQUEST_METHOD: 'POST'
  REQUEST_URI: '/api/user/machalus/feed'
  SCRIPT_FILENAME: sanitized
  SERVER_ADDR: '127.0.0.1'
  SERVER_NAME: 'pumpdev.goblinrefuge.com'
  SERVER_PORT: '8443'
  SERVER_PROTOCOL: 'HTTP/1.1'
  SERVER_SOFTWARE: 'lighttpd/1.4.35'


WSGI Variables
--------------
  application: <mediagoblin.app.MediaGoblinApp object at 0x7f32872a4150>
  paste.throw_errors: True 
  werkzeug.request: <Request 'https://pumpdev.goblinrefuge.com/api/user/machalus/feed' [POST]>
  wsgi process: 'Multiprocess'
------------------------------------------------------------

edit: removed email addresses

Last edited 9 years ago by ayleph (previous) (diff)

comment:3 by ayleph, 9 years ago

Judging by the deletion of test code, it looks like activity.get_object was removed sometime around 2d73983.

commit 2d73983e8ca353f47ad25642e1e0c292d94b20d5
Author: Jessica Tallon <>
Date:   Mon May 25 17:22:32 2015 +0200

    Fix some problems with activity mixins and migrations

...
--- a/mediagoblin/tests/test_modelmethods.py
+++ b/mediagoblin/tests/test_modelmethods.py
@@ -232,55 +232,3 @@ class TestUserUrlForSelf(MGClientTestCase):
             self.user(u'lindsay').url_for_self(fake_urlgen())
         assert excinfo.errisinstance(TypeError)
         assert 'object is not callable' in str(excinfo)
-
-class TestActivitySetGet(object):
-    """ Test methods on the Activity and ActivityIntermediator models """
-
-    @pytest.fixture(autouse=True)
-    def setup(self, test_app):
-        self.app = test_app
-        self.user = fixture_add_user()
-        self.obj = fixture_media_entry()
-        self.target = fixture_media_entry()
-
-    def test_set_activity_object(self):
-        """ Activity.set_object should produce ActivityIntermediator """
-        # The fixture will set self.obj as the object on the activity.
-        activity = fixture_add_activity(self.obj, actor=self.user)
-
-        # Assert the media has been associated with an AI
-        assert self.obj.activity is not None
-
-        # Assert the AI on the media and object are the same
-        assert activity.object == self.obj.activity
-
-    def test_activity_set_target(self):
-        """ Activity.set_target should produce ActivityIntermediator """
-        # This should set everything needed on the target
-        activity = fixture_add_activity(self.obj, actor=self.user)
-        activity.set_target(self.target)
-
-        # Assert the media has been associated with the AI
-        assert self.target.activity is not None
-
-        # assert the AI on the media and target are the same
-        assert activity.target == self.target.activity
-
-    def test_get_activity_object(self):
-        """ Activity.get_object should return a set object """
-        activity = fixture_add_activity(self.obj, actor=self.user)
-
-        print("self.obj.activity = {0}".format(self.obj.activity))
-
-        # check we now can get the object
-        assert activity.get_object is not None
-        assert activity.get_object.id == self.obj.id
-
-    def test_get_activity_target(self):
-        """ Activity.set_target should return a set target """
-        activity = fixture_add_activity(self.obj, actor=self.user)
-        activity.set_target(self.target)
-
-        # check we can get the target
-        assert activity.get_target is not None
-        assert activity.get_target.id == self.target.id

...

edit: removed email addresses

Last edited 9 years ago by ayleph (previous) (diff)

comment:5 by ayleph, 9 years ago

Cc: tsyesika removed
Resolution: fixed
Status: newclosed
Summary: API feed request failsurlparse fails

This ticket ended up having multiple issues. I'm going to split it. Since berker has already solved the urlparse issue, I'll rename this ticket to something about urlparse and close it. I'll open a new ticket for the API feed issue.

Note: See TracTickets for help on using tickets.