fastcgi fails on Python3

[Michael Lynch]

Tested this on e34916.

In previous versions, I used MediaGoblin's fastcgi support to connect nginx to MediaGoblin. In the current Python3 build, fastcgi causes a failure:

$ docker build -t mediagoblin-python3 -f Dockerfile-python3 . && \
  docker run --entrypoint "./lazyserver.sh" mediagoblin-python3 --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543

+ export CELERY_ALWAYS_EAGER=true
+ ./bin/paster serve paste.ini --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543 --reload
Using paster config: paste.ini
Using ./bin/paster
Traceback (most recent call last):
  File "./bin/paster", line 11, in <module>
    load_entry_point('PasteScript', 'console_scripts', 'paster')()
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteScript-2.0.2-py3.7.egg/paste/script/command.py", line 102, in run
    invoke(command, command_name, options, args[1:])
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteScript-2.0.2-py3.7.egg/paste/script/command.py", line 141, in invoke
    exit_code = runner.run(args)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteScript-2.0.2-py3.7.egg/paste/script/command.py", line 236, in run
    result = self.command()
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteScript-2.0.2-py3.7.egg/paste/script/serve.py", line 282, in command
    relative_to=base, global_conf=vars)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteScript-2.0.2-py3.7.egg/paste/script/serve.py", line 324, in loadserver
    relative_to=relative_to, **kw)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/loadwsgi.py", line 261, in loadserver
    return loadobj(SERVER, uri, name=name, **kw)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/loadwsgi.py", line 277, in loadobj
    global_conf=global_conf)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/loadwsgi.py", line 302, in loadcontext
    global_conf=global_conf)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/loadwsgi.py", line 326, in _loadconfig
    return loader.get_context(object_type, name, global_conf)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/loadwsgi.py", line 459, in get_context
    section)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/loadwsgi.py", line 481, in _context_from_use
    object_type, name=use, global_conf=global_conf)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/loadwsgi.py", line 412, in get_context
    global_conf=global_conf)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/loadwsgi.py", line 302, in loadcontext
    global_conf=global_conf)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/loadwsgi.py", line 334, in _loadegg
    return loader.get_context(object_type, name, global_conf)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/loadwsgi.py", line 625, in get_context
    object_type, name=name)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/loadwsgi.py", line 645, in find_egg_entry_point
    pkg_resources.require(self.spec)
  File "/opt/mediagoblin/lib/python3.7/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/mediagoblin/lib/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'flup' distribution was not found and is required by the application

If I add ./bin/pip install flup to Dockerfile-python3, I still get the following failure:

Starting server in PID 11.
Traceback (most recent call last):
  File "./bin/paster", line 11, in <module>
    load_entry_point('PasteScript', 'console_scripts', 'paster')()
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteScript-2.0.2-py3.7.egg/paste/script/command.py", line 102, in run
    invoke(command, command_name, options, args[1:])
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteScript-2.0.2-py3.7.egg/paste/script/command.py", line 141, in invoke
    exit_code = runner.run(args)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteScript-2.0.2-py3.7.egg/paste/script/command.py", line 236, in run
    result = self.command()
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteScript-2.0.2-py3.7.egg/paste/script/serve.py", line 319, in command
    serve()
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteScript-2.0.2-py3.7.egg/paste/script/serve.py", line 303, in serve
    server(app)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/loadwsgi.py", line 195, in server_wrapper
    **context.local_conf)
  File "/opt/mediagoblin/lib/python3.7/site-packages/PasteDeploy-2.0.1-py3.7.egg/paste/deploy/util.py", line 55, in fix_call
    val = callable(*args, **kw)
  File "/opt/mediagoblin/lib/python3.7/site-packages/flup/server/paste_factory.py", line 138, in run_fcgi_fork
    debug = asbool(debug)
  File "/opt/mediagoblin/lib/python3.7/site-packages/flup/server/paste_factory.py", line 7, in asbool
    if isinstance(obj, (str, unicode)):
NameError: name 'unicode' is not defined

[Ben Sturmfels]

Hi Michael,

Again sorry for the delay. I can also replicate this on the current MediaGoblin master branch. It looks to me as though the flup package simply doesn't support Python 3. What you be interested in contributing changes to fix the fastcgi support, or perhaps alternately removing the fastcgi support entirely inc. documentation?

What do you think? How have you been dealing with this in the 6 months since this issue was raised?

Regards,
Ben

[Ben Sturmfels]

See also for reference:
https://issues.mediagoblin.org/ticket/5560#comment:1

[Michael Lynch]

How have you been dealing with this in the 6 months since this issue was raised?

I just have nginx talk to MediaGoblin via proxy_pass:

​https://github.com/mtlynch/mediagoblin-docker/blob/6bf661b51011ff562a6be58dd22dfa190e8a7696/default.conf.tmpl#L56

It's hard for me to agree to make these changes. The last two times I've tried to contribute, there have been 6+ month turnarounds before I got any sort of response.

[Ben Sturmfels]

Thanks Michael. If you're willing to look at making the relevant documentation changes, I'm happy to commit to reviewing within 1 week. I'll close this ticket and have created one here regarding the required documentation updates:

https://issues.mediagoblin.org/ticket/5587#ticket