upload transcode fails on mp3 or ogg

[Josh Cox]

I have tried with both an ogg and mp3, and I get the feeling I might have something else wrong, and thought I'd post here for some more experienced eyes. I have forked dachary's docker repo here: ​https://notabug.org/thoth/mediagoblin-docker and rancher template here ​https://github.com/ohmydocker/ohmydocker-catalog/tree/master/templates/mediagoblin Noob here with this project, so I certainly could have borked something. Here's the last section of logs right after the failure:

7/1/2017 7:49:11 PM2017-07-02 00:49:11,476 INFO    [mediagoblin.media_types.audio.transcoders] Done
7/1/2017 7:49:11 PM2017-07-02 00:49:11,619 INFO    [mediagoblin.media_types.audio.transcoders] Quitting MainLoop gracefully...
7/1/2017 7:49:40 PM2017-07-02 00:49:40,777 ERROR   [mediagoblin.processing.task] An unhandled exception was raised while processing <MediaEntry 2: RockinDoc-1-24-95>
7/1/2017 7:49:40 PM2017-07-02 00:49:40,779 WARNING [mediagoblin.processing] No idea what happened here, but it failed: OSError(39, 'Directory not empty')
7/1/2017 7:49:40 PM2017-07-02 00:49:40,900 WARNING [mediagoblin.processing] No idea what happened here, but it failed: OSError(39, 'Directory not empty')
7/1/2017 7:49:40 PMStarting server in PID 37.
7/1/2017 7:49:40 PMTraceback (most recent call last):
7/1/2017 7:49:40 PM  File "./bin/paster", line 11, in <module>
7/1/2017 7:49:40 PM    load_entry_point('PasteScript', 'console_scripts', 'paster')()
7/1/2017 7:49:40 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/PasteScript-2.0.2-py2.7.egg/paste/script/command.py", line 102, in run
7/1/2017 7:49:41 PM    invoke(command, command_name, options, args[1:])
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/PasteScript-2.0.2-py2.7.egg/paste/script/command.py", line 141, in invoke
7/1/2017 7:49:41 PM    exit_code = runner.run(args)
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/PasteScript-2.0.2-py2.7.egg/paste/script/command.py", line 236, in run
7/1/2017 7:49:41 PM    result = self.command()
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/PasteScript-2.0.2-py2.7.egg/paste/script/serve.py", line 319, in command
7/1/2017 7:49:41 PM    serve()
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/PasteScript-2.0.2-py2.7.egg/paste/script/serve.py", line 303, in serve
7/1/2017 7:49:41 PM    server(app)
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/PasteDeploy-1.5.2-py2.7.egg/paste/deploy/loadwsgi.py", line 189, in server_wrapper
7/1/2017 7:49:41 PM    **context.local_conf)
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/PasteDeploy-1.5.2-py2.7.egg/paste/deploy/util.py", line 55, in fix_call
7/1/2017 7:49:41 PM    val = callable(*args, **kw)
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/paste_factory.py", line 154, in run_fcgi_fork
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_fork.py", line 140, in run
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/preforkserver.py", line 135, in run
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/preforkserver.py", line 328, in _spawnChild
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/preforkserver.py", line 406, in _child
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py", line 669, in run
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py", line 705, in process_input
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py", line 805, in _do_params
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py", line 789, in _start_request
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py", line 577, in run
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py", line 328, in flush
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py", line 299, in _write
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py", line 720, in writeRecord
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py", line 532, in write
7/1/2017 7:49:41 PM  File "/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py", line 507, in _sendall
7/1/2017 7:49:41 PMsocket.error: [Errno 32] Broken pipe
7/1/2017 9:49:30 PM/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.12.2-py2.7.egg/werkzeug/filesystem.py:63: BrokenFilesystemWarning: Detected a misconfigured UNIX filesystem: Will use UTF-8 as filesystem encoding instead of 'ANSI_X3.4-1968'
7/1/2017 9:49:30 PM  BrokenFilesystemWarning)
7/1/2017 9:49:31 PM/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/SQLAlchemy-1.1.11-py2.7-linux-x86_64.egg/sqlalchemy/sql/sqltypes.py:219: SAWarning: Unicode type received non-unicode bind param value 'active'. (this warning may be suppressed after 10 occurrences)
7/1/2017 9:49:32 PM/srv/mediagoblin.example.org/mediagoblin/lib/python2.7/site-packages/SQLAlchemy-1.1.11-py2.7-linux-x86_64.egg/sqlalchemy/sql/sqltypes.py:219: SAWarning: Unicode type received non-unicode bind param value 'moderator'. (this warning may be suppressed after 10 occurrences)

Connected

[ayleph]

Hi thoth,

The docker image you referenced isn't supported by the MediaGoblin maintainers. I have no idea if it was configured correctly to support different media types or if it's running even remotely recent code. Can you reproduce this error running mediagoblin directly on your machine instead of through docker?

[Josh Cox]

That docker image is a fork of the docker repo refernced on the main page of the site as "Easy Deployment" here ​https://mediagoblin.org/

which references this wiki page:
​https://wiki.mediagoblin.org/EasyDeployment

the fork is mine, and I did "update" to the best of my knowledge, but I believe I need help at this point.

If there is an official supported docker image point me to it, if not help me fix the errors here and hopefully we can get an official one going and contributed back into the project.

[ayleph]

There is not an official supported docker image.

As for your errors, I can't tell from the traceback exactly what the issue is, but it appears there was an error processing the media.

2017-07-02 00:49:40,777 ERROR   [mediagoblin.processing.task] An unhandled exception was raised while processing <MediaEntry 2: RockinDoc-1-24-95>

You could try to modify the processing code to output more information, such as below (untested, just a quick mock-up).

diff --git a/mediagoblin/processing/task.py b/mediagoblin/processing/task.py
index 5e0e772d..0e52a601 100644
--- a/mediagoblin/processing/task.py
+++ b/mediagoblin/processing/task.py
@@ -137,8 +137,8 @@ class ProcessMedia(celery.Task):
 
         except Exception as exc:
             _log.error('An unhandled exception was raised while'
-                    + ' processing {0}'.format(
-                        entry))
+                    + ' processing {0}: {1}'.format(
+                        entry, exc))
 
             mark_entry_failed(entry.id, exc)
             json_processing_callback(entry)

It may tell you the exact same thing that you can see next in the log:

OSError(39, 'Directory not empty')

Taking a quick look through the code, I see a few places in the local file storage class where we note that deleting a file might cause an OSError. As far as the audio plugin goes, the most likely candidate for throwing this is probably the following code in mediagoblin/media_types/audio/processing.py.

    def process(self, quality=None, fft_size=None, thumb_size=None,
                medium_width=None):
        self.common_setup()

        self.transcode(quality=quality)
        self.copy_original()

        self.create_spectrogram(max_width=medium_width, fft_size=fft_size)
        self.generate_thumb(size=thumb_size)

        self.delete_queue_file()

So maybe as a quick test, I'd comment out the line self.delete_queue_file() and then try uploading an audio file again. If you still get the OSError about the directory not being empty, then that line probably isn't responsible. But if you do get that error, it gives you a clue toward solving the issue.

[Ben Sturmfels]

Closing this issue. I'm not aware of any issues with transcoding mp3/ogg currently and there hasn't been any update in this issue.