Python 3: Audio upload fails due to missing scikits.audiolab

[Ben Sturmfels]

As noted in #813, there's no Python 3 version of scikits.audiolab; a required dependency for the current audio plugin to work. Uploading without scikits.audiolab gives the below backtrace.

While we can't easily solve the missing Python 3 dependency, there's no need for the upload to fail entirely, since scikits.audiolab is (I think) only used for generating the spectrogram thumbnail and background image.

2016-08-12 20:37:27,956 ERROR   [waitress] Exception when serving /submit/
Traceback (most recent call last):
  File "/home/ben/work/ws/mediagoblin/local/lib/python3.4/site-packages/waitress-1.0a2-py3.4.egg/waitress/channel.py", line 338, in service
    task.service()
  File "/home/ben/work/ws/mediagoblin/local/lib/python3.4/site-packages/waitress-1.0a2-py3.4.egg/waitress/task.py", line 169, in service
    self.execute()
  File "/home/ben/work/ws/mediagoblin/local/lib/python3.4/site-packages/waitress-1.0a2-py3.4.egg/waitress/task.py", line 399, in execute
    app_iter = self.channel.server.application(env, start_response)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/app.py", line 342, in __call__
    return self.call_backend(environ, start_response)
  File "/home/ben/work/ws/mediagoblin/local/lib/python3.4/site-packages/Werkzeug-0.11.10-py3.4.egg/werkzeug/wsgi.py", line 599, in __call__
    return self.app(environ, start_response)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/app.py", line 276, in call_backend
    return self._finish_call_backend(request, environ, start_response)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/app.py", line 318, in _finish_call_backend
    response = controller(request)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/decorators.py", line 47, in wrapper
    return controller(request, *args, **kwargs)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/decorators.py", line 74, in new_controller_func
    return controller(request, *args, **kwargs)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/decorators.py", line 47, in wrapper
    return controller(request, *args, **kwargs)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/decorators.py", line 74, in new_controller_func
    return controller(request, *args, **kwargs)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/decorators.py", line 103, in wrapper
    return controller(request, *args, **kwargs)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/submit/views.py", line 79, in submit_start
    urlgen=request.urlgen)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/submit/lib.py", line 210, in submit_media
    run_process_media(entry, feed_url)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/submit/lib.py", line 259, in run_process_media
    task_id=entry.queued_task_id)
  File "/home/ben/work/ws/mediagoblin/local/lib/python3.4/site-packages/celery-4.0.0rc3-py3.4.egg/celery/app/task.py", line 508, in apply_async
    link=link, link_error=link_error, **options)
  File "/home/ben/work/ws/mediagoblin/local/lib/python3.4/site-packages/celery-4.0.0rc3-py3.4.egg/celery/app/task.py", line 715, in apply
    ret = tracer(task_id, args, kwargs, request)
  File "/home/ben/work/ws/mediagoblin/local/lib/python3.4/site-packages/celery-4.0.0rc3-py3.4.egg/celery/app/trace.py", line 372, in trace_task
    I, R, state, retval = on_error(task_request, exc, uuid)
  File "/home/ben/work/ws/mediagoblin/local/lib/python3.4/site-packages/celery-4.0.0rc3-py3.4.egg/celery/app/trace.py", line 358, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/processing/task.py", line 101, in run
    processor.process(**reprocess_info)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/media_types/audio/processing.py", line 260, in process
    self.create_spectrogram(max_width=medium_width, fft_size=fft_size)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/media_types/audio/processing.py", line 158, in create_spectrogram
    fft_size=fft_size)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/media_types/audio/transcoders.py", line 63, in spectrogram
    numpy.hanning)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/media_types/audio/audioprocessing.py", line 109, in __init__
    max_level = get_max_level(input_filename)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/media_types/audio/audioprocessing.py", line 79, in get_max_level
    audio_file = audiolab.Sndfile(filename, 'r')
NameError: name 'audiolab' is not defined

[Ben Sturmfels]

FWIW, I did notice a couple of other Python packages on PyPI that wrap libsndfile. Could be worth a look as an alternative to scikits.audiolab:
​https://pypi.python.org/pypi?%3Aaction=search&term=libsndfile&submit=search

[Ben Sturmfels]

Making as "small", on the assumption allowing the system to continue to run (ie. not crash) in the absence of scikits.audiolab would be a suitable solution.

[Ben Sturmfels]

Here's a temporary patch that will at least allow audio uploads to proceed without crashing. A better place to do this audiolab monkey-patching would probably be transcoders.AudioThumbnailer.spectrogram, but I just need to make it work on my site for now. Obviously we don't want to be modifying upstream libraries also.

[Ben Sturmfels]

Temporary fix to mock out missing audiolab

[Jorge]

Ok, let me try to understand this.

The external freesound library used by mediagoblin uses numpy for 2-D waveform calculations. Numpy dropped backward compatibility for 2-D arrays in version ​1.10.2. Since then the also used scikits.audiolab is out of bound for axis=1 and causes an error for the waveform image created by freesound. As it had no release since 2010 it is not python3 compliant and is not available via pip3. That there will be an update is not very likely.

As the audio upload is not a core feature but a plugin - even a quite valuable one - it is not essential to be python3 compatible at the moment, although it's on the wishlist. Also the waveform image is not necessary for the actual functionality of this plugin and could be dropped. Being python3 compliant however gains a higher priority.

I suggest

• For the moment document this issue in "doc:media-types" and recommend to install numpy=1.9.1 and scikits.audiolab=0.10.2 as said in ticket:5457#comment:12.

• Create a fallback mechanism to show a ​static image instead of waveforms.

• Drop hard dependency on Audiolab and replace it with ​another waveform library or implement a function like transcoders.AudioThumbnailer.spectrogram.

• Document the option to monkeypatch freesound with axis=0 ​following numpy's deprecation error for all friends of git and python2.

[Jorge]

This patch addresses the first two points.

​PySoundFile is actively developed and can replace scikits.audiolab. ​wavefile lists more ​wrappers to libsndfile.

[Ben Sturmfels]

[Correct ticket this time]

Thanks Jorge, I've merged these changes which complement the recent changes to provide separate audio Thumbnailer clases for Python2/3 in 1038aea8.