Spectrogram thumbnail creation fails with non-local file storage

[ayleph]

Audio uploads fail when using the cloudfiles storage mechanism because the spectrogram thumbnail cannot be created. The spectrogram thumbnail generator assumes a local file path for the spectrogram and calls get_local_path which is not implemented for cloudfiles storage.

2014-08-31 01:51:09,264 INFO    [mediagoblin.media_types.audio.transcoders] 100% done...
2014-08-31 01:51:09,264 DEBUG   [mediagoblin.media_types.audio.transcoders] <gst.Message (none) from pipeline1 at 0x7f77440680c0>
2014-08-31 01:51:09,265 INFO    [mediagoblin.media_types.audio.transcoders] Done
2014-08-31 01:51:09,266 INFO    [mediagoblin.media_types.audio.transcoders] Quitting MainLoop gracefully...
/path/to/mediagoblin/mediagoblin/media_types/audio/audioprocessing.py:168: DeprecationWarning: axis != 0 for ndim == 1; this will raise an error in future versions of numpy
  samples = numpy.concatenate((numpy.zeros(add_to_start), samples), axis=1)
2014-08-31 01:51:12,125 DEBUG   [mediagoblin.media_types.audio.processing] Saving spectrogram...
2014-08-31 01:51:14,408 DEBUG   [mediagoblin.storage.cloudfiles] Sending [u'media_entries', u'814', u'AX11_-_Fiasco.spectrogram.jpg'] to cloudfiles...
2014-08-31 01:51:17,180 ERROR   [mediagoblin.processing.task] An unhandled exception was raised while processing <MediaEntry 814: AX11_-_Fiasco>
2014-08-31 01:51:17,180 WARNING [mediagoblin.processing] No idea what happened here, but it failed: NotImplementedError('This feature not implemented in this storage API implementation.',)
2014-08-31 01:51:17,186 DEBUG   [mediagoblin.tools.processing] No processing callback for <MediaEntry 814: AX11_-_Fiasco>
2014-08-31 01:51:17,186 WARNING [mediagoblin.processing] No idea what happened here, but it failed: NotImplementedError('This feature not implemented in this storage API implementation.',)
Error - <class 'mediagoblin.storage.NotImplementedError'>: This feature not implemented in this storage API implementation.
URL: https://goblinrefuge.com/mediagoblin/submit/
File '/path/to/mediagoblin/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in __call__
  app_iter = self.application(environ, sr_checker)
File '/path/to/mediagoblin/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/urlmap.py', line 203 in __call__
  return app(environ, start_response)
File '/path/to/mediagoblin/mediagoblin/app.py', line 265 in __call__
  return self.call_backend(environ, start_response)
File '/path/to/mediagoblin/mediagoblin/app.py', line 242 in call_backend
  response = controller(request)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 45 in wrapper
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 72 in new_controller_func
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 45 in wrapper
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 72 in new_controller_func
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 101 in wrapper
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/submit/views.py', line 67 in submit_start
  urlgen=request.urlgen)
File '/path/to/mediagoblin/mediagoblin/submit/lib.py', line 199 in submit_media
  run_process_media(entry, feed_url)
File '/path/to/mediagoblin/mediagoblin/submit/lib.py', line 250 in run_process_media
  task_id=entry.queued_task_id)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.13-py2.7.egg/celery/app/task.py', line 547 in apply_async
  link=link, link_error=link_error, **options)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.13-py2.7.egg/celery/app/task.py', line 736 in apply
  request=request, propagate=throw)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.13-py2.7.egg/celery/app/trace.py', line 354 in eager_trace_task
  uuid, args, kwargs, request)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.13-py2.7.egg/celery/app/trace.py', line 253 in trace_task
  I, R, state, retval = on_error(task_request, exc, uuid)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.13-py2.7.egg/celery/app/trace.py', line 240 in trace_task
  R = retval = fun(*args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/processing/task.py', line 98 in run
  processor.process(**reprocess_info)
File '/path/to/mediagoblin/mediagoblin/media_types/audio/processing.py', line 270 in process
  self.generate_thumb(size=thumb_size)
File '/path/to/mediagoblin/mediagoblin/media_types/audio/processing.py', line 196 in generate_thumb
  spectrogram_filepath = mgg.public_store.get_local_path(spectrogram)
File '/path/to/mediagoblin/mediagoblin/storage/__init__.py', line 160 in get_local_path
  self.__raise_not_implemented()
File '/path/to/mediagoblin/mediagoblin/storage/__init__.py', line 80 in __raise_not_implemented
  "This feature not implemented in this storage API implementation.")
NotImplementedError: This feature not implemented in this storage API implementation.

[Christopher Allan Webber]

Using the workbench should fix this. The file should be copied over locally before meddling with.

[ayleph]

I've handled this a different way on my instance. Instead of using the workbench to create a thumbnail of the spectrogram, I just add a default audio thumbnail and apply that to all audio files. No workbench needed. I prefer this to the resized spectrogram anyway. If this sounds interesting I can share the thumbnail image (which I took from a KDE theme) and the code to implement this change.

[ayleph]

Just to update, this is still an issue for people using remote storage. I hacked together my own solution on my own instance, but it hasn't been added to the GMG code. A proper workbench solution is still probably the preferred solution.