Could not discover <file.mp4>

[Danilo Tomasoni]

Hi all, I followed the guide
​http://docs.mediagoblin.org/siteadmin/media-types.html#video
to play videos, but when I try to load an mp4 video celeryd gives me this traceback

2013-06-21 08:35:06,303 INFO    [celery] Got task from broker: mediagoblin.processing.task.ProcessMedia[79d48707-aeee-46d7-85be-f26fc5f35ae7]
2013-06-21 08:35:13,030 WARNING [celery] Traceback (most recent call last):
2013-06-21 08:35:13,032 WARNING [celery] File "/media/LaCie/home/mediagoblin/thompson-mediagoblin-2.6/mediagoblin/media_types/video/transcoders.py", line 458, in __on_discovered
2013-06-21 08:35:13,063 WARNING [celery] raise Exception('Could not discover {0}'.format(self.source_path))
2013-06-21 08:35:13,065 WARNING [celery] Exception
2013-06-21 08:35:13,066 WARNING [celery] :
2013-06-21 08:35:13,068 WARNING [celery] Could not discover /home/mediagoblin/thompson-mediagoblin-2.6/user_dev/media/queue/media_entries/79d48707-aeee-46d7-85be-f26fc5f35ae7/Lolo__Show__Sulla_neve.mp4
2013-06-21 08:35:13,721 ERROR   [celery] Task mediagoblin.processing.task.ProcessMedia[79d48707-aeee-46d7-85be-f26fc5f35ae7] raised exception: TypeError("argument of type 'NoneType' is not iterable",)
Traceback (most recent call last):
  File "/media/LaCie/home/mediagoblin/thompson-mediagoblin-2.6/lib/python2.6/site-packages/celery-2.5.3-py2.6.egg/celery/execute/trace.py", line 181, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/media/LaCie/home/mediagoblin/thompson-mediagoblin-2.6/mediagoblin/processing/task.py", line 92, in run
    entry.media_manager.processor(proc_state)
  File "/media/LaCie/home/mediagoblin/thompson-mediagoblin-2.6/mediagoblin/media_types/video/processing.py", line 91, in process_video
    store_metadata(entry, metadata)
  File "/media/LaCie/home/mediagoblin/thompson-mediagoblin-2.6/mediagoblin/media_types/video/processing.py", line 170, in store_metadata
    if key in metadata])
TypeError: argument of type 'NoneType' is not iterable

I'm currently using mediagoblin 0.4.1.dev in the thompson 421-mysql branch.

[Emily O'Leary]

If I recall correctly from IRC you are also running this on Python 2.6, correct?

What operating system is this running on? I recall there being an issue with Debian Squeeze, mentioned in #529.

I also remember running into this issue but I can't recall what the problem or solution was.

Does the file you are uploading have any associated meta data?

[Danilo Tomasoni]

Yes I'm using python 2.6.2
the OS is Ubuntu 9.04

How can I know if the uploaded file had associated metadata?..

I currently have an issue also with audio files

2013-06-24 11:14:39,582 INFO    [celery] Got task from broker: mediagoblin.processing.task.ProcessMedia[4e1a8690-6082-48a8-bca1-162e1a09d920]
2013-06-24 11:14:45,072 WARNING [celery] /media/LaCie/home/mediagoblin/thompson-mediagoblin-2.6/lib/python2.6/site-packages/SQLAlchemy-0.8.1-py2.6-linux-armv5tel.egg/sqlalchemy/engine/default.py:459: SAWarning: Unicode type received non-unicode bind param value
2013-06-24 11:14:47,464 ERROR   [celery] Task mediagoblin.processing.task.ProcessMedia[4e1a8690-6082-48a8-bca1-162e1a09d920] raised exception: GError('no property "tolerance" in element "audiorate0"',)
Traceback (most recent call last):
  File "/media/LaCie/home/mediagoblin/thompson-mediagoblin-2.6/lib/python2.6/site-packages/celery-2.5.3-py2.6.egg/celery/execute/trace.py", line 181, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/media/LaCie/home/mediagoblin/thompson-mediagoblin-2.6/mediagoblin/processing/task.py", line 92, in run
    entry.media_manager.processor(proc_state)
  File "/media/LaCie/home/mediagoblin/thompson-mediagoblin-2.6/mediagoblin/media_types/audio/processing.py", line 88, in process_audio
    progress_callback=progress_callback)
  File "/media/LaCie/home/mediagoblin/thompson-mediagoblin-2.6/mediagoblin/media_types/audio/transcoders.py", line 193, in transcode
    dst=dst))
GError: no property "tolerance" in element "audiorate0"

I don't know if it is related..

[rodney757]

I wonder if this is a gstreamer problem? I am having the same issues with discovery of audio and video files.

I'm using osx 10.7.

[Christopher Allan Webber]

Okay, interesting. So there's two places this has happened: on an older distro and on OSX. Notably it looks like rodney757 has the same gstreamer version I have (0.10.36) and same gst-python (0.10.22)... so it seems like it's unlikely that that's it.

It's possible it's a codec that's not installed though.

The root of the problem is that in this part of the code (on the video side at least) is that we should be getting back metadata here:

        # Extract metadata and keep a record of it
        metadata = transcoders.VideoTranscoder().discover(queued_filename)

Instead, None is being returned. Because of this, attempts to introspect or iterate over things result in things going explodey.

Why that is I'm not sure...

[Christopher Allan Webber]

I don't think this is a mediagoblin bug. There might be something we can do about the exception being raised here. I'm uncertain if this should be closed or not since it seems to have something to do with dependencies, but what, I'm not sure.

[Danilo Tomasoni]

In the mediagoblin IRC channel someone (I think ShawnRisk) said me that it is a version problem.
I use gstreamer-base 0.10.22 and in that version the tolerance property does not exist.

I think this should be accepted at least as a documentation bug
(it should be specified the minimum version of gstreamer to make the things work).

[Emily O'Leary]

Did you verify that a different version works? It might be important to add graceful error handling there even if a newer version works.

[Danilo Tomasoni]

Different how? the lastest? the one before the lastest? the 0.10.23?
On my server I cannot install different versions, that one is the only available.
My server is a sheevaplug arm powered..
I can try with my desktop but it has the version 0.10.36 which is the lastest..
and surely with that version it will work.
Right?

[David Ward]

Same issue here:

FreeBSD 9.1-RELEASE-p3 #0: Mon Apr 29 18:27:25 UTC 2013
gstreamer-0.10.36
gstreamer-plugins-0.10.36_3,3
py27-gstreamer-0.10.22_1

PS How do you see what version of mediagoblin you are running?

Error:

2013-07-18 01:54:55,366 DEBUG   [mediagoblin.processing.task] Processing <MediaEntry 1: A test of an avid>
2013-07-18 01:54:55,368 INFO    [mediagoblin.media_types.video.transcoders] Initializing VideoTranscoder...
2013-07-18 01:54:55,368 INFO    [mediagoblin.media_types.video.transcoders] Discovering /var/www/mediagoblin/user_dev/media/queue/media_entries/8ff37bdd-58ce-4d56-b7f1-944e77d9494a/00138-xvid-lrg.avi
Traceback (most recent call last):
  File "/var/www/mediagoblin/mediagoblin/media_types/video/transcoders.py", line 458, in __on_discovered
    raise Exception('Could not discover {0}'.format(self.source_path))
Exception: Could not discover /var/www/mediagoblin/user_dev/media/queue/media_entries/8ff37bdd-58ce-4d56-b7f1-944e77d9494a/00138-xvid-lrg.avi
2013-07-18 01:54:55,383 INFO    [mediagoblin.media_types.video.transcoders] Terminating MainLoop
2013-07-18 01:54:55,384 ERROR   [mediagoblin.processing.task] An unhandled exception was raised while processing <MediaEntry 1: A test of an avid>
2013-07-18 01:54:55,385 WARNING [mediagoblin.processing] No idea what happened here, but it failed: TypeError("argument of type 'NoneType' is not iterable",)
2013-07-18 01:54:55,398 WARNING [mediagoblin.processing] No idea what happened here, but it failed: TypeError("argument of type 'NoneType' is not iterable",)
Error - <type 'exceptions.TypeError'>: argument of type 'NoneType' is not iterable
URL: http://localhost:6543/submit/
File '/var/www/mediagoblin/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in __call__
  app_iter = self.application(environ, sr_checker)
File '/var/www/mediagoblin/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/urlmap.py', line 203 in __call__
  return app(environ, start_response)
File '/var/www/mediagoblin/mediagoblin/app.py', line 247 in __call__
  return self.call_backend(environ, start_response)
File '/var/www/mediagoblin/mediagoblin/app.py', line 224 in call_backend
  response = controller(request)
File '/var/www/mediagoblin/mediagoblin/decorators.py', line 48 in new_controller_func
  return controller(request, *args, **kwargs)
File '/var/www/mediagoblin/mediagoblin/submit/views.py', line 92 in submit_start
  run_process_media(entry, feed_url)
File '/var/www/mediagoblin/mediagoblin/submit/lib.py', line 90 in run_process_media
  task_id=entry.queued_task_id)
File '/var/www/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/app/task/__init__.py', line 445 in apply_async
  return self.apply(args, kwargs, task_id=task_id, **options)
File '/var/www/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/app/task/__init__.py', line 601 in apply
  request=request, propagate=throw)
File '/var/www/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/execute/trace.py', line 248 in eager_trace_task
  uuid, args, kwargs, request)
File '/var/www/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/execute/trace.py', line 181 in trace_task
  R = retval = fun(*args, **kwargs)
File '/var/www/mediagoblin/mediagoblin/processing/task.py', line 92 in run
  entry.media_manager.processor(proc_state)
File '/var/www/mediagoblin/mediagoblin/media_types/video/processing.py', line 91 in process_video
  store_metadata(entry, metadata)
File '/var/www/mediagoblin/mediagoblin/media_types/video/processing.py', line 170 in store_metadata
  if key in metadata])
TypeError: argument of type 'NoneType' is not iterable

[Christopher Allan Webber]

Hm, it's surprising that you'd be getting that same error in gstreamer 0.10.36... I wonder if all the components are being installed on FreeBSD.

We could support "metadata or {}" in the code, but I'm not sure why we're getting None back in the first place... I think this bug needs to investigate that.

DaveQB: is this happening on all videos? Could you pick a "working video" off of gobblin.se and see if that works better at all?

[Emily O'Leary]

I'm wondering what Metadata is in this case. If it only includes the metadata that is optional to have on a file (Title, etc) then empty meta data could exist (if that is the case then that probably explains how I got this error before on an MP4 file).

[rodney757]

I don't think we need to support "metadata or {}" in the code, because the reason it is returning None is because it is failing to discover the file.

[David Ward]

Hi,

Is there an easy way to filter for videos? Seems mostly photos there.
New to Media Goblin.

thanks!

Replying to cwebber:

Hm, it's surprising that you'd be getting that same error in gstreamer 0.10.36... I wonder if all the components are being installed on FreeBSD.

We could support "metadata or {}" in the code, but I'm not sure why we're getting None back in the first place... I think this bug needs to investigate that.

DaveQB: is this happening on all videos? Could you pick a "working video" off of gobblin.se and see if that works better at all?

[David Ward]

Tried that and we have an error still.
Exactly the same in fact :(

Replying to cwebber:

DaveQB: is this happening on all videos? Could you pick a "working video" off of gobblin.se and see if that works better at all?

[Christopher Allan Webber]

Okay, so that makes me feel fairly confirmed that this is not a bug in MediaGoblin but it is an issue with certain configurations or dependencies on certain distributions and operating systems.

I'm not going to close this immediately, but maybe I will in a few weeks? It seems like we need help debugging for these other operating systems, or possibly investigating what reason they can't provide this information.

This means that maybe updating the documentation for some platforms should be done, or filing bugs upstream for gstreamer.

I'm giving this bug 3 weeks before I close it out for someone to convince me otherwise ;)

[rodney757]

Okay, so I've been messing around trying to figure out how to get audio and video to upload. This isn't related to mg, but can possible help figure out this issue.

From the command line, run gst-launch-0.10 playbin uri=file:///<filepath>. This will give some more information wether you are missing dependencies or not. Then you can figure out what you need to install to be able to discover the file.

[saul]

In order to get discovery to work, I had to install libvpx (along with its -dev files if you are on Debian) and then compile gstreamer-bad. It was then necessary to rebuild gst-python.

[Christopher Allan Webber]

saulgoode, not sure I totally am clear; does the libvpx thing fix the problem? I wonder if it does for others also? It sounds like it.

I'm closing this bug again since nobody gave me indications that it wasn't a dependency issue. If someone proves it isn't, they can reopen the bug ;)