GStreamer bails unexpectedly on video file

[ayleph]

I get this error when trying to upload a working mp4 file, using current master (​https://gitorious.org/mediagoblin/mediagoblin/commit/714008693e08fc8a229fb3f7822c70c132aade7f). The same file is processed successfully on 0.7.2-dev.

2015-03-15 11:24:14,893 INFO    [mediagoblin.media_types] Info about file found, checking further
2015-03-15 11:24:14,893 INFO    [mediagoblin.media_types] checking the contents with sniffer
2015-03-15 11:24:14,893 INFO    [mediagoblin.media_types.video.processing] Sniffing mediagoblin.media_types.video
2015-03-15 11:24:14,893 INFO    [mediagoblin.media_types.tools] Discovering /tmp/tmpn4ekgH...
2015-03-15 11:24:14,898 WARNING [mediagoblin.media_types.video.processing] GStreamer: gst-stream-error-quark: This file is incomplete and cannot be played. (9)
2015-03-15 11:24:14,898 INFO    [mediagoblin.media_types] sniffer says it will not accept the file
2015-03-15 11:24:14,899 DEBUG   [mediagoblin.media_types] GStreamer: gst-stream-error-quark: This file is incomplete and cannot be played. (9)

[ayleph]

More details: here's the output I get when trying to add the file locally via bin/gmg addmedia.

(mediagoblin-0.8.0) ~/mediagoblin $ bin/gmg addmedia ayleph ~/3582\ Clamp\ coupon\ 2014-02-15.mp4
/path/to/mediagoblin/lib/python2.7/site-packages/SQLAlchemy-0.8.7-py2.7-linux-x86_64.egg/sqlalchemy/engine/default.py:471: SAWarning: Unicode type received non-unicode bind param value.
INFO:mediagoblin.media_types.video.processing:Sniffing mediagoblin.media_types.video
WARNING:mediagoblin.media_types.video.processing:GStreamer: gst-stream-error-quark: This file is incomplete and cannot be played. (9)
Traceback (most recent call last):
  File "bin/gmg", line 9, in <module>
    load_entry_point('mediagoblin==0.8.1.dev0', 'console_scripts', 'gmg')()
  File "/path/to/mediagoblin/mediagoblin/gmg_commands/__init__.py", line 142, in main_cli
    args.func(args)
  File "/path/to/mediagoblin/mediagoblin/gmg_commands/addmedia.py", line 104, in addmedia
    upload_limit=upload_limit, max_file_size=max_file_size)
  File "/path/to/mediagoblin/mediagoblin/submit/lib.py", line 138, in submit_media
    media_type, media_manager = sniff_media(submitted_file, filename)
  File "/path/to/mediagoblin/mediagoblin/media_types/__init__.py", line 150, in sniff_media
    return type_match_handler(tmp_media_file, filename)
  File "/path/to/mediagoblin/mediagoblin/media_types/__init__.py", line 124, in type_match_handler
    sniffer(media_file)
  File "/path/to/mediagoblin/mediagoblin/media_types/video/processing.py", line 56, in sniffer
    raise MissingComponents(u'GStreamer: {0}'.format(unicode(e)))
mediagoblin.media_types.MissingComponents: GStreamer: gst-stream-error-quark: This file is incomplete and cannot be played. (9)

[Boris Bobrov]

-

[Boris Bobrov]

I don't think there is much I can do here. The code in mg works as expected.

One of the options would be to add an option to skip the check and try to convert the video even if discoverer says such things. But sorry, it will not be in 0.8.0.

Though the bug is still valid. Maybe someone should try talking to gstreamer devs?

[anongoblin]

I solved this problem by installing this packages (in Debian 8):
gstreamer0.10-plugins-ugly gstreamer0.10-plugins-bad gstreamer0.10-tools gstreamer0.10-plugins-good gstreamer0.10-plugins-base gstreamer0.10-x python-gst0.10

[Christopher Allan Webber]

ayleph, does the above suggestion help with your issue at all?

[ayleph]

Nope. gstreamer0.10 doesn't apply to latest master, and I already have all of my distro's gstreamer1.x plugins installed.

This wasn't an issue back with gstreamer0.10 though.

[anongoblin]

I'm seeing the same issue, but only on video files that are larger than ~4MB.

I was able to partially work around this issue by modifying the sniff_media method to call type_match_handler on the media file itself rather than the tmpfile copy. I suspect that the earlier call to shutil.copyfileobj is somehow producing a corrupt file object.

Note I say this is a partial workaround because I was only able to use gmg addmedia to add the file. The web interface still fails with the gstreamer error. Perhaps there's another file copy going wrong.

I didn't have time to find someone to get me an account here, so I'm posting anonymously. If I can help debug this further I'm ssweeny on Freenode.

[ayleph]

I think I found a simple fix for this. It's working great so far on my public instance. More details below the patch.

From 5b7a05abe0be7178e02292ac00cdec87891446cc Mon Sep 17 00:00:00 2001
From: ayleph <ayleph@thisshitistemp.com>
Date: Tue, 17 Nov 2015 23:56:31 -0500
Subject: [PATCH] Fix for issue 5070 gst incomplete file

Seek tmp_media_file to 0 before attempting to discover the file type.
This prevents the following error when discovering mp4 video files.

"gst-stream-error-quark: This file is incomplete and cannot be played"
---
 mediagoblin/media_types/__init__.py | 1 + 
 1 file changed, 1 insertion(+)

diff --git a/mediagoblin/media_types/__init__.py b/mediagoblin/media_types/__init__.py
index 97e4fac..9f6043e 100644
--- a/mediagoblin/media_types/__init__.py
+++ b/mediagoblin/media_types/__init__.py
@@ -146,6 +146,7 @@ def sniff_media(media_file, filename):
     tmp_media_file = tempfile.NamedTemporaryFile()
     shutil.copyfileobj(media_file, tmp_media_file)
     media_file.seek(0)
+    tmp_media_file.seek(0)
     try:
         return type_match_handler(tmp_media_file, filename)
     except TypeNotFound as e:
-- 
2.6.2

This appears to only affect mp4 files. The gstreamer error This file is incomplete and cannot be played appears in gst-plugins-good-1.y.z/gst/isomp4/qtdemux.c in the code which demultiplexes the file header. If the file doesn't get back the expected number of bytes, this error is given. Seeking the file to offset 0 first seems to address this issue.

And yes, media_file.seek(0) is still necessary. Without that line, a separate gstreamer error pops up: Stream contains no data.

[Christopher Allan Webber]

I haven't tested this but I trust your judgement on this one... the code change looks simple and seems logical. Pushed!

[Christopher Allan Webber]

All 0.8.2 tickets are being rolled over to 0.9.0