Audio submission failing (DetachedInstanceError in test_audio)

[Ben Sturmfels]

I'm seeing the following error in master (2a1082e3) when running tests:

$ bin/py.test mediagoblin/tests/test_submission.py::TestSubmission::test_audio --boxed

______ TestSubmission.test_audio_______
Traceback (most recent call last):
  File "/home/ben/work/ws/mediagoblin/mediagoblin/tests/test_submission.py", line 377, in test_audio
    self.check_normal_upload('Audio', path)
  File "/home/ben/work/ws/mediagoblin/mediagoblin/tests/test_submission.py", line 121, in check_normal_upload
    **self.upload_data(filename))
  File "/home/ben/work/ws/mediagoblin/mediagoblin/tests/test_submission.py", line 90, in do_post
    response = self.test_app.post(url, data, **kwargs)
  ⋮
  File "/home/ben/work/ws/mediagoblin/mediagoblin/submit/views.py", line 73, in submit_start
    user=request.user.username)
  ⋮
DetachedInstanceError: Instance <LocalUser at 0x7f607207d110> is not bound to a Session; attribute refresh operation cannot proceed

I've chased it down to mediagoblin/submit/lib.py:208. Before this line, user is ok, but afterwards, it's detached from the SQLAlchemy session.

I've looked back through the version control history, and this error is occurring for me right back to when test_audio was added in c41705bf. That suggests that the error is related to an incompatible version of a dependency installed on my computer. I've attached the output of pip freeze.

This appears to not be related to #5323: Audio submission failing.

[Ben Sturmfels]

Output of pip freeze --local

[Loic Dachary]

I think this is a duplicate of ​https://issues.mediagoblin.org/ticket/5385. The root of the problem is that installation instructions are incomplete. It was fixed for Debian GNU/Linux jessie but the same kind of test / patch should be done for all distributions.

[Christopher Allan Webber]

I'm pretty sure this is a session issue with SQL, not a dependency issue. We've had these before.

I think it's possibly a race condition, which might explain why it happens sometimes and not others(?).

Anyway, I'll look into it. Assigning to myself!

[Christopher Allan Webber]

I'm pretty baffled as to why this is happening with audio, but nothing else. I'm investigating further, but so far I'm confused.

[Christopher Allan Webber]

I left a previous comment that was completely wrong, because I accidentally ran the wrong test, so I deleted it.

Anyway, yes, this can be tested in isolation:

cwebber@oolong:~/devel/mediagoblin$ ./runtests.sh mediagoblin/tests/test_submission.py::TestSubmission::test_audio
Using ./bin/py.test
+ exec ./bin/py.test mediagoblin/tests/test_submission.py::TestSubmission::test_audio --boxed
========================================================== test session starts ===========================================================
platform linux2 -- Python 2.7.9, pytest-2.8.5, py-1.4.31, pluggy-0.3.1
rootdir: /home/cwebber/devel/mediagoblin/mediagoblin/tests, inifile: pytest.ini
plugins: xdist-1.13.1
collected 23 items

mediagoblin/tests/test_submission.py F

[Christopher Allan Webber]

I've confirmed through pdb that somehow the user becomes detached during the submit_media() step.

[Christopher Allan Webber]

Oh my goodness, it IS a dependency problem! Deep, deep down stepping through the mines of the processing system, I run into this:

: (Pdb) n
: > /home/cwebber/devel/mediagoblin/mediagoblin/processing/task.py(105)run()
: -> except Exception as exc:
: (Pdb) l
: 100  	
: 101  	                _log.debug('Processing {0}'.format(entry))
: 102  	
: 103  	                try:
: 104  	                    processor.process(**reprocess_info)
: 105  ->	                except Exception as exc:
: 106  	                    if processor.entry_orig_state == 'processed':
: 107  	                        _log.error(
: 108  	                            'Entry {0} failed to process due to the following'
: 109  	                            ' error: {1}'.format(entry.id, exc))
: 110  	                        _log.info(
: (Pdb) n
: > /home/cwebber/devel/mediagoblin/mediagoblin/processing/task.py(106)run()
: -> if processor.entry_orig_state == 'processed':
: (Pdb) exc
: NameError("global name 'audiolab' is not defined",)

Whaaaat

So apparenly on the way back up, Celery somehow invalidates our sqlalchemy session. Maybe because it's using an sqlite sqlalchemy session of its own or something. Anyway, this means we don't get the error we should, we get a totally different error. Infuriating!

Anyway, I'll work on making it so these tests only run if audiolab is importable.

[Christopher Allan Webber]

So anyway, yeah Loic was right. :)

I pushed a fix in 5573c41, so tests will be skipped in the right circumstances. Whew!

[Christopher Allan Webber]

All 0.8.2 tickets are being rolled over to 0.9.0