my setup:

virtual machine with 2 cores and 4 gb ram debian 7 postgresql paster nginx mediagoblin 0.6.2.dev ​init script from joar

i uploaded some videos. they get transcoded, but after some time it stops. new uploaded media don't appear on the website.

also stopping of celery don't work.

root@mediagoblin:~# /etc/init.d/mediagoblin-celeryd stop
Stopping GNU MediaGoblin Celery task queue: mediagoblin-celerydFailed to shut down after 20 seconds. Aborting.
 failed!

very often the same lines in the celery log:

Traceback (most recent call last):
  File "/var/www/mediagoblin/mediagoblin/media_types/video/transcoders.py", line 687, in _on_message
    self._progress_callback(data.get('percent'))
  File "/var/www/mediagoblin/mediagoblin/processing/__init__.py", line 43, in __call__
    self.entry.save()
  File "/var/www/mediagoblin/mediagoblin/db/base.py", line 39, in save
    sess.commit()
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/SQLAlchemy-0.8.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 721, in commit
    self.transaction.commit()
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/SQLAlchemy-0.8.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 352, in commit
    self._assert_active(prepared_ok=True)
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/SQLAlchemy-0.8.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 194, in _assert_active
    % self._rollback_exception
sqlalchemy.exc.InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: This result object does not return rows. It has been closed automatically.
/var/www/mediagoblin/local/lib/python2.7/site-packages/celery-3.1.10-py2.7.egg/celery/app/trace.py:348: RuntimeWarning: Exception raised outside body: InvalidRequestError("This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: This result object does not return rows. It has been closed automatically.",):
Traceback (most recent call last):
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/celery-3.1.10-py2.7.egg/celery/app/trace.py", line 253, in trace_task
    I, R, state, retval = on_error(task_request, exc, uuid)
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/celery-3.1.10-py2.7.egg/celery/app/trace.py", line 201, in on_error
    R = I.handle_error_state(task, eager=eager)
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/celery-3.1.10-py2.7.egg/celery/app/trace.py", line 85, in handle_error_state
    }[self.state](task, store_errors=store_errors)
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/celery-3.1.10-py2.7.egg/celery/app/trace.py", line 120, in handle_failure
    task.on_failure(exc, req.id, req.args, req.kwargs, einfo)
  File "/var/www/mediagoblin/mediagoblin/processing/task.py", line 154, in on_failure
    mark_entry_failed(entry_id, exc)
  File "/var/www/mediagoblin/mediagoblin/processing/__init__.py", line 334, in mark_entry_failed
    u'fail_metadata': {}})
  File "/var/www/mediagoblin/mediagoblin/db/util.py", line 31, in atomic_update
    synchronize_session=False)
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/SQLAlchemy-0.8.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/query.py", line 2710, in update
    update_op.exec_()
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/SQLAlchemy-0.8.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/persistence.py", line 829, in exec_
    self._do_pre()
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/SQLAlchemy-0.8.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/persistence.py", line 851, in _do_pre
    session._autoflush()
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/SQLAlchemy-0.8.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 1139, in _autoflush
    self.flush()
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/SQLAlchemy-0.8.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 1824, in flush
    self._flush(objects)
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/SQLAlchemy-0.8.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 1902, in _flush
    subtransactions=True)
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/SQLAlchemy-0.8.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 645, in begin
    nested=nested)
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/SQLAlchemy-0.8.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 215, in _begin
    self._assert_active()
  File "/var/www/mediagoblin/local/lib/python2.7/site-packages/SQLAlchemy-0.8.6-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 194, in _assert_active
    % self._rollback_exception
InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: This result object does not return rows. It has been closed automatically.

i uploaded 3 videos and 3 pictures, after some seconds celery stops working. the processes are still there but can't get stopped with the init script from joar. (some processes are still there, also the pidfile)

root@mediagoblin2:/var/www/mediagoblin# /etc/init.d/mediagoblin-celeryd status
mediagoblin-celeryd start/running, process 2851
root@mediagoblin2:/var/www/mediagoblin# /etc/init.d/mediagoblin-celeryd stop
Stopping GNU MediaGoblin Celery task queue: mediagoblin-celerydFailed to shut down after 20 seconds. Aborting.
 failed!

also strange is, that 2 files get the id of celery task as name (see picture).

my setup: vm with 8 cores and 2048 mb ram debian 7 postgres gunicorn nginx newest mediagoblin

this is a test of the image captcha system.