Right now if you have celery separated (processing is not synchronous) the PuSH push will happen *before* your processing ends up finishing. This is because the code actually happens in the view. Instead, we should move this code to the end of the processing code.

This is probably a fairly easy ticket, but I'm somewhat hesitant to mark it as "bitesized".

Our markdown library wraps the text in a div, that div breaks out of the <p> scope (as we are working with HTML, not XHTML) and then we get an orphaned </p> tag.

​http://validator.w3.org/check?uri=http%3A%2F%2Fgobblin.se%2Fu%2Fjoar%2F&charset=%28detect+automatically%29&doctype=Inline&group=0&user-agent=W3C_Validator%2F1.3

since the last code update of mediagoblin in goblinartists.net, I upload a music or video file but it doesn't appears on the site, so I did something to catch some debug info and this is what I got:

[goblinartists@goblinartists mediagoblin]$ CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd -f celeryd.log -l debug --autoscale=20,3
Segmentation fault
[goblinartists@goblinartists mediagoblin]$ cat celeryd.log 
Using celeryd config: mediagoblin_local.ini
Using ./bin/celeryd
No CloudFiles host URL specified, defaulting to Rackspace US
[2012-04-19 22:28:47,816: WARNING/MainProcess] -------------- celery@goblinartists.net v2.5.1
---- **** -----
--- * ***  * -- [Configuration]
-- * - **** ---   . broker:      sqlalchemy://sqlite:////srv/http/goblinartists.net/mediagoblin/kombu.db//
- ** ----------   . loader:      celery.loaders.default.Loader
- ** ----------   . logfile:     celeryd.log@DEBUG
- ** ----------   . concurrency: {min=3, max=20}
- ** ----------   . events:      OFF
- *** --- * ---   . beat:        OFF
-- ******* ----
--- ***** ----- [Queues]
 --------------   . celery:      exchange:celery (direct) binding:celery
                  

[Tasks]
  . celery.backend_cleanup
  . celery.chord
  . celery.chord_unlock
  . mediagoblin.processing.task.ProcessMedia
[2012-04-19 22:28:47,817: DEBUG/MainProcess] [Worker] Loading modules.
[2012-04-19 22:28:47,830: DEBUG/MainProcess] [Worker] Claiming components.
[2012-04-19 22:28:47,831: DEBUG/MainProcess] [Worker] Building boot step graph.
[2012-04-19 22:28:47,832: DEBUG/MainProcess] [Worker] New boot order: ['queues', 'pool', 'mediator', 'beat', 'autoreloader', 'timers', 'state-db', 'autoscaler', 'consumer']
[2012-04-19 22:28:47,834: DEBUG/MainProcess] Starting celery.concurrency.processes.TaskPool...
[2012-04-19 22:28:47,859: DEBUG/MainProcess] created semlock with handle 140224292536320
[2012-04-19 22:28:47,860: DEBUG/MainProcess] created semlock with handle 140224292532224
[2012-04-19 22:28:47,860: DEBUG/MainProcess] created semlock with handle 140224292528128
[2012-04-19 22:28:47,860: DEBUG/MainProcess] created semlock with handle 140224292524032
[2012-04-19 22:28:47,860: DEBUG/MainProcess] created semlock with handle 140224292519936
[2012-04-19 22:28:47,860: DEBUG/MainProcess] created semlock with handle 140224292515840
[2012-04-19 22:28:47,860: DEBUG/MainProcess] created semlock with handle 140224292511744
[2012-04-19 22:28:47,861: DEBUG/MainProcess] created semlock with handle 140224292507648
[2012-04-19 22:28:47,861: DEBUG/MainProcess] created semlock with handle 140224292503552
[2012-04-19 22:28:47,873: DEBUG/MainProcess] created semlock with handle 140224292499456
[2012-04-19 22:28:47,874: DEBUG/MainProcess] created semlock with handle 140224292495360
[2012-04-19 22:28:47,875: DEBUG/MainProcess] created semlock with handle 140224292491264
[2012-04-19 22:28:47,875: DEBUG/MainProcess] created semlock with handle 140224292487168
[2012-04-19 22:28:47,875: DEBUG/MainProcess] created semlock with handle 140224292483072
[2012-04-19 22:28:47,875: INFO/PoolWorker-1] child process calling self.run()
[2012-04-19 22:28:47,886: DEBUG/MainProcess] created semlock with handle 140224292478976
[2012-04-19 22:28:47,887: DEBUG/MainProcess] created semlock with handle 140224292474880
[2012-04-19 22:28:47,888: DEBUG/MainProcess] created semlock with handle 140224291917824
[2012-04-19 22:28:47,887: INFO/PoolWorker-2] child process calling self.run()
[2012-04-19 22:28:47,888: DEBUG/MainProcess] created semlock with handle 140224291913728
[2012-04-19 22:28:47,888: DEBUG/MainProcess] created semlock with handle 140224291909632
[2012-04-19 22:28:47,901: DEBUG/MainProcess] worker handler starting
[2012-04-19 22:28:47,901: INFO/PoolWorker-3] child process calling self.run()
[2012-04-19 22:28:47,903: DEBUG/MainProcess] celery.concurrency.processes.TaskPool OK!
[2012-04-19 22:28:47,903: DEBUG/MainProcess] Starting celery.worker.mediator.Mediator...
[2012-04-19 22:28:47,903: DEBUG/MainProcess] result handler starting
[2012-04-19 22:28:47,904: DEBUG/MainProcess] celery.worker.mediator.Mediator OK!
[2012-04-19 22:28:47,904: DEBUG/MainProcess] Starting celery.worker.autoscale.Autoscaler...
[2012-04-19 22:28:47,904: DEBUG/MainProcess] celery.worker.autoscale.Autoscaler OK!
[2012-04-19 22:28:47,905: DEBUG/MainProcess] Starting celery.worker.consumer.Consumer...
[2012-04-19 22:28:47,905: WARNING/MainProcess] celery@goblinartists.net has started.
[2012-04-19 22:28:47,905: DEBUG/MainProcess] Consumer: Re-establishing connection to the broker...
[2012-04-19 22:28:47,906: DEBUG/MainProcess] Consumer: Connection established.
[2012-04-19 22:28:47,989: DEBUG/MainProcess] basic.qos: prefetch_count->8
[2012-04-19 22:28:48,002: DEBUG/MainProcess] Consumer: Starting message consumer...
[2012-04-19 22:28:48,003: DEBUG/MainProcess] Consumer: Ready to accept tasks!
[2012-04-19 22:29:11,291: INFO/MainProcess] Got task from broker: mediagoblin.processing.task.ProcessMedia[c680cb6d-e17d-4d61-8729-55dae31b89b9]
[2012-04-19 22:29:11,326: DEBUG/MainProcess] Mediator: Running callback for task: mediagoblin.processing.task.ProcessMedia[c680cb6d-e17d-4d61-8729-55dae31b89b9]
[2012-04-19 22:29:11,326: DEBUG/MainProcess] TaskPool: Apply <function execute_and_trace at 0x3a2fb18> (args:('mediagoblin.processing.task.ProcessMedia', u'c680cb6d-e17d-4d61-8729-55dae31b89b9', [u'38'], {}) kwargs:{'hostname': 'goblinartists.net', 'request': {'retries': 0, 'task': 'mediagoblin.processing.task.ProcessMedia', 'utc': False, 'loglevel': 10, 'delivery_info': {'routing_key': u'celery', 'exchange': u'celery'}, 'args': [u'38'], 'expires': None, 'is_eager': False, 'eta': None, 'hostname': 'goblinartists.net', 'kwargs': {}, 'logfile': 'celeryd.log', 'id': u'c680cb6d-e17d-4d61-8729-55dae31b89b9'}})
[2012-04-19 22:29:11,328: DEBUG/MainProcess] Task accepted: mediagoblin.processing.task.ProcessMedia[c680cb6d-e17d-4d61-8729-55dae31b89b9] pid:1572
[2012-04-19 22:29:11,414: WARNING/PoolWorker-3] No idea what happened here, but it failed: NoResultFound('No row was found for one()',)
[2012-04-19 22:29:11,421: DEBUG/PoolWorker-2] worker got EOFError or IOError -- exiting
[2012-04-19 22:29:11,421: DEBUG/PoolWorker-1] worker got EOFError or IOError -- exiting
[2012-04-19 22:29:11,421: DEBUG/PoolWorker-2] worker exiting after 0 tasks
[2012-04-19 22:29:11,421: DEBUG/PoolWorker-1] worker exiting after 0 tasks
[2012-04-19 22:29:11,421: INFO/PoolWorker-2] process shutting down
[2012-04-19 22:29:11,421: INFO/PoolWorker-1] process shutting down
[2012-04-19 22:29:11,421: DEBUG/PoolWorker-2] running all "atexit" finalizers with priority >= 0
[2012-04-19 22:29:11,421: DEBUG/PoolWorker-1] running all "atexit" finalizers with priority >= 0
[2012-04-19 22:29:11,421: DEBUG/PoolWorker-2] running the remaining "atexit" finalizers
[2012-04-19 22:29:11,421: DEBUG/PoolWorker-1] running the remaining "atexit" finalizers
[2012-04-19 22:29:11,421: INFO/PoolWorker-2] process exiting with exitcode 0
[2012-04-19 22:29:11,421: INFO/PoolWorker-1] process exiting with exitcode 0
[2012-04-19 22:29:11,420: DEBUG/PoolWorker-3] worker got EOFError or IOError -- exiting
[2012-04-19 22:29:11,424: DEBUG/PoolWorker-3] worker exiting after 1 tasks
[2012-04-19 22:29:11,424: INFO/PoolWorker-3] process shutting down
[2012-04-19 22:29:11,424: DEBUG/PoolWorker-3] running all "atexit" finalizers with priority >= 0
[2012-04-19 22:29:11,424: DEBUG/PoolWorker-3] running the remaining "atexit" finalizers
[2012-04-19 22:29:11,424: INFO/PoolWorker-3] process exiting with exitcode 0

looks like [2012-04-19 22:29:11,414: WARNING/PoolWorker-3] No idea what happened here, but it failed: NoResultFound('No row was found for one()',) is some sqlalchemy problem, maybe my mediagoblin_local.ini should help to find out if this is a bug or not.

[goblinartists@goblinartists mediagoblin]$ cat mediagoblin_local.ini 
# If you want to make changes to this file, first copy it to
# mediagoblin_local.ini, then make the changes there.

[mediagoblin]
direct_remote_path = /mgoblin_static/
email_sender_address = "noreply@goblinartists.net"

## Uncomment and change to your DB's appropiate setting.
## Default is a local sqlite db "mediagoblin.db".
sql_engine = postgresql:///mediagoblin

# set to false to enable sending notices
email_debug_mode = false

# Set to false to disable registrations
allow_registration = true

## Uncomment this to turn on video or enable other media types
media_types = mediagoblin.media_types.image, mediagoblin.media_types.video, mediagoblin.media_types.ascii, mediagoblin.media_types.audio
## Uncomment this to put some user-overriding templates here
#local_templates = %(here)s/user_dev/templates/

[storage:queuestore]
base_dir = %(here)s/user_dev/media/queue

[storage:publicstore]
#base_dir = %(here)s/user_dev/media/public
#base_url = /mgoblin_media/
storage_class = mediagoblin.storage.cloudfiles:CloudFilesStorage
#storage_class = mediagoblin.storage.cloudfiles
cloudfiles_user = shackra
cloudfiles_api_key = asdasdasdasdasdasdasdasdasdads
cloudfiles_container = mediagoblin
cloudfiles_use_servicenet = false

[celery]
[goblinartists@goblinartists mediagoblin]$ 

and no, gmg dbupdate doesn't show any special output

[goblinartists@goblinartists mediagoblin]$ ./bin/gmg dbupdate
** Message: pygobject_register_sinkfunc is deprecated (GstObject)
/srv/http/goblinartists.net/mediagoblin/lib/python2.7/site-packages/SQLAlchemy-0.7.6-py2.7-linux-x86_64.egg/sqlalchemy/engine/default.py:473: SAWarning: Unicode type received non-unicode bind param value.
[goblinartists@goblinartists mediagoblin]$