Opened 9 years ago

Closed 8 years ago

#380 closed defect (fixed)

SQL: celery/kombu transport database

Reported by: Elrond Owned by:
Priority: major Milestone: 0.3.0
Component: programming Keywords: sql
Cc: Parent Tickets:

Description

celery (and kombu) need a new transport and result database when
moving away from mongodb.


-  celery/kombu and sqlalchemy:
   `http://docs.celeryproject.org/en/latest/getting-started/brokers/sqlalchemy.html <http://docs.celeryproject.org/en/latest/getting-started/brokers/sqlalchemy.html>`_
-  whether a database is actually a good event oriented transport,
   is an open question. Maybe amqplib and rabbitmq are the better
   answer?

For the result thing we maybe should really use sqlalchemy?

Goal of this bug:


1. Research options for celery/kombu
2. come to a conclusion what to use
3. Implement it



Subtickets

Change History (5)

comment:1 Changed 9 years ago by Will Kahn-Greene

The original url for this bug was http://bugs.foocorp.net/issues/739 .
Relations:
#379: blocks

comment:2 Changed 9 years ago by Christopher Allan Webber

Keywords: sql added
Milestone: 0.2.1
Priority: minormajor

comment:3 Changed 8 years ago by Elrond

Milestone: 0.2.10.2.2

If we want sql in 0.2.2, we should have an answer to this ticket.

comment:4 Changed 8 years ago by Elrond

Okay, this is ready in cwebber's celerysql branch.

What this branch changes:

  • Switch default kombu transport from mongo to sqlalchemy, using a local kombu.db file next to the mediagoblin.ini.
  • Switch the result backend for celery from mongo to sqlalchemy, using a local celery.db file next to the mediagoblin.ini.

What does this mean for you?
If you're not running celeryd: Nothing!

If you're running celeryd:

  1. Install all new dependencies: sqlalchemy, kombu-sqlalchemy. For the future you will need sqlalchemy-migrate also, so just install it too. setup.py has the proper dependencies knowledge, so use it, if you can.
  2. Remember to restart celeryd and the main MediaGoblin server.

If you're using celeryd in a very special way, especially, if you're using the results in the mongodb, contact us sooner than later, please.

comment:5 Changed 8 years ago by Elrond

Resolution: fixed
Status: acceptedclosed

Merged!

Note: See TracTickets for help on using tickets.