Opened 8 years ago

Closed 8 years ago

#585 closed defect (fixed)

When publishing through API, no PuSH to push_urls

Reported by: mrb Owned by:
Priority: major Milestone: 0.3.3
Component: programming Keywords: review
Cc: Parent Tickets:

Description

The block of code that takes care of pushing the item to push_urls is absent in the api plugin.

Subtickets

Change History (2)

comment:1 Changed 8 years ago by spaetz

Keywords: review added
Milestone: 0.3.3

I fixed this issue in my branch 436_celery_push. Please review:

    Make PuSHing the Pubhubsubbub server an async task (#536, #585)
    
    Notifying the PuSH servers had 3 problems. 
    
    1) it was done immediately after sending of the processing task to celery. So if celery was run in a separate
    process we would notify the PuSH servers before the new media was processed/
    visible. (#536)
    
    2) Notification code was called in submit/views.py, so submitting via the
       API never resulted in notifications. (#585)
    
    3) If Notifying the PuSH server failed, we would never retry.
    
    The solution was to make the PuSH notification an asynchronous subtask. This
    way: 1) it will only be called once async processing has finished, 2) it
    is in the main processing code path, so even API calls will result in
    notifications, and 3) We retry 3 times in case of failure before giving up.
    If the server is in a separate process, we will wait 3x 2 minutes before
    retrying the notification.
    
    The only downside is that the celery server needs to have access to the internet
    to ping the PuSH server. If that is a problem, we need to make the task belong
    to a special group of celery servers that has access to the internet.
    
    As a side effect, I believe I removed the limitation that prevented us from
    upgrading celery.

comment:2 Changed 8 years ago by Christopher Allan Webber

Resolution: fixed
Status: newclosed

This is also fixed as part of merging 436_celery_push / #436. Thanks much! :)

Note: See TracTickets for help on using tickets.