Opened 8 years ago

Last modified 8 years ago

#5361 new enhancement

Allow continuing where left off in csv for batchaddmedia

Reported by: Daniel Krol Owned by:
Priority: minor Milestone:
Component: programming Keywords:
Cc: Parent Tickets:

Description

My personal use case for csv is such that I'd rather have it break, fix it, and then continue where I left off, than have it continue if it fails, because I'd like to preserve the order of the files. I was hoping to add some features for this workflow. So here's the changes I would make:

0) Make sure batchaddmedia submits the files in the csv in order (right now it loops over a dict!)
1) Add a --start option to specify which entry to start on
2) Add a --stop-on-failure option.
3) In case of exception, output a helpful bit of text that says to add --start 20 (for instance) to continue where you left off.

Change History (3)

comment:1 by Daniel Krol, 8 years ago

I decided to stick 0) into ticket #5360.

So here's a WIP:

https://github.com/orblivion/mediagoblin/commit/75d993805b2d3105b8c02d2667baa8563b2dbffb

(This is on top of the change in #5360, mainly because I wanted to expand the existing test case I wrote there)

The problem here is that I don't think I can print out a guaranteed correct stack trace for the catch-all exception case at the end of the function (which includes transcoding errors for the non---celery case), and then print out advice on --start xx _after_ that. I tried traceback.print_exc() but it seems to print something different than raise does. So I'm resorting to giving the --start xx advice before trying the submission rather than after. But that's not great before a long stack trace. It will likely be missed if the user isn't looking for it.

Any better ideas? Otherwise I'll just wrap this up and put it in, and maybe somebody else can improve it later.

comment:2 by Daniel Krol, 8 years ago

For clarity, the branch for this issue is continue-batchaddmedia, however keep in mind that it includes a commit from #5360 that is as of this writing not merged. As such it should probably be rebased if there are changes before it's merged.

comment:3 by Christopher Allan Webber, 8 years ago

This probably looks fairly feasible; I need to re-read before merging though.

Note: See TracTickets for help on using tickets.