Opened 9 years ago
Last modified 9 years ago
#5361 new enhancement
Allow continuing where left off in csv for batchaddmedia
Reported by: | Daniel Krol | Owned by: | |
---|---|---|---|
Priority: | minor | Milestone: | |
Component: | programming | Keywords: | |
Cc: | Parent Tickets: |
Description
My personal use case for csv is such that I'd rather have it break, fix it, and then continue where I left off, than have it continue if it fails, because I'd like to preserve the order of the files. I was hoping to add some features for this workflow. So here's the changes I would make:
0) Make sure batchaddmedia
submits the files in the csv in order (right now it loops over a dict!)
1) Add a --start
option to specify which entry to start on
2) Add a --stop-on-failure
option.
3) In case of exception, output a helpful bit of text that says to add --start 20
(for instance) to continue where you left off.
Change History (3)
comment:1 by , 9 years ago
comment:2 by , 9 years ago
For clarity, the branch for this issue is continue-batchaddmedia
, however keep in mind that it includes a commit from #5360 that is as of this writing not merged. As such it should probably be rebased if there are changes before it's merged.
comment:3 by , 9 years ago
This probably looks fairly feasible; I need to re-read before merging though.
I decided to stick 0) into ticket #5360.
So here's a WIP:
https://github.com/orblivion/mediagoblin/commit/75d993805b2d3105b8c02d2667baa8563b2dbffb
(This is on top of the change in #5360, mainly because I wanted to expand the existing test case I wrote there)
The problem here is that I don't think I can print out a guaranteed correct stack trace for the catch-all exception case at the end of the function (which includes transcoding errors for the non-
--celery
case), and then print out advice on--start xx
_after_ that. I triedtraceback.print_exc()
but it seems to print something different thanraise
does. So I'm resorting to giving the--start xx
advice before trying the submission rather than after. But that's not great before a long stack trace. It will likely be missed if the user isn't looking for it.Any better ideas? Otherwise I'll just wrap this up and put it in, and maybe somebody else can improve it later.