Tool to create a tarball of the current environment (database, media files, etc)

[Christopher Allan Webber]

It would be great if we had a tool that could do something like:

::

    ./bin/gmg environment_tarball mediagoblin.ini tarball_name.tar.gz

That contained something like:

::

    ./mediagoblin_db.json
    ./public_store/ # all public store files kept here
    ./queue_store/ # all queue store files kept here

[Christopher Allan Webber]

I guess it would also be nice to have:

::

    ./bin/gmg load_environment_tarball mediagoblin.ini tarball_name.tar.gz

which does the opposite: it sucks up all the info from that tarball
and installs it in place. Having these tools would make developing
a lot easier I think :)

[Aaron Williamson]

Right now, the easiest way to back up the databases is with
mongodump, which exports bson (binary json). To export all of the
relevant databases:

::

    mongodump --db mediagoblin --out /path/to/dump/to/
    mongodump --db kombu_default --out /path/to/dump/to/

These can then be restored using ``mongorestore``

To get ASCII json instead of bson, you have to use ``mongoexport``.
``mongoexport`` requires you to specify a collection (table) to
export -- you can't just export an entire mongo db as json (see the
relevant
`feature request <https://jira.mongodb.org/browse/SERVER-201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel>`_).
Example usage (export the "media\_entries" collection from the
"mediagoblin" database:

::

    mongoexport -d mediagoblin -c media_entries

[Will Kahn-Greene]

Grabbing this one to implement whatever is missing for allowing a
hacker-type person to backup and restore a single user's stuff from
the command line.

[Elrond]

This is coding, or maybe documentation later. Definitely not
infrastructure.

[Will Kahn-Greene]

I was thinking this was infrastructure because it's part of the
administration scripts and build environment stuff. I guess I don't
understand the categories.

[Caleb Davis]

I would like to use the 'import' side of this feature to load test
data during index unit testing, so I added that this feature blocks
`#432 </issues/432>`_. My thinking is, if we had good test data we
could assert limits on how long queries should take using
explain().

[Will Kahn-Greene]

I can't get to this for 0.0.4, so I want to push it off to 0.0.5.

[joar]

Muhahah

[joar]

Merge request at
`https://gitorious.org/mediagoblin/mediagoblin/merge\_requests/17 <https://gitorious.org/mediagoblin/mediagoblin/merge_requests/17>`_

[Christopher Allan Webber]

Made a few changes. You can now not have to provide the cache\_dir
to the command line, and instead of using /tmp/mediagoblin/ it'll
make a new temporary directory using the tempfile module.

Tried it locally... worked great! :) Merged and pushed.

Thanks so much Joar... this is an awesome and much needed feature!

[Will Kahn-Greene]

The original url for this bug was ​http://bugs.foocorp.net/issues/298 .
Relations:
#142: blocks, #55: blocks