Opened 3 years ago

Closed 3 years ago

#1095 closed defect (fixed)

Smarter extlib installs

Reported by: Christopher Allan Webber Owned by:
Priority: major Milestone: 0.8.0
Component: programming Keywords:
Cc: Parent Tickets:

Description

Unfortunately, we have a last minute yak to shave for 0.8.0: if we want to fix #918, we should do it right, and that means no more committing (often minified or otherwise compiled) javascript libraries into our git tree.

But how to do that? System packagers can take advantage of other system packages of javascript and the like (though admittedly we might not make this easy at present). In the meanwhile, if we want to make development easy for contributors and *not* have them battle about 5 language-specific package managers that may or may not cover all our needs to fill out extlib, it would be nice if there was a way to just script out "fetch all this stuff and unpack it here"

I've been trying to model out what the data needs of such a file would look like, and this looks maybe about right-ish:

EXTLIBS = [
    Extlib(
        "lato",
        verify=check_md5("ab516ef108"),
        install=fetch_and_unpack(
            "http://example.org/lato-1.7.tar.gz",
            move_from="lato-1.7"),
        # complete and corresponding source
        c_and_cs=fetch_and_unpack("http://example.org/lato-source.git")
    ),
    Extlib(
        "pdf.js",
        verify=check_md5("adc95ed383ec00cf4342bf4dd343e9bb"),
        install=fetch_and_unpack(
            "http://example.org/pdf.js-8.7.tar.gz",
            move_from="pdfjs-8.7"),
        # complete and corresponding source
        c_and_cs=print_instructions("git clone git://gitpit.example/pdfjs/")
    )
]

A script would still have to be written and carried alongside it. It might be nice to give it a more lax license than the rest of our code but I am somewhat loose about that.

Subtickets

#918: video.js recommending Adobe Flash and Chrome in some browsersclosed

Attachments (1)

better_extlib.txt (11.2 KB) - added by Christopher Allan Webber 3 years ago.
A conversation with Kuno / warp about a better extlib solution

Download all attachments as: .zip

Change History (5)

Changed 3 years ago by Christopher Allan Webber

Attachment: better_extlib.txt added

A conversation with Kuno / warp about a better extlib solution

comment:1 Changed 3 years ago by warp

I have some further thoughts and quick experiments in an extlib-experiments branch here:

https://github.com/warpr/mediagoblin/blob/extlib-experiments/extlib/notes.txt

I later noticed that for some dependencies the not-including-the-library-in-the-source-tree thing is solved by using git submodules, how feasable is it to just do that for everything else in extlib?

comment:2 Changed 3 years ago by Asheesh Laroia

Owner: set to Asheesh Laroia
Status: newin_progress

Chris and I are considering doing something like:

  • Make npm be a dependency for development, but
  • Distribute a tarball of the dependencies with the release, so "end users" (people installing MediaGoblin on their own systems) don't have to know we use npm, and
  • Also distribute a tarball without the dependencies embedded to satisfy nice people in Debian who want a pristine tarball of just MediaGoblin so they can fetch their own dependencies.

comment:3 Changed 3 years ago by Christopher Allan Webber

So, paulproteus started a branch related to this, and I've been extending it and pushing up to my personal repo. I think the code is now approximately good enough for merge (??):

https://gitorious.org/mediagoblin/cwebbers-mediagoblin/commits/npm-or-bower

It needs some testing. I've verified that most things work. I need to check that the workflow works right for users. If anyone wants to volunteer... :)

comment:4 Changed 3 years ago by Christopher Allan Webber

Owner: Asheesh Laroia deleted
Resolution: fixed
Status: in_progressclosed

Okay, I've tested things hopefully good enough! ;) This is now merged into git master.

Note: See TracTickets for help on using tickets.