Opened 13 years ago
Last modified 8 years ago
#212 review defect
Use werkzeug as a caching abstraction layer (was: «use beaker»)
Reported by: | Caleb Davis | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | |
Component: | programming | Keywords: | |
Cc: | Parent Tickets: |
Description (last modified by )
There are a few bugs circulating around that block on actually having an interface to a formal cache. This bug is to build this cache interface. Then we can use it to do stuff!
Change History (31)
comment:2 by , 13 years ago
[https://gitorious.org/\ :sub:`cwebber/mediagoblin/cwebbers-mediagoblin/commits/i507\_beaker\_cache](https://gitorious.org/`\ cwebber/mediagoblin/cwebbers-mediagoblin/commits/i507\_beaker\_cache) Added basic functionality. No use and no tests yet, though.
comment:3 by , 13 years ago
I'm also curious what Elrond thinks about me using [beaker.cache] as the config section, considering he used [storage:foo] pattern for storage stuff...
comment:4 by , 13 years ago
Status: | New → In Progress |
---|
1. I already put that on irc a while back: I think, I like "``beaker.cache``". dot for fixed things. "``beaker``" and "``cache``" are fixed strings, no variations needed. While the colon in "``storage:FOO``" splits a fixed "``storage``" from a not so fixed string. storage\_from\_config() can be called with "FOO" and will happily create a StorageInterface. So the part after the colon is dynamic. 2. Instead of ``(u'cache.%s' % key, value)`` I would have used ``(u'cache.' + key, value)`` (it's shorter and stops a confusion where the right argument to ``%`` ends). Do we even need the ``u``? Does parse\_cache\_config\_options expect unicode strings and converts them to non unicode keyword-args? 3. If this does not break anything, I'd vote for merging it now (despite missing tests) 4. Leave this bug open for writing tests!
comment:5 by , 13 years ago
Milestone: | 0.0.5 → 0.1.0 |
---|
comment:5 by , 13 years ago
Status: | In Progress → Closed |
---|
Added a test and merged. We might want a way to disable during tests though? Not sure. If we do want that, not too hard to add. Closing this though.
comment:6 by , 13 years ago
Status: | Closed → In Progress |
---|
Re-opening. At least one user (Larissa) has reported errors with Beaker initializing on OS X. I'd like to figure out why that is. I think I generally know.
comment:7 by , 13 years ago
Milestone: | 0.1.0 → 0.2.0 |
---|
comment:8 by , 13 years ago
Milestone: | 0.2.0 → 0.2.1 |
---|
comment:7 by , 13 years ago
The original url for this bug was http://bugs.foocorp.net/issues/507 .
Relations:
#126: related, #110: related, #183: related, #164: blocks
comment:8 by , 13 years ago
Unless anyone says what's up on OSX, we should just turn this on and start using it.
I guess the main thing to ask is two things:
- Beaker does caching in a sort of "bucket" system, where you have entire sections of stuff dedicated to being cached.
- I guess a good question is, "what could really use some caching right now?" Possibly the homepage would be a good first place?
Otherwise, I'm not sure why we would need to delay putting this to use and closing it out.
comment:9 by , 13 years ago
Milestone: | 0.2.1 → 0.2.2 |
---|
We should start to use beaker (or whatever) to cache the markdown conversion.
This is relevant, as we're now converting markdown on the fly.
comment:10 by , 13 years ago
Description: | modified (diff) |
---|---|
Milestone: | 0.3.0 |
We're not going to get this done for 0.3.0 but we need to finish it soon. It's pretty close as-is?
comment:11 by , 12 years ago
Priority: | minor → major |
---|
Bumping to major. It's pretty close, we should get it wrapped up.
comment:12 by , 12 years ago
So this is old... it was pretty close to being done, but I think we should pivot (erk) and switch to using Werkzeug's cashing layer: http://werkzeug.pocoo.org/docs/contrib/cache/
comment:13 by , 12 years ago
Milestone: | → 0.3.3 |
---|---|
Status: | accepted → assigned |
It seems like getting this in 0.3.3 wouldn't be too hard. I'm assigning it to myself.
comment:14 by , 12 years ago
Milestone: | 0.3.3 → 0.3.4 |
---|
Not happening for 0.3.3 anyway...
I would like to see it happen soon though.
comment:15 by , 12 years ago
If someone wants to claim this they can, but leaving claimed to me for now.
comment:16 by , 12 years ago
Keywords: | sprint added |
---|---|
Owner: | removed |
Summary: | Use beaker as a caching abstraction layer → Use beaker as a caching abstraction layer (updated: use werkzeug) |
Removing claim from myself
comment:17 by , 11 years ago
Status: | assigned → accepted |
---|
comment:18 by , 11 years ago
Milestone: | 0.4.0 → 0.4.1 |
---|
I doubt we get this done for 0.4.0, really.
Proof me wrong!
comment:19 by , 11 years ago
Summary: | Use beaker as a caching abstraction layer (updated: use werkzeug) → Use werkzeug as a caching abstraction layer (was: «use beaker») |
---|
comment:21 by , 11 years ago
Milestone: | 0.5.0 → 0.6.0 |
---|
comment:22 by , 11 years ago
Basically, there's a caching system here:
Seems there's a few things to consider:
- How to attach the cache object to the application?
- How to make it configurable which cache system to use, and under which settings?
- Which cache system should we turn on by default, if any? Maybe the null one actually, so people explicitly have to set up memcached if they want it?
- What would the API be like for interacting with it? Might be useful to look at what Flask people do.
comment:23 by , 11 years ago
Milestone: | 0.6.0 → 0.7.0 |
---|
comment:24 by , 10 years ago
Milestone: | 0.7.0 → 0.8.0 |
---|
comment:25 by , 10 years ago
I took a look into this and whether we should maybe use dogpile.cache instead. As far as I can tell, dogpile doesn't provide a huge gain.
It would be really nice to have this for 0.8.0, but it's a bit lower priority than some others.
As to answer some of my previous questions:
How to attach the cache object to the application?
I think the pattern that currently exists in master is right... the cache system generally can be attached to MediaGoblinApp during the __init__()
method, and any specific request-level cache objects can be set up either during the _gen_context()
method (if general enough) or the _request_only_gen_context()
method, if we find that's for requests only. (I suspect the former will be best, though.)
How to make it configurable which cache system to use, and under which settings?
My suggestion would be to use a hook_handle()
method and let a plugin determine what the settings are. It's up to the plugin to determine what the config parameters are. Of course, we should try to provide all the base ones and use a consistent pattern to the setting parameters.
Which cache system should we turn on by default, if any? Maybe the null one actually, so people explicitly have to set up memcached if they want it?
Yes, by default, NullCache can be used if nothing is returned by hook_handle().
What would the API be like for interacting with it? Might be useful to look at what Flask people do.
No answer on this one yet... maybe worth looking at what Flask-Cache does.
comment:26 by , 10 years ago
Keywords: | sprint removed |
---|---|
Milestone: | 0.8.0 |
comment:27 by , 8 years ago
Status: | accepted → review |
---|
To answer Chris's last question:
What would the API be like for interacting with it? Might be useful to look at what Flask people do.
The Flask docs recommend the Werkzeug caching that is already being proposed. API would be something like:
>>> app.cache.set('key', 'value') >>> app.cache.get('key') value
I've done a prototype integration here, feedback would be appreciated. Happy to add deployment docs if I'm on the right track:
https://gitlab.com/BenSturmfels/mediagoblin/compare/master...212-werkzeug-cache
comment:28 by , 8 years ago
Sorry, I'd skimmed over Chris's suggestion about hook_handle()
. Could you give an example of how a plugin might want to change the caching configuration? Would it be to tweak a cache setting such as default_timeout
, or would it be providing an alternative cache backend or something else?