Non-ASCII characters in the slug make the media page inaccessible (Server Error).

[Aleksej]

E.g. at ​https://gobblin.se/u/avrs/, the image whose title begins with “6.10 Со стороны левого бо…” has a Cyrillic slug, so its page cannot be opened.

[joar]

Could you please supply a stacktrace?

translitcodec lets *some* non-ascii characters through. We should either strip away all non-ascii characters when generating the slug or make Routes accept non-ascii characters.

[Aleksej]

The slug is “wфw”.

Error - <type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode character u'\u0444' in position 1: ordinal not in range(128)
URL: ​http://127.0.0.1:6543/u/qqc/m/w%D1%84w/
File '/opt/dev/gmg/mediagoblin/local/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in call

app_iter = self.application(environ, sr_checker)

File '/opt/dev/gmg/mediagoblin/local/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/urlmap.py', line 203 in call

return app(environ, start_response)

File '/opt/dev/gmg/mediagoblin/local/lib/python2.7/site-packages/Beaker-1.6.3-py2.7.egg/beaker/middleware.py', line 155 in call

return self.wrap_app(environ, session_start_response)

File '/opt/dev/gmg/mediagoblin/mediagoblin/app.py', line 221 in call

return self.call_backend(environ, start_response)

File '/opt/dev/gmg/mediagoblin/mediagoblin/app.py', line 138 in call_backend

route_match = self.routing.match(path_info)

File '/opt/dev/gmg/mediagoblin/local/lib/python2.7/site-packages/Routes-1.13-py2.7.egg/routes/mapper.py', line 669 in match

result = self._match(url, environ)

File '/opt/dev/gmg/mediagoblin/local/lib/python2.7/site-packages/Routes-1.13-py2.7.egg/routes/mapper.py', line 646 in _match

domain_match)

File '/opt/dev/gmg/mediagoblin/local/lib/python2.7/site-packages/Routes-1.13-py2.7.egg/routes/route.py', line 560 in match

val = val and val.decode(self.encoding, self.decode_errors)

File '/opt/dev/gmg/mediagoblin/lib/python2.7/encodings/utf_8.py', line 16 in decode

return codecs.utf_8_decode(input, errors, True)

UnicodeEncodeError: 'ascii' codec can't encode character u'\u0444' in position 1: ordinal not in range(128)

CGI Variables

---

CONTENT_LENGTH: '0'
HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
HTTP_ACCEPT_ENCODING: 'gzip, deflate'
HTTP_ACCEPT_LANGUAGE: 'ru-ru,ru;q=0.8,en-us;q=0.5,en;q=0.3'
HTTP_CONNECTION: 'keep-alive'
HTTP_COOKIE: 'mediagoblin_csrftoken=4692806606904755232; mediagoblin_csrftoken=4692806606904755232; mediagoblin=d17702e90b574360a1135621115c29b8'
HTTP_DNT: '1'
HTTP_HOST: '127.0.0.1:6543'
HTTP_REFERER: '​http://127.0.0.1:6543/u/qqc/'
HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/17.0 Firefox/17.0'
PATH_INFO: '/u/qqc/m/w\xd1\x84w/'
REMOTE_ADDR: '127.0.0.1'
REQUEST_METHOD: 'GET'
SERVER_NAME: '127.0.0.1'
SERVER_PORT: '6543'
SERVER_PROTOCOL: 'HTTP/1.1'

WSGI Variables

---

application: {(None, '/mgoblin_static'): <StaticURLParser '/opt/dev/gmg/mediagoblin/mediagoblin/static'>, (None, '/mgoblin_media'): <StaticURLParser '/opt/dev/gmg/mediagoblin/user_dev/media/public'>, (None, ): <beaker.middleware.SessionMiddleware object at 0x426f850>, (None, '/theme_static'): <StaticURLParser '/opt/dev/gmg/mediagoblin/user_dev/theme_static'>}
beaker.get_session: <bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x426f850>>
beaker.session: {'_accessed_time': 1350152884.63729, 'user_id': u'2', 'messages': [], '_creation_time': 1350152804.395771}
paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x21914d0>
paste.throw_errors: True
werkzeug.request: <Request '​http://127.0.0.1:6543/u/qqc/m/w%D1%84w/' [GET]>
wsgi process: 'Multithreaded'

---

[joar]

This issue will probably be fixed with the transition from Routes to werkzeug.routing, this works on werkzeug.routing: ​http://joar.pagekite.me/u/joar/m/foeoe%C4%A7/

[joar]

This has been fixed since the move to werkzeug.routing <3