Cannot upload media with non-ASCII characters in file names; such attachments end up typeless.

[Aleksej]

Server Error

URL: ​http://127.0.0.1:6543/submit/
File '…/mediagoblin/local/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in call

app_iter = self.application(environ, sr_checker)

File '…/mediagoblin/local/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/urlmap.py', line 203 in call

return app(environ, start_response)

File '…/mediagoblin/local/lib/python2.7/site-packages/Beaker-1.6.3-py2.7.egg/beaker/middleware.py', line 155 in call

return self.wrap_app(environ, session_start_response)

File '…/mediagoblin/mediagoblin/app.py', line 215 in call

return self.call_backend(environ, start_response)

File '…/mediagoblin/mediagoblin/app.py', line 202 in call_backend

response = controller(request)

File '…/mediagoblin/mediagoblin/decorators.py', line 51 in new_controller_func

return controller(request, *args, kwargs)

File '…/mediagoblin/mediagoblin/submit/views.py', line 65 in submit_start

request.POSTfile)

File '…/mediagoblin/mediagoblin/media_types/init.py', line 52 in sniff_media

if managersniff_handler(media_file, media=media):

File '…/mediagoblin/mediagoblin/media_types/image/processing.py', line 69 in sniff_handler

clean_ext))

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-6: ordinal not in range(128)

[Aleksej]

Attachments like “рисунок.svg” end up named “svg” and without a useful type.

[Duncan]

Created a simple patch for this problem. Werkzeug create safe name was returning an empty string for unicode chars, so I've just generated a uuid if the filename contains any 128+ chars.

​https://gitorious.org/~dunkyp/mediagoblin/dunkyps-mediagoblin

branch 500_non_ascii

[Aleksej]

Upload error is now:

2013-06-06 20:28:51,377 DEBUG [mediagoblin.processing.task] Processing <MediaEntry 17: rusajo>
2013-06-06 20:28:51,382 ERROR [mediagoblin.processing.task] An unhandled exception was raised while processing <MediaEntry 17: rusajo>
2013-06-06 20:28:51,382 WARNING [mediagoblin.processing] No idea what happened here, but it failed: KeyError(u'.thumbnail',)
2013-06-06 20:28:51,524 WARNING [mediagoblin.processing] No idea what happened here, but it failed: KeyError(u'.thumbnail',)
Error - <type 'exceptions.KeyError'>: u'.thumbnail'
URL: ​http://127.0.0.1:6543/submit/
File '/opt/dev/net/gmg/mediagoblin/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in call

app_iter = self.application(environ, sr_checker)

File '/opt/dev/net/gmg/mediagoblin/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/urlmap.py', line 203 in call

return app(environ, start_response)

File '/opt/dev/net/gmg/mediagoblin/mediagoblin/app.py', line 247 in call

return self.call_backend(environ, start_response)

File '/opt/dev/net/gmg/mediagoblin/mediagoblin/app.py', line 224 in call_backend

response = controller(request)

File '/opt/dev/net/gmg/mediagoblin/mediagoblin/decorators.py', line 48 in new_controller_func

return controller(request, *args, kwargs)

File '/opt/dev/net/gmg/mediagoblin/mediagoblin/submit/views.py', line 92 in submit_start

run_process_media(entry, feed_url)

File '/opt/dev/net/gmg/mediagoblin/mediagoblin/submit/lib.py', line 90 in run_process_media

task_id=entry.queued_task_id)

File '/opt/dev/net/gmg/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/app/task/init.py', line 445 in apply_async

return self.apply(args, kwargs, task_id=task_id, options)

File '/opt/dev/net/gmg/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/app/task/init.py', line 601 in apply

request=request, propagate=throw)

File '/opt/dev/net/gmg/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/execute/trace.py', line 248 in eager_trace_task

uuid, args, kwargs, request)

File '/opt/dev/net/gmg/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/execute/trace.py', line 181 in trace_task

R = retval = fun(*args, kwargs)

File '/opt/dev/net/gmg/mediagoblin/mediagoblin/processing/task.py', line 92 in run

entry.media_manager.processor(proc_state)

File '/opt/dev/net/gmg/mediagoblin/mediagoblin/media_types/image/processing.py', line 143 in process_image

conversions_subdir, exif_tags)

File '/opt/dev/net/gmg/mediagoblin/mediagoblin/media_types/image/processing.py', line 95 in resize_tool

exif_tags, conversions_subdir)

File '/opt/dev/net/gmg/mediagoblin/mediagoblin/media_types/image/processing.py', line 70 in resize_image

resized.save(resized_file, quality=configquality)

File '/usr/lib/python2.7/dist-packages/PIL/Image.py', line 1423 in save

raise KeyError(ext) # unknown extension

KeyError: u'.thumbnail'

CGI Variables

---

CONTENT_LENGTH: '7833'
CONTENT_TYPE: 'multipart/form-data; boundary=---------------------------108717406270077165864240165'

[snip]

HTTP_CONNECTION: 'keep-alive'
HTTP_COOKIE: [snip]
HTTP_DNT: '1'
HTTP_HOST: '127.0.0.1:6543'
HTTP_REFERER: '​http://127.0.0.1:6543/submit/'
HTTP_USER_AGENT: '[snip]'
PATH_INFO: '/submit/'
REMOTE_ADDR: '127.0.0.1'
REQUEST_METHOD: 'POST'
SERVER_NAME: '127.0.0.1'
SERVER_PORT: '6543'
SERVER_PROTOCOL: 'HTTP/1.1'

WSGI Variables

---

application: {(None, '/mgoblin_static'): <StaticURLParser '/opt/dev/net/gmg/mediagoblin/mediagoblin/static'>, (None, '/mgoblin_media'): <StaticURLParser '/opt/dev/net/gmg/mediagoblin/user_dev/media/public'>, (None, ): <mediagoblin.app.MediaGoblinApp object at 0x26fa650>, (None, '/plugin_static'): <StaticURLParser '/opt/dev/net/gmg/mediagoblin/user_dev/plugin_static'>, (None, '/theme_static'): <StaticURLParser '/opt/dev/net/gmg/mediagoblin/user_dev/theme_static'>}
paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x4338250>
paste.throw_errors: True
werkzeug.request: <Request '​http://127.0.0.1:6543/submit/' [POST]>
wsgi process: 'Multithreaded'

---

[rodney757]

Okay, so I cherry-picked that commit and added import uuid. This is now merged. Thanks!