Opened 4 years ago

Last modified 3 years ago

#5449 accepted defect

UnicodeEncodeError with non-ASCII characters in comment emails

Reported by: ayleph Owned by:
Priority: minor Milestone:
Component: programming Keywords: ascii, unicode, comment, notification, email
Cc: Parent Tickets:

Description

Adding a comment with a non-ASCII character results in a server error when sending a notification email. Running py2 with close-to-latest master code. Eg, trying to add something like "テストコメントです!" results in the error below.

2016-03-28 20:45:19,114 INFO    [mediagoblin.notifications.task] Sending notification email about <Notification #7: <LocalUser #13 verified user "ayleph">: None (unseen)>
===== Email =====
From address: mediagoblin@goblinrefuge.com
To addresses: ayleph@thisshitistemp.com
Subject: Goblin Refuge Beta - machalus commented on your post
-- Body: --
Error - <type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode characters in position 130-139: ordinal not in range(128)
URL: http://pumpdev.goblinrefuge.com/u/ayleph/m/43/comment/add/
File '/path/to/mediagoblin/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in __call__
  app_iter = self.application(environ, sr_checker)
File '/path/to/mediagoblin/mediagoblin/app.py', line 342 in __call__
  return self.call_backend(environ, start_response)
File '/path/to/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.9.6-py2.7.egg/werkzeug/wsgi.py', line 588 in __call__
  return self.app(environ, start_response)
File '/path/to/mediagoblin/mediagoblin/app.py', line 276 in call_backend
  return self._finish_call_backend(request, environ, start_response)
File '/path/to/mediagoblin/mediagoblin/app.py', line 318 in _finish_call_backend
  response = controller(request)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 281 in wrapper
  return controller(request, media=media, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 47 in wrapper
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 74 in new_controller_func
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 103 in wrapper
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/user_pages/views.py', line 210 in media_post_comment
  trigger_notification(link, media, request)
File '/path/to/mediagoblin/mediagoblin/notifications/__init__.py', line 63 in trigger_notification
  email_notification_task.apply_async([cn.id, message])
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.20-py2.7.egg/celery/app/task.py', line 557 in apply_async
  link=link, link_error=link_error, **options)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.20-py2.7.egg/celery/app/task.py', line 755 in apply
  request=request, propagate=throw)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.20-py2.7.egg/celery/app/trace.py', line 355 in eager_trace_task
  uuid, args, kwargs, request)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.20-py2.7.egg/celery/app/trace.py', line 253 in trace_task
  I, R, state, retval = on_error(task_request, exc, uuid)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.20-py2.7.egg/celery/app/trace.py', line 240 in trace_task
  R = retval = fun(*args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/notifications/task.py', line 44 in run
  message['body'])
File '/path/to/mediagoblin/mediagoblin/tools/mail.py', line 140 in send_email
  print(message_body)
File '/path/to/mediagoblin/lib/python2.7/site-packages/PasteScript-1.7.5-py2.7.egg/paste/script/serve.py', line 530 in write
  fileobj.write(text)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 130-139: ordinal not in range(128)

Subtickets

Change History (7)

comment:1 Changed 4 years ago by ayleph

Priority: criticalminor

Turns out this is only an issue when email_debug_mode = True. Dropping severity drastically.

comment:2 Changed 4 years ago by jsandoval

Is this ticket still active?. I can't seem to be able to replicate it.

Last edited 4 years ago by jsandoval (previous) (diff)

comment:3 Changed 4 years ago by ayleph

Yes, this still occurs for me on v0.9.0 with python2.7.

comment:4 Changed 3 years ago by Dusan Ristic

Owner: set to Dusan Ristic
Status: newin_progress

comment:5 Changed 3 years ago by Dusan Ristic

Owner: Dusan Ristic deleted
Status: in_progressreview

Please review..
https://github.com/dusan87/mediagoblin/tree/dusan_bug_5449

I think it's caused by encode() method, since we already get encoded Chinese chars which are tried to be encoded again.

comment:6 Changed 3 years ago by Boris Bobrov

Status: reviewaccepted

Although this looks good, could you please add unit tests? Examples can be found in mediagoblin/tests/test_util.py

comment:7 Changed 3 years ago by Ben Sturmfels

Hi Dusan,

Thanks very much for spending time with us at the PyConAU sprints!

I'm sure you showed me some tests for this patch at the event. Did you forget to add and commit the test file? Essentially what we need is for this test to demonstrate the existence of the bug before your patch is applied.

Regards,
Ben

Note: See TracTickets for help on using tickets.