Opened 8 years ago

Closed 3 years ago

#5449 closed defect (cant-reproduce)

UnicodeEncodeError with non-ASCII characters in comment emails

Reported by: ayleph Owned by:
Priority: minor Milestone:
Component: programming Keywords: ascii, unicode, comment, notification, email
Cc: Parent Tickets:

Description

Adding a comment with a non-ASCII character results in a server error when sending a notification email. Running py2 with close-to-latest master code. Eg, trying to add something like "テストコメントです!" results in the error below.

2016-03-28 20:45:19,114 INFO    [mediagoblin.notifications.task] Sending notification email about <Notification #7: <LocalUser #13 verified user "ayleph">: None (unseen)>
===== Email =====
From address: mediagoblin@goblinrefuge.com
To addresses: ayleph@thisshitistemp.com
Subject: Goblin Refuge Beta - machalus commented on your post
-- Body: --
Error - <type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode characters in position 130-139: ordinal not in range(128)
URL: http://pumpdev.goblinrefuge.com/u/ayleph/m/43/comment/add/
File '/path/to/mediagoblin/lib/python2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in __call__
  app_iter = self.application(environ, sr_checker)
File '/path/to/mediagoblin/mediagoblin/app.py', line 342 in __call__
  return self.call_backend(environ, start_response)
File '/path/to/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.9.6-py2.7.egg/werkzeug/wsgi.py', line 588 in __call__
  return self.app(environ, start_response)
File '/path/to/mediagoblin/mediagoblin/app.py', line 276 in call_backend
  return self._finish_call_backend(request, environ, start_response)
File '/path/to/mediagoblin/mediagoblin/app.py', line 318 in _finish_call_backend
  response = controller(request)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 281 in wrapper
  return controller(request, media=media, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 47 in wrapper
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 74 in new_controller_func
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/decorators.py', line 103 in wrapper
  return controller(request, *args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/user_pages/views.py', line 210 in media_post_comment
  trigger_notification(link, media, request)
File '/path/to/mediagoblin/mediagoblin/notifications/__init__.py', line 63 in trigger_notification
  email_notification_task.apply_async([cn.id, message])
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.20-py2.7.egg/celery/app/task.py', line 557 in apply_async
  link=link, link_error=link_error, **options)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.20-py2.7.egg/celery/app/task.py', line 755 in apply
  request=request, propagate=throw)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.20-py2.7.egg/celery/app/trace.py', line 355 in eager_trace_task
  uuid, args, kwargs, request)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.20-py2.7.egg/celery/app/trace.py', line 253 in trace_task
  I, R, state, retval = on_error(task_request, exc, uuid)
File '/path/to/mediagoblin/lib/python2.7/site-packages/celery-3.1.20-py2.7.egg/celery/app/trace.py', line 240 in trace_task
  R = retval = fun(*args, **kwargs)
File '/path/to/mediagoblin/mediagoblin/notifications/task.py', line 44 in run
  message['body'])
File '/path/to/mediagoblin/mediagoblin/tools/mail.py', line 140 in send_email
  print(message_body)
File '/path/to/mediagoblin/lib/python2.7/site-packages/PasteScript-1.7.5-py2.7.egg/paste/script/serve.py', line 530 in write
  fileobj.write(text)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 130-139: ordinal not in range(128)

Change History (8)

comment:1 by ayleph, 8 years ago

Priority: criticalminor

Turns out this is only an issue when email_debug_mode = True. Dropping severity drastically.

comment:2 by jsandoval, 8 years ago

Is this ticket still active?. I can't seem to replicate it.

Version 0, edited 8 years ago by jsandoval (next)

comment:3 by ayleph, 8 years ago

Yes, this still occurs for me on v0.9.0 with python2.7.

comment:4 by Dusan Ristic, 8 years ago

Owner: set to Dusan Ristic
Status: newin_progress

comment:5 by Dusan Ristic, 8 years ago

Owner: Dusan Ristic removed
Status: in_progressreview

Please review..
https://github.com/dusan87/mediagoblin/tree/dusan_bug_5449

I think it's caused by encode() method, since we already get encoded Chinese chars which are tried to be encoded again.

comment:6 by Boris Bobrov, 8 years ago

Status: reviewaccepted

Although this looks good, could you please add unit tests? Examples can be found in mediagoblin/tests/test_util.py

comment:7 by Ben Sturmfels, 8 years ago

Hi Dusan,

Thanks very much for spending time with us at the PyConAU sprints!

I'm sure you showed me some tests for this patch at the event. Did you forget to add and commit the test file? Essentially what we need is for this test to demonstrate the existence of the bug before your patch is applied.

Regards,
Ben

in reply to:  3 comment:8 by Ben Sturmfels, 3 years ago

Resolution: cant-reproduce
Status: acceptedclosed

Hi Ayleph,

Replying to ayleph:

Yes, this still occurs for me on v0.9.0 with python2.7.

I've just had a quick look at this old bug report and tried to reproduce it, but it seems like MediaGoblin isn't sending comment notification emails at all. We have a function send_comment_email, but it doesn't appear to be called anywhere in the codebase. I'll create a separate report for that.

Given we're now Python 3-only, there's a good chance this unicode problem is now a non-issue, so I'll close this bug report.

Regards,
Ben

Note: See TracTickets for help on using tickets.