I recently encountered an exception in Python 2, using subprocess on Windows. If the process name or any of the arguments contain non-ascii/Unicode characters, an error like the following is raised: UnicodeEncodeError: 'ascii' codec can't encode character u'\xc5' in position 5: ordinal not in range(128).
The issue was opened several years ago, on the official bug tracker, and fixed in Python 3 but not Python 2. It looks like the ultimate source of the issue is the use internally of CreateProcessA instead of CreateProcessW. (Some of the workarounds on this page, like specifying a code page, aren't full solutions since they'll still fail for most unicode characters).
Here's my workaround. It uses winprocess.py, which is MIT Licensed and available here as well as many other places on GitHub.
def runWithoutWaitUnicode(listArgs):
# in Windows, non-ascii characters cause subprocess.Popen to fail.
# https://bugs.python.org/issue1759845
import subprocess
if sys.platform != 'win32' or all(isinstance(arg, str) for arg in listArgs):
p = subprocess.Popen(listArgs, shell=False)
return p.pid
else:
import winprocess
import types
if isinstance(listArgs, types.StringTypes):
combinedArgs = listArgs
else:
combinedArgs = subprocess.list2cmdline(listArgs)
combinedArgs = unicode(combinedArgs)
executable = None
close_fds = False
creationflags = 0
env = None
cwd = None
startupinfo = winprocess.STARTUPINFO()
handle, ht, pid, tid = winprocess.CreateProcess(executable, combinedArgs,
None, None,
int(not close_fds),
creationflags,
env,
cwd,
startupinfo)
ht.Close()
handle.Close()
return pid
This only accounts for CreateProcess, and not ShellExecute (i.e. passing shell=True to subprocess). However, you can use the "start" command as a way to ShellExecute. For example, in Windows, to open a file with its default program, you can use runWithoutWaitUnicode([u'cmd', u'/c', u'start', filePath]). (As a side note, if a directory name is passed, the directory will be opened in Explorer UI, which can be useful).
For tests, including tests that specifically exercise the Unicode case that was previously broken, see files.py and tests.py on my GitHub page here.