howto-pyporting.pdf

Porting Python 2 Code to Python 3

Release2.7.6

Guido van Rossum

Fred L. Drake, Jr., editor

March 17, 2014

Python Software Foundation

Email:docs@python.org

Contents

The Short Version

Before You Begin

Writing Source-Compatible Python 2/3 Code

3.1

Projects to Consider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

iii

3.2

Tips & Tricks

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

iii

Support Python 2.7

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

iii

Try to Support Python 2.6 and Newer Only

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

iii

Supporting Python 2.5 and Newer Only

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Handle Common “Gotchas”

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.3

Eliminate -3 Warnings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

viii

Alternative Approaches

viii

4.1

Supporting Only Python 3 Going Forward From Python 2 Code

. . . . . . . . . . . . . . . . . .

4.2

Backporting Python 3 code to Python 2

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Other Resources

author Brett Cannon

Abstract

With Python 3 being the future of Python while Python 2 is still in active use, it is good to have your project

available for both major releases of Python. This guide is meant to help you ﬁgure out how best to support

both Python 2 & 3 simultaneously.

If you are looking to port an extension module instead of pure Python code, please seecporting-howto.

If you would like to read one core Python developer’s take on why Python 3 came into existence, you can

read Nick Coghlan’s Python 3 Q & A .

If you prefer to read a (free) book on porting a project to Python 3, consider reading Porting to Python 3 by

Lennart Regebro which should cover much of what is discussed in this HOWTO.

For help with porting, you can email the python-porting mailing list with questions.

1 The Short Version

• Decide what’s the oldest version of Python 2 you want to support (if at all)

• Make sure you have a thorough test suite and use continuous integration testing to make sure you stay

compatible with the versions of Python you care about

• If you have dependencies, check their Python 3 status using caniusepython3 ( command-line tool , web app )

With that done, your options are:

• If you are dropping Python 2 support, use 2to3 to port to Python 3

• If you are keeping Python 2 support, then start writing Python 2/3-compatible code starting TODAY

– If you have dependencies that have not been ported, reach out to them to port their project while

working to make your code compatible with Python 3 so you’re ready when your dependencies are all

ported

– If all your dependencies have been ported (or you have none), go ahead and port to Python 3

• If you are creating a new project that wants to have 2/3 compatibility, code in Python 3 and then backport

to Python 2

2 Before You Begin

If your project is on the Cheeseshop / PyPI , make sure it has the proper trove classiﬁers to signify what versions

of Python it currently supports. At minimum you should specify the major version(s), e.g. Programming

Language::Python::2 if your project currently only supports Python 2. It is preferrable that you

be as speciﬁc as possible by listing every major/minor version of Python that you support, e.g. if your project

supports Python 2.6 and 2.7, then you want the classiﬁers of:

ProgrammingLanguage::Python::2

ProgrammingLanguage::Python::2.6

ProgrammingLanguage::Python::2.7

Once your project supports Python 3 you will want to go back and add the appropriate classiﬁers for Python 3 as

well. This is important as setting the ProgrammingLanguage::Python::3 classiﬁer will lead

to your project being listed under the Python 3 Packages section of PyPI.

Make sure you have a robust test suite. You need to make sure everything continues to work, just like when you

support a new minor/feature release of Python. This means making sure your test suite is thorough and is ported

properly between Python 2 & 3 (consider using coverage to measure that you have effective test coverage). You

will also most likely want to use something like tox to automate testing between all of your supported versions of

Python. You will also want to port your tests ﬁrst so that you can make sure that you detect breakage during the

transition. Tests also tend to be simpler than the code they are testing so it gives you an idea of how easy it can be

to port code.

Drop support for older Python versions if possible. Python 2.5 introduced a lot of useful syntax and libraries

which have become idiomatic in Python 3. Python 2.6 introduced future statements which makes compatibility

much easier if you are going from Python 2 to 3. Python 2.7 continues the trend in the stdlib. Choose the newest

version of Python which you believe can be your minimum support version and work from there.

Target the newest version of Python 3 that you can. Beyond just the usual bugﬁxes, compatibility has continued to

improve between Python 2 and 3 as time has passed. E.g. Python 3.3 added back the u preﬁx for strings, making

source-compatible Python code easier to write.

3 Writing Source-Compatible Python 2/3 Code

Over the years the Python community has discovered that the easiest way to support both Python 2 and 3 in parallel

is to write Python code that works in either version. While this might sound counter-intuitive at ﬁrst, it actually

is not difﬁcult and typically only requires following some select (non-idiomatic) practices and using some key

projects to help make bridging between Python 2 and 3 easier.

3.1 Projects to Consider

The lowest level library for supporting Python 2 & 3 simultaneously is six . Reading through its documentation

will give you an idea of where exactly the Python language changed between versions 2 & 3 and thus what you

will want the library to help you continue to support.

To help automate porting your code over to using six, you can use modernize . This project will attempt to rewrite

your code to be as modern as possible while using six to smooth out any differences between Python 2 & 3.

If you want to write your compatible code to feel more like Python 3 there is the future project. It tries to provide

backports of objects from Python 3 so that you can use them from Python 2-compatible code, e.g. replacing the

bytes type from Python 2 with the one from Python 3. It also provides a translation script like modernize (its

translation code is actually partially based on it) to help start working with a pre-existing code base. It is also

unique in that its translation script will also port Python 3 code backwards as well as Python 2 code forwards.

3.2 Tips & Tricks

To help with writing source-compatible code using one of the projects mentioned in Projects to Consider , consider

following the below suggestions. Some of them are handled by the suggested projects, so if you do use one of

them then read their documentation ﬁrst to see which suggestions below will taken care of for you.

Support Python 2.7

As a ﬁrst step, make sure that your project is compatible with Python 2.7 . This is just good to do as Python 2.7

is the last release of Python 2 and thus will be used for a rather long time. It also allows for use of the -3 ﬂag to

Python to help discover places in your code where compatibility might be an issue (the -3 ﬂag is in Python 2.6

but Python 2.7 adds more warnings).

Try to Support Python 2.6 and Newer Only

While not possible for all projects, if you can support Python 2.6 and newer only, your life will be much easier.

Various future statements, stdlib additions, etc. exist only in Python 2.6 and later which greatly assist in supporting

Python 3. But if you project must keep support for Python 2.5 then it is still possible to simultaneously support

Python 3.

Below are the beneﬁts you gain if you only have to support Python 2.6 and newer. Some of these options are

personal choice while others are strongly recommended (the ones that are more for personal choice are labeled as

such). If you continue to support older versions of Python then you at least need to watch out for situations that

these solutions ﬁx and handle them appropriately (which is where library help from e.g. six comes in handy).

from__future__importprint_function

It will not only get you used to typing print() as a function instead of a statement, but it will also give you the

various beneﬁts the function has over the Python 2 statement ( six provides a function if you support Python 2.5 or

older).

from__future__importunicode_literals

If you choose to use this future statement then all string literals in Python 2 will be assumed to be Unicode (as is

already the case in Python 3). If you choose not to use this future statement then you should mark all of your text

strings with a u preﬁx and only support Python 3.3 or newer. But you are strongly advised to do one or the other

( six provides a function in case you don’t want to use the future statement and you want to support Python 3.2 or

older).

Bytes/string literals

This is a very important one. Preﬁx Python 2 strings that are meant to contain bytes with a b preﬁx to very clearly

delineate what is and is not a Python 3 text string ( six provides a function to use for Python 2.5 compatibility).

This point cannot be stressed enough: make sure you know what all of your string literals in Python 2 are meant

to be in Python 3. Any string literal that should be treated as bytes should have the b preﬁx. Any string literal

that should be Unicode/text in Python 2 should either have the u literal (supported, but ignored, in Python 3.3 and

later) or you should have from__future__importunicode_literals at the top of the ﬁle. But the

key point is you should know how Python 3 will treat every one one of your string literals and you should mark

them as appropriate.

There are some differences between byte literals in Python 2 and those in Python 3 thanks to the bytes type just

being an alias to str in Python 2. See the Handle Common “Gotchas” section for what to watch out for.

from__future__importabsolute_import

Discussed in more detail below, but you should use this future statement to prevent yourself from accidentally

using implicit relative imports.

Supporting Python 2.5 and Newer Only

If you are supporting Python 2.5 and newer there are still some features of Python that you can utilize.

from__future__importabsolute_import

Implicit relative imports (e.g., importing spam.bacon from within spam.eggs with the statement import

bacon ) do not work in Python 3. This future statement moves away from that and allows the use of explicit

relative imports (e.g., from.importbacon ).

In Python 2.5 you must use the __future__ statement to get to use explicit relative imports and prevent implicit

ones. In Python 2.6 explicit relative imports are available without the statement, but you still want the __fu-

ture__ statement to prevent implicit relative imports. In Python 2.7 the __future__ statement is not needed. In

other words, unless you are only supporting Python 2.7 or a version earlier than Python 2.5, use this __future__

statement.

Mark all Unicode strings with a u preﬁx

While Python 2.6 has a __future__ statement to automatically cause Python 2 to treat all string literals as

Unicode, Python 2.5 does not have that shortcut. This means you should go through and mark all string literals

with a u preﬁx to turn them explicitly into text strings where appropriate and only support Python 3.3 or newer.

Otherwise use a project like six which provides a function to pass all text string literals through.

Capturing the Currently Raised Exception

In Python 2.5 and earlier the syntax to access the current exception is:

try :

raise Exception ()

except Exception ,exc:

#Currentexceptionis’exc’.

pass

This syntax changed in Python 3 (and backported to Python 2.6 and later) to:

try :

raise Exception ()

except Exception as exc:

#Currentexceptionis’exc’.

#InPython3,’exc’isrestrictedtotheblock;inPython2.6/2.7itwill"leak".

pass

Because of this syntax change you must change how you capture the current exception in Python 2.5 and earlier

to:

try :

raise Exception ()

except Exception :

import sys

exc = sys . exc_info()[ 1 ]

#Currentexceptionis’exc’.

pass

You can get more information about the raised exception from sys.exc_info() than simply the current ex-

ception instance, but you most likely don’t need it.

Note: In Python 3, the traceback is attached to the exception instance through the __traceback__ attribute.

If the instance is saved in a local variable that persists outside of the except block, the traceback will create

a reference cycle with the current frame and its dictionary of local variables.

This will delay reclaiming dead

resources until the next cyclicgarbagecollectionpass.

In Python 2, this problem only occurs if you save the traceback itself (e.g. the third element of the tuple returned

by sys.exc_info() ) in a variable.

Handle Common “Gotchas”

These are things to watch out for no matter what version of Python 2 you are supporting which are not syntactic

considerations.

from__future__importdivision

While the exact same outcome can be had by using the -Qnew argument to Python, using this future statement

lifts the requirement that your users use the ﬂag to get the expected behavior of division in Python 3 (e.g., 1/2

==0.5;1//2==0 ).

Specify when opening a ﬁle as binary

Unless you have been working on Windows, there is a chance you have not always bothered to add the b mode

when opening a binary ﬁle (e.g., rb for binary reading). Under Python 3, binary ﬁles and text ﬁles are clearly

distinct and mutually incompatible; see the io module for details. Therefore, you must make a decision of

whether a ﬁle will be used for binary access (allowing to read and/or write bytes data) or text access (allowing to

read and/or write unicode data).

Text ﬁles

Text ﬁles created using open() under Python 2 return byte strings, while under Python 3 they return unicode

strings. Depending on your porting strategy, this can be an issue.

If you want text ﬁles to return unicode strings in Python 2, you have two possibilities:

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika: