10 11-hart installing pythonsoftware
Download as PPT, PDF0 likes664 views
The document discusses the installation of Python software packages, highlighting the methods for both successful and problematic installations across different operating systems. It categorizes Python packages and provides step-by-step instructions for installation from various sources, including PyPI and source files, while addressing common issues related to user privileges and environmental management. The final thoughts emphasize the ease of installing pure Python packages versus the challenges of installing extension packages, advocating for the use of virtual environments for better management.
10 11-hart installing pythonsoftware
- 1. Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. Installing Python Software Packages: The Good, The Bad and the Ugly William E. Hart Sandia National Laboratories wehart@sandia.gov
- 2. Overview This talks focuses on installing on Python packages – Installing Python itself is usually a no-brainer The Good 1. Installing on Windows with an installer executable 2. Installing with Linux application utility 3. Installing a Python package from the PyPI repository 4. Installing a Python package from source The Bad 5. Using a virtual environment to isolate package installations 6. Using an installer executable on Windows with a virtual environment The Ugly 7. Installing a Python extension package from source 8. PyCoinInstall – Managing builds for Python extension packages Slide 2
- 3. About Python Recent releases: – 2.5 Very stable – 2.6 Very stable – 2.7 Recently released. Supports transition to Python 3.x – 3.0 Includes changes that are not backwards compatible. – 3.1 Stable 3.x release. Resolves 3.0 performance issues. Package management for 3.x is still a bit snarky. We’ll focus on examples for 2.x Slide 3
- 4. Python Packages Package distributions – Include one or more *.py files – May include compiled source files (e.g. C++) – Include a top-level setup.py file to coordinate installation Three categories: – Pure-Python packages • Only include Python *.py files – Independent extension packages • Include compiled source files only on standard system libraries – Dependent extension packages • Include compiled source files that rely on third-party libraries Slide 4
- 5. The Good Assumption: you have administrative privileges – Windows administrator group – Linux super-user Impact: you can install Python in system directories! Idea: – Install packages in the system Python directories • Specifically, the site-packages directory is used – Can easily install pure-Python packages or pre-compiled extension packages Slide 5
- 6. Example 1 Installing on Windows with an installer executable Steps: 1.Download installer executable from package website 2.Execute installer Notes: – This uses the registry to find the Python installation – The installer can contain pre-compiled extension packages • These may differ for 32- and 64-bit installations – The installer will likely be specific to the Python version – This may work OK for dependent extension packages if the dependent libraries are stored in a standard location Slide 6
- 7. Example 2 Installing with Linux application utility Idea: use a standard Linux package manager – Ubuntu (and other Debian-based Linux variants) • apt-get install packagename – Fedora • yum install packagename Notes: – The package manager can install pre-compiled extension packages – The package manager handles package dependencies automatically – Not all Python packages are available within these managers – It’s unclear whether these packages are updated frequently Slide 7
- 8. Example 3 Installing a Python package from the PyPI repository PyPI: – A web-based service for hosting Python packages – Includes some pre-compiled extension packages – Several Python packages are used to install from PyPI • setuptools, distribute, pip Installing with pip: – pip install packagename Notes: – The package manager handles package dependencies automatically – Most Python packages are available from PyPI, including most releases – PyPI does not include many pre-compiled extension packages – The PyPI service has become a bit overloaded in the past year – You need to specify the HTTP_PROXY environment to talk to proxy servers Slide 8
- 9. Example 3 (cont) Problem: pip is not a standard Python package! Installing pip: – wget http://peak.telecommunity.com/dist/ez_setup.py – python ez_setup.py – easy_install pip Notes: – pip is now generally recommended over setuptools and distribute – The wget command is not a standard MS Windows utility • Download from http://users.ugent.be/~bpuype/wget/ Slide 9
- 10. Example 4 Installing a Python package from source Steps: – Download package source from PyPI or project website – Extract package from tarball or zipfile – python setup.py install Notes: – This process avoids the need to install pip – This is useful for packages that are not available on PyPI – This installation process can build and install some extension packages • Configuration of the compiler may be a problem • Configuration of extension package dependencies is a problem – This installation does not handle package dependencies • Package dependencies may cause an install to fail Slide 10
- 11. The Bad Problem: users do not have administrative privileges – This is not uncommon in industry Problem: different users on a computer need different package versions Idea: – A user can create a virtual Python environment – The user installs packages in this virtual Python directories Impact: – Users have complete control over their Python environment • Can even have multiple environments (e.g. for developers) – Can install both pure-Python packages or pre-compiled extension packages Slide 11
- 12. Virtual Python Idea: create an isolated Python environment virtualenv: – Creates a directory structure that mimics the system Python installation – Works on MS Windows, Linux and Mac OS Installing virtualenv (with administrative privileges) – Debian Linux • apt-get install python-virtualenv – Other • wget http://peak.telecommunity.com/dist/ez_setup.py • python ez_setup.py • easy_install virtualenv Slide 12
- 13. Virtual Python (cont) Creating a virtual Python environment: – virtualenv directory Virtual Python executable: – MS Windows: directory/Scripts/python.exe – Linux: directory/bin/python Notes: – On MS Windows the executables are installed in the Scripts directory • For simplicity, all subsequent examples will use the bin directory – The setuptools package is installed in a virtual environment • Can use the easy_install command to install packages Example: – directory/bin/easy_install pip Slide 13
- 14. Virtual Python (cont) Problem: setup a virtual environment without administrative privileges vpy_install: – Bootstrap a virtualenv installation from a script Steps: – wget http://doiop.com/pyutilib/vpy_install – python vpy_install directory Notes: – Standard tools like setuptools and pip are installed by vpy_install – The specified directory is a virtualenv Python environment – On Linux, you can use a simpler command-line: • vpy_install directory – The –-site-packages option can be used to expose packages that are installed in the system Python installation • This is particularly useful when using precompiled extension packages, which are installed in system Python directories Slide 14
- 15. Example 5 Using a virtual environment to isolate package installations Idea: – Create a virtual python installation with vpy_install – Install packages with pip Example (Linux): – wget http://doiop.com/pyutilib/vpy_install – vpy_install mypy – mypy/bin/pip install packagename Notes: – Within the virtual environment, pip is installed as an executable • You do not need to execute it with the Python executable Slide 15
- 16. Example 5 (cont) Using a virtual environment to isolate package installations Note: source package installations using a virtual Python executable are installed in the virtual environment Example (Linux): – wget http://doiop.com/pyutilib/vpy_install – vpy_install mypy – cd packagename – ../mypy/bin/python setup.py install Slide 16
- 17. Example 6 Using an installer executable on Windows with a virtual environment Problem: – MS Windows installers use the system Python installation – If a user does not have administrative privileges, then an installer cannot be used directly! Idea: – “Install” the executable with the easy_install command Example: – easy_install.exe matplotlib-1.0.0.win32-py2.6.exe Note: this only works if the executable was created by the Python distutils package! Slide 17
- 18. Example 6 (cont) Using an installer executable on Windows with a virtual environment Idea: – Modify the MS Windows registry to register the virtual python executable Steps: 1.Download the the_python.py script that is described at http://doiop.com/nedbatchelder/register_python.html 2.Register the virtual python executable (as administrator) • mypyScriptspython.exe the_python.py 1.Execute the MS Windows installer 2.Reregister the original python executable (as administrator) Slide 18
- 19. The Ugly Problem: – you need to use an extension package that has dependencies – prebuilt packages are not available Observation: – There is very little consistency between installation setup for different packages – Few packages use configuration management tools like autotools or cmake – Build instructions can be very different for different platforms There are few options beyond diving deep into the package build process! Slide 19
- 20. Example 7 Installing a Python extension package from source Note: there is no standard way for doing this Example: SciPy documentation – General build instructions – Separate instructions for Linux, Mac OS, Windows, and Solaris Recommendations: – Make sure that this package is going to be worth your time – Don’t hesitate to ask the developer team for help – Have patience ... Slide 20
- 21. Example 8 PyCoinInstall – Managing builds for Python extension packages Note: this script is a relatively new project within CoinBazaar – https://projects.coin-or.org/CoinBazaar/wiki/Projects/PyCoinInstall Idea: – Create a virtual environment with virtualenv – Define package installation with a simple plugin system • Packages define download, build and/or install operations – A user creates a simple configuration file to specify the packages that will be installed – The pci command is used to manage the installation Notes: – Current development is focused on COIN-OR related packages – Current plugins are tailored for building under Linux Slide 21
- 22. Final Thoughts Installing and distributing pure-Python packages is quite easy. Installing extension packages is much more difficult. – Try to leverage pre-compiled extension packages! Virtual Python environments offer a flexible paradigm for managing different Python configurations – These are worth using even if you have administrative privileges! Slide 22
Editor's Notes
- #2: SAND 2010-7908C