A virtual environment is a tool that helps you keep dependencies required by different projects separate by creating isolated Python environments for them. This means that each project can have its own dependencies, regardless of what dependencies every other project has.
Imagine a scenario where you are working on two Machine Learning projects both based on TensorFlow fo Python. One project uses TensorFlow 1.5 while the other one uses TensorFlow 2.1. In such cases, a virtual environment can be really useful to maintain dependencies of both projects.
By default, every project on your system will use these same directories to store and retrieve site packages (third party libraries). How does this matter? Now, in the previous example of two web applications you have two versions of TensorFlow. This
is a real problem for Python since it can’t differentiate between versions in the site-packages
directory. So both v1.5 and v2.1 would reside in the same directory with the same name. This is where virtual environments
come into play. To solve this problem, we just need to create two separate virtual environments for both projects.The great thing about this is that there are no limits to the number of environments you can have since they’re just directories
containing a few scripts.
Virtual Environments
should be used whenever you work on any Python based project. It is generally good to have one new virtual environment for every Python based project you work on. So the dependencies of every project are isolated
from the system and each other.
Note that we only install Python 3. We highly recommend you upgrade your Python scripts to version 3 since version 2 came to EOL (end of life) on January 2020.
pip
is the package installer for Python. You can use pip to install packages from the Python Package Index
and other indexes.
pip supports installing from PyPI, version control, local projects, and directly from distribution files.
$ pip install SomePackage
$ pip install SomePackage==1.0.4
$ pip install 'SomePackage>=1.0.4
$ pip install -r requirements.txt
$ pip freeze > requirements.txt
This will create a requirements.txt file, which contains a simple list of all the packages in the current environment, and their respective versions. You can see the list of installed packages without the requirements format using pip list. Later it will be easier for a different developer (or you, if you need to re-create the environment) to install the same packages using the same versions:
$ pip install -r requirements.txt
This can help ensure consistency across installations, across deployments, and across developers.
$ pip install –user SomePackage
$ pip install –upgrade SomePackage
$ pip uninstall SomePackage
pip also performs an automatic uninstall of an old version of a package before upgrading to a newer version.
$ pip list
$ pip show SomePackage
$ pip search “query”
The query will be used to search the names and summaries of all packages.
virtualenv
is a tool to create isolated Python environments.
virtualenv creates a folder which contains all the necessary executables to use the packages that a Python project would need.
There are a number of ways to install virtualenv on your system.
$ sudo apt install python-virtualenv
$ sudo easy_install virtualenv
$ sudo pip install virtualenv
$ cd project_folder
$ virtualenv venv
virtualenv venv will create a folder in the current directory which will contain the Python executable files, and a copy of the pip library which you can use to install other packages. The name of the virtual name will place the files in the current directory instead.
This creates a copy of Python in whichever directory you ran the command in, placing it in a folder named venv.
You can also use the Python interpreter of your choice (like python3.5).
$ virtualenv -p python3.5 venv
or change the interpreter globally with an env variable in ~/.bashrc:
$ export VIRTUALENVWRAPPER\_PYTHON=/usr/bin/python3.5
$ source venv/bin/activate
(venv)Your-Computer:project_folder UserName$
) to let you know that it’s active. From now on, any package that you install using
pip will be placed in the venv folder, isolated from the global Python installation.$ pip install SomePackage
$ deactivate
Delete its folder:
$ rm -rf venv
Conda
is an open source package management system and environment management system for installing multiple versions of software packages and their dependencies and switching easily between them. It works on Linux, macOS and Windows,
and was created for Python programs but can package and distribute any software.
$ conda –version
$ conda update conda
Conda allows you to create separate environments containing files, packages and their dependencies that will not interact with other environments.
When you begin using conda, you already have a default environment named base.
$ conda create –name EnvironmentName PackageName
$ conda activate EnvironmentName
$ source activate EnvironmentName
$ conda deactivate
$ conda info –envs
$ conda environments
base /home/username/Anaconda3
snowflakes \* /home/username/Anaconda3/envs/snowflakes
$ conda activate
$ source activate
$ conda remove –name EnvironmentName –all
$ conda search scipy
$ conda install scipy
$ conda list
We install pip
, virtualenv
and conda
on Remote & Physical workstations.
We deliver Conda
through miniconda3
on the scientific application stack.
1. Create your virtual environment in your project directory
$ virtualenv venv
2. Add your venv directory to .gitignore
$ echo 'venv' > .gitignore
3. Activate the environment
$ source venv/bin/activate
4. Install the packages
$ pip install SomePackage
5. Freeze the requirements
$ pip freeze > requirements.txt
6. Check requirements.txt into source control
$ git add requirements.txt
The basic idea here is simple. Your virtual environment stays in its own folder, which you don’t check into source control. That way your environment does not pollute your project’s root. The requirements, on the other hand, do get checked into source control.