When you begin working on multiple projects in Python that depend on external libraries like anything you install with pip, you should be using a virtual environment. A virtual environment is a self contained directory with a python executable that you can run, in addition you can use pip to install packages directly to that virtual environment to keep them isolated from your global install or other virtual environments. In this post I will show you what a virtual environment is, how to set one up, how to use it, and clear up any confusion you might have.

What is a Virtual Environment

Like I stated above a virtual environment is a directory on your system where all of the Python files are copied to, and some fancy linking is done to make it run in a self contained manor apart from your global Python installation or any other virtual environments. Virtual environments are designed to be disposable and while if you try to copy an existing virtual environment around to different parts of your system you will have issues, you can just quickly setup a new one where you want it and install the dependencies with 2 commands.

Why Use a Virtual Environment

Again I alluded to this in the introduction but the key here is to keep the packages and dependencies for your project contained in an area that only your project uses. Lets imagine the scenario where you have 2 projects that utilize the request library, one of those projects uses the latest version but the other project only supports 2 versions behind. If you were to be using your global install, you would have to uninstall the requests library and install the desired version each time you are switching between projects which is not ideal. This goes for your deployment server also, each app your server hosts can use its own virtual environment to guarantee that updates to ones dependencies will not affect the other.

It is my opinion that when you need to install even a single package for your project, you should be using a virtual environment. With modern IDE's like PyCharm it will setup one for you when you create a project so there is no real good reason not to just use one right out of the gate.

Setting up a Virtual Environment

The tool to build a virtual environment is virtualenv which is a Python package. This will be the only package you should install to your global python environment, you can install it by the following command.

$ pip install virtualenv

Once that is installed you can setup a new virtual environment with the following command

$ python -m virtualenv /path/to/create/env

I typically name my virtual environments 'env' but you can name them anything you want.

And that is it, it will do everything under the hood to create a virtual environment at the path you specified.

Using Your Virtual Environment

A very quick way of calling your python program using your virtual environment is to reference the environments Python executable directly.

$ /path/to/env/bin/python /path/to/my/program.py

However I find the best way to interact with your virtual environment is to "activate" it so that whenever you reference "python" on the command line it references your virtual environment and not your global environment. Below is the command to activate your python virtual environment.

 $ source /path/to/env/bin/activate
(env) $

In this command your virtual environment is located at /path/to/env/ and inside that directory you have a bin/ directory with an activate command inside of it. Calling this command activates that specific virtual environment.

You can tell that you have activated your virtual environment by looking at the start of your prompt. It will have the name of your virtual environment prepended.

(env) $

And now regardless of the directory you are in, when you call "python" it will reference the python executable file at /path/to/env/bin/python

Installing Packages to Virtual Environment

To install packages without your virtual environment activated, you can install directly via calling your environments pip executable directly.

$ /path/to/env/bin/pip install <packagename>

In order to install packages, assuming you have your virtual environment activated you can just install them as normal using pip

(env) $ pip install <package>

And it will install it to the virtual environment you have activated.

Validating

This process may seem kind of confusing if you are new to it, so to demonstrate this lets launch an interactive python shell in various ways and look up its path location.

First without a virtual environment activated, just launch an interactive shell

$ python
Python 3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

And import the sys library, an call sys.executable which will list the path on the system of your python executable.

>>> import sys
>>> sys.executable
'C:\\Program Files\\Python39\\python.exe'
>>>

And as you can see this is my global python install python environment, now to exit.

>>> exit()
$ 

Now lets launch an interactive Python shell from calling our python executable in our virtual environment

$ /path/to/env/bin/python
Python 3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:52:42) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

Now lets issue the same 3 lines of code.

>>> import sys
>>> sys.executable
'/path/to/env/bin/python.exe'
>>> exit()

And as you would expect, we manually call our virtual environments python and it sys.executable shows that. 

Now lets activate the virtual environment

 $ source /path/to/env/bin/activate
(env) $

And launch a Python interactive shell by calling it like we did in our first example.

(env) $ python
Python 3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:58:42) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

And run our same 3 lines of code.

>>> import sys
>>> sys.executable
'/path/to/env/bin/python.exe'
>>> exit()

And you can see that it shows you are running python from your virtual environment.

Running your code from a Virtual Environment

You likely can assume how this is done by now, but if you have your virtual environment active, just call python and point it at your programs entry script.

(env) $ python /path/to/myprogram.py

And if you don't have your environment active or just don't want to activate it this go around you can enter the command similar to below.

/path/to/env/bin/python.exe /path/to/my/program.py

And that is really it, once you understand how it is working it is fairly easy to get the concept and will really save you from some goofy dependency headaches in the future! If anything here isn't clear or you have additional