Install Python dependencies in an application | Divio Documentation (2024)

It's beyond the scope of this documentation to discuss all the ways in which Python dependencies can be installed inDivio applications. However, the options described here are sufficient to cover most needs.

If you are using Aldryn Django, refer to the section Python package installation in Aldryn Django applications below.

pip install and requirements.txt​

The simplest option is to list Python requirements in a requirements.txt file, and include the command:

RUN pip install -r requirements.txt

in the Dockerfile. See How to configure an existing Django application for deployment on Divio for an example.

However this is only adequate as a quick expedient in the early stages of development and is not recommended beyondthat, as it does not allow for complete pinning of all dependencies.

Pin all dependencies​

All Python dependencies, including implicit sub-dependencies, should be pinned to particular versions.

If any dependency is unpinned (that is, a particular version is not specified in the application's requirements)pip will install the latest version it finds, even if a different version was previously installed. This can causeyour application to fail with an deployment error or worse, a runtime error, the next time it is built - even if youdidn't change anything in it yourself.

To pin all dependencies, your application's requirements should be compiled to a complete list of explicitly specifiedpackage versions. This list should then be committed in the application repository, and not be changed until you need toupdate dependencies.

With pip​

Once you are able to build and run your application successfully, you know have a working set of Python dependenciesinstalled. Use pip freeze to write them in a new file:

docker-compose run web pip freeze > compiled_requirements.txt

And then ensure that the pip command in the Dockerfile uses that list:

RUN pip install -r compiled_requirements.txt

Other tools​

There are multiple Python tools such as pip-tools and Poetry that are more sophisticated than pip, that can also generate a complete list ofpinned dependencies.

You can use the tool of your choice. In each case, the tool itself needs to be available in the Docker buildenvironment. You can expect to find pip to be installed by default, but other tools will generally need to beinstalled manually in the Dockerfile.

An example using pip-tools:

RUN pip install pip-tools==5.5.0
RUN pip-compile requirements.in
RUN pip-sync requirements.txt

This installs pip-tools, compiles requirements.in to requirements.txt, then installs the components listedin requirements.txt.

Once you have a working set of dependencies, remove the pip-compile instruction so that the dependencies are pinnedand frozen in requirements.txt.

Python package installation in Aldryn Django applications​

By default, applications using an Aldryn Django Dockerfile use our own pip-reqs tool to compile a list wheel URLs from our wheels proxy server,and installs all packages as wheels.

To install Python dependencies an Aldryn application, list them in the requirements.in file. They need to beoutside the:

# <INSTALLED_ADDONS>
...
# </INSTALLED_ADDONS>

tags, since that part of the file is maintained automatically and is overwritten automatically withthe requirements from the Addons system.

This list is processed by the pip commands in the Dockerfile when the image is built.

Pinning dependencies in an Aldryn application​

Compile requirements.txt​

First, you need to have a working local set-up. Then run:

docker-compose run --rm web pip-reqs compile

This will create a requirements.txt file in the application, containing a list of all the packages in theenvironment, along with their versions.

When your application is built using the new requirements.txt instead of requirements.in,you'll have a guarantee that no unexpected changes will be permitted to find their way in to theapplication.

Amend the Dockerfile​

In order to have your application built using requirements.txt instead of requirements.in, youneed to remove the pip-reqs compile instruction from your application's Dockerfile.

First, remove the Divio-specific comment tags from the Dockerfile:

# <PYTHON>
...
# </PYTHON>

otherwise the Control Panel will simply overwrite your changes.

Then remove the pip-reqs compile instruction, so that requirements.txt will not be amended at the next build.

The next time you need to create a fresh requirements.txt, run:

docker-compose run web pip-reqs compile

Specifying packages via a URL​

Please use a commit hash when specifying packages via a URL of a tarballed or zipped archive.

For example:

https://github.com/account/repository/archive/2d8197e2ec4d01d714dc68810997aeef65e81bc1.zip#egg=package-name==1.0