Python Corner

updated: 2024-05-18

Table of Contents:

  1. Handling Certificates with Python
    1. Using PIP Config
      1. Setting PIP Configuration
      2. PIP Configuration Locations
  2. Handling Certificates in code
  3. PEP 476 - Enabling Certificate Verification

Handling Certificates with Python

StackOverflow: How To Add A Custom CA

Using PIP Config

A quick way to set Python configuration is to set it in the pip.conf which contains

NOTE: Python uses a hierarchy to search for the configuration file and what takes presedence. I.E. Your local project pip.conf takes presedence over your global pip.conf which does make it difficult if you do not have admin writes to set global configs on some Operating systems.

Setting PIP Configuration 

pip config set global.cert path/to/ca-bundle.crt
pip config list
conda config --set ssl_verify path/to/ca-bundle.crt
conda config --show ssl_verify

# Bonus while we are here...
git config --global http.sslVerify true
git config --global http.sslCAInfo path/to/ca-bundle.crt

PIP Configuration Locations

pip has 3 “levels” of configuration files:

  • global: system-wide configuration file, shared across *users.
  • user: per-user configuration file.
  • site: per-environment configuration file; i.e. per-virtualenv.

Use the following command to figure out the locations where python will search for your configurations:

$ python -m pip config -v list
For variant 'global', will try loading '[MASK]'
For variant 'user', will try loading '[MASK]'
For variant 'user', will try loading '[MASK]'
For variant 'site', will try loading '[Mask]' # This is if you are using a virtual env or penv manager
>>
PIP Config Locations
OSLevelLocation
MacOSglobal/Library/Application Support/pip/pip.conf
user$HOME/.pip/pip.conf
site$VIRTUAL_ENV/pip.conf
Linuxglobal/etc/pip.conf# If paths set in ENV XDG_CONFIG_DIRS then:/etc/xdg/pip/pip.conf
user$HOME/.pip/pip.conf# If paths set in ENV XDG_CONFIG_DIRS then:$HOME/.config/pip.conf
site$VIRTUAL_ENV/pip.conf
WindowsglobalC:\ProgramData\pip\pip.ini
user%APPDATA%\pip\pip.ini
site%VIRTUAL_ENV%\pip.ini

Handling Certificates in code

For most systems it is always good to use an environment variable to point to a certificate bundle you want to use. This allows you to adjust code as needed and specify a different verification certificate if needed within code.

In linux/mac you can update your ~/.basrc or ~/.zshrc (depending on the shell you are using)

export CERT='<path-to-cert>'

Once that is done you can always import the values into python using something like this:

import os

CUST_CERT = os.getenv("CERT", False)

Depending on the use case you can default out to None or False to avoid SSL verifcation in most cases or you can set the value to True to leverage the default store. You could also set the value to none and do a quick check in your code to determine if you want to use the new variable you created ‘CUST_CERT’ or set the values within your code.

PEP 476 - Enabling Certificate Verification

PEP 476 – Enabling certificate verification by default for stdlib http clients is referenced to allow for proper ways one can leverge Certificate Trust Stores on a system.

OpenSSL also has a pair of environment variables, SSL_CERT_DIR and SSL_CERT_FILE which can be used to point Python at a different certificate database.

NOTE: It is recommended to leverage the OpenSSL environment variable in order to ensure Python will leverage the correct store when interating with a URL library. There are documents that suggest the usage of REQUESTS_CA_BUNDLE can also be used this works with the Python Requests module as discussed in GitHub Requests Issue which is further pointed out in the above mentioned PEP 476 on avoiding unforseen issues if you are not using the Requests module .