Setting up a Python debug environment to investigate a segfaulting Python C extension

python_debug_env.md

Notes on setting up a Python debug environment

Used to investigate a segfault in a Python C extension within a Python package.

1. Install debug Python build using pyenv then check is available for use:

   pyenv install -g 3.8.5
   pyenv versions
   

2. Switch to locally using that interpreter within a dir:

   cd path/to/some/repo
   pyenv local 3.8.5-debug
   which python
   

3. Install certain dependencies for this project using that interpreter:

   python -m pip install cython matplotlib
   

4. Edit setup.py and ensure the extra_compile_args parameter is set to ['-Wall', '-O', '-g'] for relevant setuptools.Extensions to ensure they can be (re)built with debug support.

5. (Re)build all Python C extensions for the package:

   python setup.py clean --all
   python setup.py develop
   

6. Run Python in a debugger:

   gdb -ex r --args /bin/bash python tests/segfaulting_script.py
   

Further resources:

• Debugging Python C extensions: https://pythonextensionpatterns.readthedocs.io/en/latest/debugging/debug.html
• Investigating segfaults using gdb: http://www.unknownroad.com/rtfm/gdbtut/gdbsegfault.html