November 2023 Summaries

The final article in a series on Python packaging explores the complexities of distributing a Python package with a C extension, emphasizing the need for additional configuration compared to pure Python packages. After introducing the concept of using a C extension to improve the performance of a Python-based merging algorithm, the article details the steps taken to integrate the C code into a Python package. This involves defining a PyMethodDef function, configuring a module with PyModuleDef, and creating an initialization function, followed by building the extension using setuptools with a setup.py file, despite the use of the Poetry tool. The process includes challenges with binary packaging due to platform-specific dependencies, which can be circumvented by opting for source builds that require a C compiler. The article also highlights the importance of a MANIFEST.in file for including necessary C source files in the package, ultimately demonstrating how to publish the package on PyPi. This technical exploration underscores the intricate work behind the seemingly simple operation of pip installation, providing valuable insights for developers aiming to package Python extensions efficiently.

In the second part of a series on Python package creation, the author explores the use of the Poetry dependency manager to package and distribute a simple Python module, specifically a merge sort example, as an alternative to using setuptools and setup.py. Poetry simplifies the packaging process by automating configuration tasks typically handled in setup.py, making it easier to build and upload distributions to PyPI. The text details the steps to configure a project with Poetry, including testing the package locally and using TestPyPI for a trial run before final publication. The workflow demonstrates how Poetry can streamline the process while maintaining similar steps to those used with setuptools, ultimately facilitating the packaging and publishing of Python projects. The author also hints at future content involving Python C extensions and emphasizes the importance of thorough testing for packages intended for broader use.

This text is part of a three-part series that guides Python developers through the process of packaging and publishing a Python script to the Python Package Index (PyPI) using setuptools and twine. It emphasizes the importance of selecting an appropriate package name, in this case, opting for "mergefast" after deciding against "PyMerge" due to naming conventions and availability on PyPI. The article walks through the creation of a package structure, setting up a minimal `setup.py` file, creating source and wheel distributions, and testing these distributions locally and within different environments using tools like Earthly for containerized testing. The series also touches on the steps required to securely upload packages to PyPI using twine, including setting up an API token. This initial article sets the stage for later parts, which will explore packaging with Poetry and creating a Python C extension.