Day 13: CI for a Python project of the National Institute of Standards and Technology

ci-2022-12 (27 Part Series)

1 The 2022 December CI Challenge
2 Day 2: Add GitHub Action CI to the Net-Async-Redis-XS Perl module
… 23 more parts…
3 Day 1: Test and CI for MoreBeautifulPython
4 Day 3: CI for farmworld, a Python package – a failed attempt
5 Day 3 v2: GitHub Action CI for wsblib Python
6 Day 4: GitHub Actions for Colombian Spanish.
7 Day 5: CI for Win32-Wlan Perl module
8 Day 6: GitHub Actions CI for the pcr_optimizer Python package
9 Day 7: Be pragmatic setting up CI for the RDF::KV
10 Day 8: Adding CI to Perl::Efl – sometimes you need to do some extra work
11 Day 9: CI for Mojo-UserAgent-Cached and Plack-Middleware-Greylist
12 Day 10: GitHub Actions CI for the rdf-kv Ruby Gem
13 Day 11: CI for wp2txt Ruby project
14 Day 12: CI for sdk-py Python packages
15 Day 13: CI for a Python project of the National Institute of Standards and Technology
16 Day 14: CI for the Log::Any Perl module
17 Day 15: Failing CI to help fixing filesystem issues in a Python package
18 Day 16: Moving from Travis-CI to GitHub Actions for Marpa::R2
19 Day 17: A JavaScript bug and GitHub Workflow CI for the four-pillars Ruby gem
20 Day 18: GitHub Workflow for ruby-request-builder – false security
21 Day 19: CI for Plack-Middleware-LogAny
22 Day 20: Skeletons are good, but can be misleading – CI for the draftjs_html Ruby Gem
23 Day 21: CI for multi_string_replace Ruby Gem
24 Day 22: Full gas in neutral – CI for prepper
25 Day 23: CI using timescaledb a PostgreSQL based time series database
26 Day 24: CI for perl5-MIME-Types
27 Day 25: CI for Data::Alias in Perl – including threaded perl

Interesting coincidences.

Just a few days ago as I started to work on the Open Source Developer Course I started
to collect Open Source projects by governments and today the first project I found on PyDigger that I wanted to try to contribute to was a project of the NIST – the National Institute of Standards and Technology.

Side note: I find it funny that in this age of globalization when anyone can easily bump into any organization (barring language barrier), organizations still call themselves “National” without including an indication of which nation, in their name. At least on the web site of NIST you can see US Department of Commerce in their logo.

The process

Anyway, what did I do?

I noticed that the project has a folder called tests but that it also has a file called runtests.py.
I am not sure why do some projects have their own test-runner instead of just running pytest, but I seem to recall I saw this in other Django-related projects as well.

So I set started a Docker container that I use for better isolation of the foreign code I run from my computer and ran

python runtests.py

Enter fullscreen mode Exit fullscreen mode

Not surprisingly it failed complaining about missing Django.

So I went ahead and installed all the requirements I could find:

pip install -r requirements.txt
pip install -r requirements.core.txt

Enter fullscreen mode Exit fullscreen mode

Running again

python runtests.py

Enter fullscreen mode Exit fullscreen mode

still failed. This time it was missing psycopg2 which is the Python package to connect to PostgreSQL.

So I tried to install it:

pip install psycopg2

Enter fullscreen mode Exit fullscreen mode

It failed, but it pointed me at the installation documentation.
That’s a very nice touch!

From that website I understood I need to install libpq. So I ran

apt-cache search libpq

Enter fullscreen mode Exit fullscreen mode

(I use an Ubuntu 22.10 based Docker container for my experiments.)

In the results of this command I saw libpq-dev. This is the development package. Libraries usually need that so I installed it:

apt-get install -y libpq-dev

Enter fullscreen mode Exit fullscreen mode

and ran the installation again:

pip install psycopg2

Enter fullscreen mode Exit fullscreen mode

After that I ran the tests again:

python runtests.py

Enter fullscreen mode Exit fullscreen mode

The output looked very much like an output from a test-run and it ended with

Ran 86 tests in 0.271s

Enter fullscreen mode Exit fullscreen mode

It was not hard at all.

I created the GitHub Actions configuration file using a Docker container and configured python 3.11. I let the developers of the package decide which other versions of Python they might want to test with and if they might want to test on Windows and macOS as well.

I sent the Pull-Request

GitHub Actions

name: CI

on:
  push:
  pull_request:
  workflow_dispatch:
  schedule:
    - cron: '42 5 * * *'

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11"]

    runs-on: ubuntu-latest
    name: OS Python ${{matrix.python-version}}
    container: python:${{matrix.python-version}}

    steps:
    - name: Checkout
      uses: actions/checkout@v3

    - name: Install dependencies
      run: |
        apt-get install -y libpq-dev
        pip install -r requirements.txt
        pip install -r requirements.core.txt
        pip install psycopg2

    - name: Check Python version
      run: python -V

    - name: Run tests
      run: python runtests.py

Enter fullscreen mode Exit fullscreen mode

Conclusion

I am already quite experienced setting up CI in general and GitHub Actions in particular, but I think even people who are new to this area could do similar task within a few hours. So I think these will be reasonable assignments in the Open Source Development Courses.

ci-2022-12 (27 Part Series)

1 The 2022 December CI Challenge
2 Day 2: Add GitHub Action CI to the Net-Async-Redis-XS Perl module
… 23 more parts…
3 Day 1: Test and CI for MoreBeautifulPython
4 Day 3: CI for farmworld, a Python package – a failed attempt
5 Day 3 v2: GitHub Action CI for wsblib Python
6 Day 4: GitHub Actions for Colombian Spanish.
7 Day 5: CI for Win32-Wlan Perl module
8 Day 6: GitHub Actions CI for the pcr_optimizer Python package
9 Day 7: Be pragmatic setting up CI for the RDF::KV
10 Day 8: Adding CI to Perl::Efl – sometimes you need to do some extra work
11 Day 9: CI for Mojo-UserAgent-Cached and Plack-Middleware-Greylist
12 Day 10: GitHub Actions CI for the rdf-kv Ruby Gem
13 Day 11: CI for wp2txt Ruby project
14 Day 12: CI for sdk-py Python packages
15 Day 13: CI for a Python project of the National Institute of Standards and Technology
16 Day 14: CI for the Log::Any Perl module
17 Day 15: Failing CI to help fixing filesystem issues in a Python package
18 Day 16: Moving from Travis-CI to GitHub Actions for Marpa::R2
19 Day 17: A JavaScript bug and GitHub Workflow CI for the four-pillars Ruby gem
20 Day 18: GitHub Workflow for ruby-request-builder – false security
21 Day 19: CI for Plack-Middleware-LogAny
22 Day 20: Skeletons are good, but can be misleading – CI for the draftjs_html Ruby Gem
23 Day 21: CI for multi_string_replace Ruby Gem
24 Day 22: Full gas in neutral – CI for prepper
25 Day 23: CI using timescaledb a PostgreSQL based time series database
26 Day 24: CI for perl5-MIME-Types
27 Day 25: CI for Data::Alias in Perl – including threaded perl

原文链接：Day 13: CI for a Python project of the National Institute of Standards and Technology