Split extra libraries and add more granular control over tests #699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open

Zeitsperre wants to merge 20 commits into main from extras-requirements

.github/workflows/main.yml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -6,70 +6,99 @@ on:
  
          - main

      pull_request:

    concurrency:

      # For a given workflow, if we push to the same branch, cancel all previous builds on that branch except on main.

      group: "${{ github.workflow }}-${{ github.ref }}"

      cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}

    jobs:

      lint:

        name: Linting Suite

        runs-on: ubuntu-latest

        steps:

          - name: Cancel previous runs

            uses: styfle/cancel-workflow-action@0.11.0

            with:

              access_token: ${{ github.token }}

          - uses: actions/checkout@v4

          - uses: actions/setup-python@v5

          - uses: actions/checkout@v6

          - uses: actions/setup-python@v6

            with:

              python-version: "3.10"

              cache: 'pip'

          - name: Install tox

            run: |

              pip install tox>=4.0

              pip install tox>=4.30.3

          - name: Run linting suite ⚙️

            run: |

              tox -e lint

      test:

        name: Testing with Python${{ matrix.python-version }}

        name: Testing with (Python${{ matrix.python-version }}, ${{ matrix.os }})

        needs: lint

        runs-on: ubuntu-latest

        runs-on: ${{ matrix.os }}-latest

        strategy:

          matrix:

            python-version: [ "3.10", "3.11", "3.12", "3.13" ]

            os: ["ubuntu"]

            include:

              - tox-env: py310-extra

                python-version: "3.10"

              - tox-env: py311-extra

                python-version: "3.11"

              - tox-env: py312-extra

                python-version: "3.12"

              - tox-env: py313-extra

                python-version: "3.13"

              # - python-version: "3.11"

              #   os: "windows"

              - python-version: "3.11"

                os: "macos"

        steps:

        - uses: actions/checkout@v4

        - name: Install packages 📦

          run: |

            sudo apt-get update

            sudo apt-get -y install libnetcdf-dev libhdf5-dev

        - uses: actions/setup-python@v5

          name: Setup Python ${{ matrix.python-version }}

          with:

            python-version: ${{ matrix.python-version }}

        - name: Install tox 📦

          run: pip install "tox>=4.0"

        - name: Run tests with tox ⚙️

          run: tox -e ${{ matrix.tox-env }}

        - name: Run coveralls ⚙️

          if: matrix.python-version == 3.10

          uses: AndreMiras/coveralls-python-action@develop

          - uses: actions/checkout@v6

          - name: Install packages 📦 (Linux)

            if: ${{ matrix.os == 'ubuntu' }}

            run: |

              sudo apt-get update

              sudo apt-get -y install libnetcdf-dev libhdf5-dev

          - name: Install packages 📦 (macOS)

            if: ${{ matrix.os == 'macos' }}

            uses: tecolicom/actions-use-homebrew-tools@b9c066b79607fa3d71e0be05d7003bb75fd9ff34 # v1.3

            with:

              tools: hdf5 netcdf

              cache: "yes"

          - uses: actions/setup-python@v6

            name: Setup Python ${{ matrix.python-version }}

            with:

              python-version: ${{ matrix.python-version }}

              cache: 'pip'

          - name: Install tox 📦

            run: |

              python -m pip install "tox>=4.30.3" "tox-gh>=1.5"

          - name: Run tests with tox ⚙️

            run: |

              tox

            env:

              TOX_GH_MAJOR_MINOR: ${{ matrix.python-version }}

          - name: Report Coverage

            uses: coverallsapp/github-action@v2

            with:

              flag-name: run-${{ matrix.python-version }}

              parallel: true

      docs:

        name: Build docs 🏗️

        needs: lint

        runs-on: ubuntu-latest

        steps:

          - uses: actions/checkout@v4

          - uses: actions/setup-python@v5

          - uses: actions/checkout@v6

          - uses: actions/setup-python@v6

            name: Setup Python 3.10

            with:

              python-version: "3.10"

              cache: 'pip'

          - name: Build documentation 🏗️

            run: |

              pip install -e .[dev]

              cd docs && make html

      finish:

        name: Finish

        needs:

          - test

          - docs

        runs-on: ubuntu-latest

        permissions:

          pull-requests: write

        steps:

          - name: Coveralls Finished

            uses: coverallsapp/github-action@v2

            with:

              parallel-finished: true

.gitignore

-Original file line number
+Diff line change
@@ Expand Up / @@ -20,3 +20,4 @@ docs/_build @@
     *.orig
     .coverage
     .pytest_cache
+    coverage.lcov

.readthedocs.yml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -10,7 +10,7 @@ formats:
  
      - epub

    build:

      os: ubuntu-22.04

      os: "ubuntu-24.04"

      tools:

        python: "3.11"  # Use standard CPython version; `mambaforge-22.9` is not valid here

CONTRIBUTING.rst

-Original file line number
+Diff line change
@@ Expand Up @@
       # clone the repository locally
       $ git clone git@github.com:USERNAME/pywps.git
       $ cd pywps
-      $ pip install -e . && pip install -r requirements.txt
+      $ pip install -e .
       # add the main PyWPS development branch to keep up to date with upstream changes
       $ git remote add upstream https://github.com/geopython/pywps.git
@@ Expand Down @@

README.md

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -20,7 +20,7 @@ As of PyWPS 4.0.0, PyWPS is released under an
  
    ## Dependencies

    See [requirements.txt](requirements.txt) file

    See [pyproject.toml](pyproject.toml) file

    ## Install

    @@ -34,11 +34,11 @@ $ pip install .
  
    ## Run tests

    ```bash

    pip install -r requirements-dev.txt

    pip install ".[dev]"

    # run unit tests

    python -m pytest tests

    # run code coverage

    python -m coverage run --source=pywps -m unittest tests

    python -m coverage run --source=pywps -m pytest

    python -m coverage report -m

    ```

docs/conf.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -81,9 +81,6 @@ def __getattr__(cls, name): @@
     MOCK_MODULES = ['lxml', 'lxml.etree', 'lxml.builder']
-    # with open('../requirements.txt') as f:
-    #    MOCK_MODULES = f.read().splitlines()
     for mod_name in MOCK_MODULES:
         sys.modules[mod_name] = Mock()
@@ Expand Down @@

docs/extensions.rst

-Original file line number
+Diff line change
@@ Expand Up / @@ -40,7 +40,7 @@ The scheduler extension uses the `DRMAA`_ @@
     library to talk to the different scheduler systems. Install the additional
     Python dependencies using pip::
-      $ pip install -r requirements-processing.txt  # drmaa
+      $ pip install ".[processing]"  # drmaa
     If you are using the `conda <https://conda.io/docs/>`_ package manager you can
     install the dependencies with::
@@ Expand Down Expand Up / @@ -95,7 +95,6 @@ Slurm. @@
     Docker Container Extension
     ---------------------------
     .. todo:: This extension is on our wish list. In can be used to encapsulate
       and control the execution of a process. It enhances also the use case of
       Web Processing Services in a cloud computing infrastructure.

docs/install.rst

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -12,7 +12,7 @@ Installation
  
    Dependencies and requirements

    -----------------------------

    PyWPS runs on Python 3.7 or higher. PyWPS is currently tested and developed on Linux (mostly Ubuntu).

    PyWPS runs on Python 3.10 or higher. PyWPS is currently tested and developed on Linux (mostly Ubuntu).

    In the documentation we take this distribution as reference.

    Prior to installing PyWPS, Git and the Python bindings for GDAL must be

    @@ -61,15 +61,9 @@ Manual installation
  
                $ tar zxf pywps-x.y.z.tar.gz

                $ cd pywps-x.y.z/

            Then install the package dependencies using pip::

            To install PyWPS system-wide with all dependencies run::

                $ pip install -r requirements.txt

                $ pip install -r requirements-gdal.txt  # for GDAL Python bindings (if python-gdal is not already installed by `apt-get`)

                $ pip install -r requirements-dev.txt  # for developer tasks

            To install PyWPS system-wide run::

                $ sudo pip install .

                $ sudo pip install ".[dev,extra] # for developer tasks and GDAL Python bindings (if python-gdal is not already installed by `apt-get`)

    For Developers

            Installation of the source code using Git and Python's virtualenv tool::

environment.yml

-Original file line number
+Diff line change
@@ Expand Up / @@ -4,20 +4,27 @@ channels: @@
     dependencies:
      - python >=3.10,<3.14
      - pip >=25.0
+     - humanize >=4.5.0
+     - jinja2 >=3.1.0
+     - jsonschema >=4.20.0
+     - lxml >=6.0.2
+     - markupsafe >=3.0.3
+     - numpy >=1.22.2
      - owslib >=0.35.0
+     - python-dateutil
      - requests >=2.32.5
-     - werkzeug >=3.1.4
      - sqlalchemy >=2.0.44
-     - lxml >=6.0.2
      - urllib3 >=2.5.0
-     - markupsafe >=3.0.3
-     - numpy >=1.22.2
-     - zarr <3
+     - werkzeug >=3.1.4
+     # extras
      - fiona
      - geotiff
+     - netcdf4
+     - tifffile <=2025.5.10
+     - zarr <3.0
      # tests
      - pytest
-     - ruff >=0.5.7
+     - ruff >=0.14.0
      # docs
      - sphinx >=7.0.0

pyproject.toml

-Original file line number
+Diff line change
@@ -1,11 +1,124 @@
     [build-system]
-    requires = ["setuptools"]
+    requires = ["setuptools>=80", "wheel"]
     build-backend = "setuptools.build_meta"
+    [project]
+    name = "pywps"
+    description = "PyWPS is an implementation of the Web Processing Service standard from the Open Geospatial Consortium. PyWPS is written in Python."
+    readme = { file = "README.md", content-type = "text/markdown" }
+    license = { text = "MIT" }
+    authors = [
+      { name = "Jachym Cepicky", email = "jachym.cepicky@gmail.com" }
+    ]
+    maintainers = [
+      { name = "Jachym Cepicky", email = "jachym.cepicky@gmail.com" }
+    ]
+    keywords = ["PyWPS", "WPS", "OGC", "processing"]
+    requires-python = ">=3.10"
+    dynamic = ["version"]
+    classifiers = [
+      "Development Status :: 5 - Production/Stable",
+      "Environment :: Web Environment",
+      "Intended Audience :: Developers",
+      "Intended Audience :: Science/Research",
+      "License :: OSI Approved :: MIT License",
+      "Operating System :: OS Independent",
+      "Programming Language :: Python",
+      "Programming Language :: Python :: 3",
+      "Programming Language :: Python :: 3.10",
+      "Programming Language :: Python :: 3.11",
+      "Programming Language :: Python :: 3.12",
+      "Programming Language :: Python :: 3.13",
+      "Topic :: Scientific/Engineering :: GIS",
+    ]
+    urls = {Homepage = "https://pywps.org", Repository = "https://github.com/geopython/pywps"}
+    dependencies = [
+      "humanize >=4.5.0",
+      "jinja2 >=3.1.0",
+      "jsonschema >=4.20.0",
+      "lxml >=6.0.2",
+      "markupsafe >=3.0.3",
+      "numpy >=1.22.2", # not directly required, pinned by Snyk to avoid a vulnerability
+      "owslib >=0.35.0",
+      "python-dateutil",
+      "requests >=2.32.5",
+      "sqlalchemy >=2.0.44",
+      "urllib3 >=2.5.0", # not directly required, pinned by Snyk to avoid a vulnerability
+      "werkzeug >=3.1.4",
+    ]
+    [project.optional-dependencies]
+    dev = [
+      "bump-my-version",
+      "coverage",
+      "docutils",
+      "pylint",
+      "pytest",
+      "pytest-cov",
+      "ruff >=0.14.0",
+      "sphinx",
+      "tox >=4.30.3",
+      "twine",
+      "wheel"
+    ]
+    extra = [
+      "fiona",
+      "geotiff",
+      "netCDF4",
+      "tifffile <=2025.5.10",
+      "zarr <3.0"]
+    processing = ["drmaa"]
+    s3 = ["boto3"]
+    [project.scripts]
+    joblauncher = "pywps.processing.job:launcher"
+    [tool.bumpversion]
+    current_version = "4.7.0"
+    commit = false
+    tag = false
+    parse = "(?P<major>\\d+)\\.(?P<minor>\\d+)\\.(?P<patch>\\d+)"
+    serialize = ["{major}.{minor}.{patch}"]
+    [[tool.bumpversion.files]]
+    filename = "pywps/__init__.py"
+    search = '__version__ = "{current_version}"'
+    replace = '__version__ = "{new_version}"'
+    [[tool.bumpversion.files]]
+    filename = "VERSION.txt"
+    search = "{current_version}"
+    replace = "{new_version}"
+    [tool.coverage.run]
+    relative_files = true
+    [tool.pytest.ini_options]
+    addopts = [
+      "--import-mode=importlib",
+    ]
+    xfail_strict = true
+    pythonpath = ["tests"]
+    testpaths = ["tests"]
+    markers = [
+      "online: marks tests requiring network",
+      "requires_fiona: marks tests requiring fiona module",
+      "requires_geotiff: marks tests requiring geotiff module",
+      "requires_netcdf4: marks tests requiring netcdf4 module",
+    ]
+    [tool.setuptools]
+    include-package-data = true
+    [tool.setuptools.packages.find]
+    exclude = ["docs", "tests", "tests.*"]
+    [tool.setuptools.dynamic]
+    version = { file = ["VERSION.txt"] }
     [tool.ruff]
     lint.select = ["E", "W", "F", "C90"]   # Flake8-equivalent rule families
     lint.ignore = ["F401", "E402", "C901"]
     line-length = 120
     exclude = ["tests"]

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split extra libraries and add more granular control over tests #699

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Split extra libraries and add more granular control over tests #699

Are you sure you want to change the base?

Uh oh!

Split extra libraries and add more granular control over tests #699

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!