python-repositories/README.md

# python-repositories

Unified repository interfaces and technology-specific adapters for Python projects.

Subclass an adapter in your own repository to add domain-specific methods while reusing connection management and CRUD operations.

## Architecture

| Layer            | Responsibility                                                    |
| ---------------- | ----------------------------------------------------------------- |
| **Interfaces**   | Abstract contracts for connection, context, and CRUD              |
| **Adapters**     | Technology-specific base classes (`RedisAdapter`, `MinioAdapter`) |
| **Your project** | Subclass an adapter and add domain methods                        |

Connection adapters expose `connect()`, `disconnect()`, and `is_connected()`. The latter verifies backend reachability with a cached health probe (default TTL: 1 second). Subclasses may override `health_check_ttl_seconds`. `connect()` is idempotent: calling it while already connected and healthy is a no-op.

## Optional dependencies

Repository **interfaces** import with the base package. **Adapters** require the matching extra; importing an adapter without its extra raises `ImportError` with install instructions.

Install with the extras you need:

```bash
uv add python-repositories[redis]
uv add python-repositories[minio]
uv add python-repositories[redis,minio]
```

### Redis (`JsonRepositoryInterface`)

Requires Redis with the RedisJSON module (e.g. redis-stack).

| Environment variable | Description                                          |
| -------------------- | ---------------------------------------------------- |
| `REDIS_URI`          | Redis connection URL (e.g. `redis://localhost:6379`) |

For key discovery:

- `list_keys(pattern)` is simple and returns a `list[str]`, but it uses Redis `KEYS` and may block on large datasets.
- `scan_keys(pattern, *, count=None)` is preferred for production use and yields keys incrementally via Redis `SCAN`.

Example:

```python
for key in repo.scan_keys("user:*"):
    print(key)
```

### MinIO (`ObjectRepositoryInterface`)

| Environment variable             | Description                                                              |
| -------------------------------- | ------------------------------------------------------------------------ |
| `MINIO_ENDPOINT`                 | MinIO server endpoint                                                    |
| `MINIO_ACCESS_KEY`               | Access key                                                               |
| `MINIO_SECRET_KEY`               | Secret key                                                               |
| `MINIO_BUCKET`                   | Bucket name                                                              |
| `MINIO_SECURE`                   | Use HTTPS (`true`/`false`; default: `true`)                              |
| `MINIO_CREATE_BUCKET_IF_MISSING` | Auto-create `MINIO_BUCKET` on connect (`true`/`false`; default: `false`) |

Copy [`.env.example`](.env.example) to `.env` for local development. `RedisConfig.from_env()` and `MinioConfig.from_env()` load `.env` automatically when resolving configuration from the environment.

For production, it is recommended to leave `MINIO_CREATE_BUCKET_IF_MISSING` unset so that `connect()` fails fast if the expected bucket is missing. For local development, you will often want `MINIO_SECURE=false` and `MINIO_CREATE_BUCKET_IF_MISSING=true`.

## Configuration injection

Adapters accept optional `config` and `client` keyword arguments for explicit setup and testing:

```python
from python_repositories import RedisAdapter, RedisConfig, MinioAdapter, MinioConfig

redis = RedisAdapter(config=RedisConfig(uri="redis://localhost:6379"))
minio = MinioAdapter(
    config=MinioConfig(
        endpoint="localhost:9000",
        access_key="minioadmin",
        secret_key="minioadmin",
        bucket="my-bucket",
        secure=False,
        # create_bucket_if_missing=True,  # convenient for local dev
    )
)
```

When both `config` and `client` are provided, `connect()` skips client creation (the caller owns the client lifecycle). `config` is required whenever `client` is injected.

Load a `.env` file explicitly:

```python
from python_repositories import load_dotenv

load_dotenv()  # optional — from_env() also loads .env by default
```

Calling `RedisAdapter()` or `MinioAdapter()` with no arguments still loads configuration from environment variables (and `.env` if present).

## Quick start

### JSON documents with Redis

```python
from python_repositories.examples.user_json_repository import UserJsonRepository

with UserJsonRepository() as repo:
    repo.save_user("alice", {"name": "Alice", "email": "alice@example.com"})
    user = repo.get_user("alice")
    repo.delete_user("alice")
```

### Binary objects with MinIO

```python
from io import BytesIO

from python_repositories.examples.artifact_object_repository import (
    ArtifactObjectRepository,
)

with ArtifactObjectRepository() as repo:
    repo.store_artifact("report-1", BytesIO(b"pdf bytes here"))
    data = repo.get_artifact("report-1")
```

### Subclassing in your own project

```python
from typing import Any

from python_repositories import RedisAdapter

class UserRepository(RedisAdapter):
    def _key(self, user_id: str) -> str:
        return f"user:{user_id}"

    def get_user(self, user_id: str) -> dict[str, Any] | None:
        return self.get(self._key(user_id))

    def save_user(self, user_id: str, user: dict[str, Any]) -> None:
        self.set(self._key(user_id), user)
```

## Public API

```python
from python_repositories import (
    ConnectionAwareInterface,
    ContextAwareInterface,
    JsonRepositoryInterface,
    MinioAdapter,
    MinioConfig,
    ObjectRepositoryInterface,
    RedisAdapter,
    RedisConfig,
    load_dotenv,
)
```

## Development

```bash
uv sync --all-extras
uv run pre-commit install   # once per clone — runs hooks on git commit
uv run pytest tests/unit/ -v                          # fast, no Docker
uv run pytest -m "not integration" -v                 # all non-Docker tests
uv run pytest -v                                      # full suite (requires Docker)
```

Integration tests are marked with `@pytest.mark.integration` and require Docker (testcontainers). Run unit tests alone for quick local feedback.

### CI base image

Gitea Actions jobs use a pre-built image (`python-repositories-ci`) with Python 3.12,
`uv`, and locked dependencies baked in. The image is rebuilt nightly and when
`uv.lock` changes; see [`docs/ci-image.md`](docs/ci-image.md) for runner setup and
bootstrap order.

CI runs unit and integration tests in parallel with coverage, then merges `.coverage` artifacts in a follow-up job (via [christopherhx/gitea-\*-artifact@v4](https://github.com/christopherHX/gitea-upload-artifact) for Gitea 1.26 compatibility). Combined coverage must meet the floor in [`fail_under` in `pyproject.toml`](pyproject.toml#L45-L48); enforcement happens after merging unit and integration coverage, not on unit-only runs.

To check coverage locally (requires Docker for the full suite):

```bash
uv run pytest --cov=python_repositories --cov-report=
uv run coverage report
```

`pre-commit` is included in the dev dependency group. `uv sync` installs the CLI, but git does not run hooks until you install them with `pre-commit install` (one time per clone). After that, commits run the checks defined in [`.pre-commit-config.yaml`](.pre-commit-config.yaml) (ruff, mypy, pyupgrade, prettier, and general file hygiene).

To run all hooks manually without committing:

```bash
uv run pre-commit run --all-files
```

## Releases

Releases are automated when a pull request is merged to `main`. CI reads the **merged PR title** to decide whether and how to bump the version.

### How it works

1. Open a PR targeting `main` (see [`.gitea/PULL_REQUEST_TEMPLATE.md`](.gitea/PULL_REQUEST_TEMPLATE.md)).
2. If the PR changes files under `python_repositories/`, the title **must** start with a version bump prefix (enforced by CI).
3. On merge, [`release.yml`](.gitea/workflows/release.yml) bumps [`pyproject.toml`](pyproject.toml) and syncs [`uv.lock`](uv.lock), commits, tags `vX.Y.Z`, creates a Gitea release with auto-generated notes, and pushes the tag.
4. [`publish.yml`](.gitea/workflows/publish.yml) builds and publishes the package to the Gitea Package Registry.

Docs-, CI-, and test-only PRs do not need a prefix and will not trigger a release.

### PR title prefixes

| Prefix                    | Bump       | Example               |
| ------------------------- | ---------- | --------------------- |
| `[patch]` or `[fix]`      | patch      | `1.2.3` → `1.2.4`     |
| `[minor]` or `[feat]`     | minor      | `1.2.3` → `1.3.0`     |
| `[major]` or `[breaking]` | major      | `1.2.3` → `2.0.0`     |
| _(none)_                  | no release | docs / CI / deps only |

Example titles:

- `[minor] Add public repository interfaces and subclassable adapter CRUD API`
- `[patch] Fix mypy context manager typing for adapter subclasses`

### Release notes

Release notes are generated from commits since the previous tag (see [`scripts/ci/generate-release-notes.sh`](scripts/ci/generate-release-notes.sh)).

### Manual release

You can still push a `v*.*.*` tag manually; `publish.yml` will build and publish. The current released version is in [`pyproject.toml`](pyproject.toml) (and on the latest `v*.*.*` tag).