Compare commits

..
Author SHA1 Message Date
CI Bot 43c5e53c7f chore(deps): update dependencies [automated]
PR Title Check / check-title (pull_request) Successful in 5s
Test Python Package / unit-tests (pull_request) Successful in 23s
Code Quality Pipeline / code-quality (pull_request) Successful in 57s
Test Python Package / integration-tests (pull_request) Successful in 1m48s
Test Python Package / coverage-report (pull_request) Successful in 18s
2026-07-20 03:01:36 +00:00
brian ac82f71347 Merge pull request '[minor] Add QueueRepositoryInterface with memory and file-backed adapters' (#62) from cursor/queue-adapters into main
Release on merge to main / release (push) Failing after 15s
Test Python Package / unit-tests (push) Successful in 14s
Code Quality Pipeline / code-quality (push) Successful in 28s
Test Python Package / integration-tests (push) Successful in 23s
Test Python Package / coverage-report (push) Successful in 8s
Reviewed-on: #62
2026-07-16 21:27:49 +02:00
Brian Bjarke JensenandCursor 24d5ea14d6 Apply Prettier formatting to README and CHANGELOG.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 15s
Test Python Package / integration-tests (pull_request) Successful in 24s
Code Quality Pipeline / code-quality (pull_request) Successful in 31s
Test Python Package / coverage-report (pull_request) Successful in 12s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-16 21:25:49 +02:00
Brian Bjarke JensenandCursor b8d8176877 Align changelog section headings with Prettier markdown spacing.
Test Python Package / unit-tests (pull_request) Successful in 13s
Test Python Package / coverage-report (pull_request) Successful in 13s
PR Title Check / check-title (pull_request) Successful in 7s
Code Quality Pipeline / code-quality (pull_request) Failing after 29s
Test Python Package / integration-tests (pull_request) Successful in 1m15s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-16 21:24:06 +02:00
Brian Bjarke JensenandCursor a78320b434 Move queue helper functions to staticmethods on their adapters.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 20s
Code Quality Pipeline / code-quality (pull_request) Failing after 32s
Test Python Package / integration-tests (pull_request) Successful in 1m33s
Test Python Package / coverage-report (pull_request) Successful in 16s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-16 21:07:35 +02:00
Brian Bjarke JensenandCursor 1822d886b1 Apply ruff formatting to queue adapter and tests.
PR Title Check / check-title (pull_request) Successful in 11s
Test Python Package / unit-tests (pull_request) Successful in 16s
Code Quality Pipeline / code-quality (pull_request) Failing after 30s
Test Python Package / integration-tests (pull_request) Successful in 34s
Test Python Package / coverage-report (pull_request) Successful in 16s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-16 20:24:58 +02:00
Brian Bjarke JensenandCursor 4f58c32dd6 Add QueueRepositoryInterface with memory and file-backed adapters.
PR Title Check / check-title (pull_request) Successful in 9s
Code Quality Pipeline / code-quality (pull_request) Failing after 53s
Test Python Package / unit-tests (pull_request) Successful in 1m1s
Test Python Package / integration-tests (pull_request) Successful in 1m44s
Test Python Package / coverage-report (pull_request) Successful in 13s
Provide a generic disk-backed FIFO queue with configurable path, retention, and dedup keys so consumers can buffer items across restarts without optional extras.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-16 20:20:52 +02:00
brian 88ea7f06ab Merge pull request 'chore(deps): update dependencies' (#61) from renovate/auto-deps-update into main
Release on merge to main / release (push) Failing after 5s
Test Python Package / integration-tests (push) Failing after 1s
Test Python Package / unit-tests (push) Failing after 2s
Test Python Package / coverage-report (push) Has been skipped
Code Quality Pipeline / code-quality (push) Failing after 1m10s
Build CI Image / build-and-push (push) Successful in 1m40s
Reviewed-on: #61
2026-07-16 20:19:13 +02:00
CI Bot 2b5718ba41 chore(deps): update dependencies [automated]
PR Title Check / check-title (pull_request) Successful in 7s
Code Quality Pipeline / code-quality (pull_request) Successful in 31s
Test Python Package / unit-tests (pull_request) Successful in 15s
Test Python Package / integration-tests (pull_request) Successful in 41s
Test Python Package / coverage-report (pull_request) Successful in 13s
2026-07-13 03:00:38 +00:00
CI Bot b6538f9e91 chore: release v2.2.0 [skip ci] 2026-07-12 16:52:22 +00:00
brian b37b2e6e50 Merge pull request '[minor] Add Postgres table adapter with TableRepositoryInterface' (#58) from cursor/postgres-table-adapter into main
Release on merge to main / release (push) Failing after 9s
Code Quality Pipeline / code-quality (push) Successful in 20s
Test Python Package / unit-tests (push) Successful in 13s
Build CI Image / build-and-push (push) Successful in 38s
Test Python Package / integration-tests (push) Successful in 1m9s
Test Python Package / coverage-report (push) Successful in 13s
Reviewed-on: LilleVemmelund/python-repositories#58
2026-07-12 18:51:59 +02:00
Brian Bjarke JensenandCursor 1babb52d09 Apply ruff formatting to postgres adapter and tests.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 13s
Code Quality Pipeline / code-quality (pull_request) Successful in 24s
Test Python Package / integration-tests (pull_request) Successful in 29s
Test Python Package / coverage-report (pull_request) Successful in 10s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 15:01:58 +02:00
Brian Bjarke JensenandCursor 63ee17d544 Apply prettier formatting to README and CHANGELOG.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 10s
Code Quality Pipeline / code-quality (pull_request) Failing after 18s
Test Python Package / integration-tests (pull_request) Successful in 56s
Test Python Package / coverage-report (pull_request) Successful in 13s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 15:01:08 +02:00
Brian Bjarke JensenandCursor 2e38bd2406 Fix ruff import ordering in postgres test files.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 10s
Code Quality Pipeline / code-quality (pull_request) Failing after 17s
Test Python Package / integration-tests (pull_request) Successful in 47s
Test Python Package / coverage-report (pull_request) Successful in 12s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 15:00:01 +02:00
Brian Bjarke JensenandCursor dc6f8a3e89 Add Postgres table adapter with TableRepositoryInterface.
PR Title Check / check-title (pull_request) Successful in 7s
Code Quality Pipeline / code-quality (pull_request) Failing after 19s
Test Python Package / unit-tests (pull_request) Successful in 47s
Test Python Package / integration-tests (pull_request) Successful in 44s
Test Python Package / coverage-report (pull_request) Successful in 11s
Introduce PostgresAdapter for dict-based row CRUD via psycopg3, including config, lazy exports, unit/integration tests with testcontainers, and an example UserTableRepository.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 14:57:34 +02:00
35 changed files with 3111 additions and 421 deletions
+9
View File
@@ -8,3 +8,12 @@ MINIO_SECRET_KEY=minioadmin
MINIO_BUCKET=my-bucket
MINIO_SECURE=false
MINIO_CREATE_BUCKET_IF_MISSING=true
# Postgres (requires postgres extra)
POSTGRES_URI=postgresql://localhost/mydb
POSTGRES_TABLE=users
POSTGRES_PRIMARY_KEY=id
# File-backed queue (no optional extra)
FILE_QUEUE_PATH=/tmp/python-repositories-queue.jsonl
# FILE_QUEUE_MAX_AGE_HOURS=24
+2 -2
View File
@@ -1,7 +1,7 @@
repos:
# General repository hygiene hooks
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
rev: v6.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
@@ -38,7 +38,7 @@ repos:
# Formatting for Markdown, JSON, and YAML with Prettier
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.1.0
rev: v4.0.0-alpha.8
hooks:
- id: prettier
files: "\\.(md|json|yaml|yml)$"
+16
View File
@@ -7,12 +7,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [2.2.0] - 2026-07-12
### Summary
Add Postgres table adapter with TableRepositoryInterface
### Changed
- b37b2e6 Merge pull request '[minor] Add Postgres table adapter with TableRepositoryInterface' (#58) from cursor/postgres-table-adapter into main
- 1babb52 Apply ruff formatting to postgres adapter and tests.
- 63ee17d Apply prettier formatting to README and CHANGELOG.
- 2e38bd2 Fix ruff import ordering in postgres test files.
- dc6f8a3 Add Postgres table adapter with TableRepositoryInterface.
## [2.1.0] - 2026-07-11
### Summary
Add runtime-checkable Protocol typing to public interfaces
### Changed
- 1cbcc1c Merge pull request '[minor] Add runtime-checkable Protocol typing to public interfaces' (#57) from cursor/protocol-abc-interfaces into main
- a0e5c9d Document test organization and add Cursor workflow rules.
- 44b15cb Move structural typing tests into interface unit test files.
+100 -13
View File
@@ -6,11 +6,11 @@ Subclass an adapter in your own repository to add domain-specific methods while
## Architecture
| Layer | Responsibility |
| ---------------- | ----------------------------------------------------------------- |
| **Interfaces** | Abstract contracts for connection, context, and CRUD |
| **Adapters** | Technology-specific base classes (`RedisAdapter`, `MinioAdapter`) |
| **Your project** | Subclass an adapter and add domain methods |
| Layer | Responsibility |
| ---------------- | ---------------------------------------------------------------------------------------------------- |
| **Interfaces** | Abstract contracts for connection, context, CRUD, and queues |
| **Adapters** | Technology-specific base classes (`RedisAdapter`, `MinioAdapter`, `PostgresAdapter`, queue adapters) |
| **Your project** | Subclass an adapter and add domain methods |
Each public interface is a `@runtime_checkable` `Protocol` with `@abstractmethod` members. **Subclass an adapter** when you need connection management and shared behavior — explicit subclasses get runtime instantiation guards and inherited default methods (e.g. `scan_keys`). **Type-annotate against an interface** when you want loose coupling — any object with the right methods satisfies the contract for mypy and `isinstance()` checks, without inheriting from this package.
@@ -22,14 +22,15 @@ The current API is synchronous. Async repository interfaces and adapters may be
## Optional dependencies
Repository **interfaces** import with the base package. **Adapters** require the matching extra; importing an adapter without its extra raises `ImportError` with install instructions.
Repository **interfaces** import with the base package. Networked **adapters** (`RedisAdapter`, `MinioAdapter`, `PostgresAdapter`) require the matching extra; importing one without its extra raises `ImportError` with install instructions. Queue adapters (`MemoryQueueAdapter`, `FileBackedQueueAdapter`) need no extra.
Install with the extras you need:
```bash
uv add python-repositories[redis]
uv add python-repositories[minio]
uv add python-repositories[redis,minio]
uv add python-repositories[postgres]
uv add python-repositories[redis,minio,postgres]
```
### Redis (`JsonRepositoryInterface`)
@@ -63,16 +64,69 @@ for key in repo.scan_keys("user:*"):
| `MINIO_SECURE` | Use HTTPS (`true`/`false`; default: `true`) |
| `MINIO_CREATE_BUCKET_IF_MISSING` | Auto-create `MINIO_BUCKET` on connect (`true`/`false`; default: `false`) |
Copy [`.env.example`](.env.example) to `.env` for local development. `RedisConfig.from_env()` and `MinioConfig.from_env()` load `.env` automatically when resolving configuration from the environment.
For production, it is recommended to leave `MINIO_CREATE_BUCKET_IF_MISSING` unset so that `connect()` fails fast if the expected bucket is missing. For local development, you will often want `MINIO_SECURE=false` and `MINIO_CREATE_BUCKET_IF_MISSING=true`.
### Postgres (`TableRepositoryInterface`)
Requires PostgreSQL 10 or later; tested against PostgreSQL 16 in CI. Tables are owned by your migrations — the adapter verifies the configured table exists on connect.
| Environment variable | Description |
| ---------------------- | --------------------------------------- |
| `POSTGRES_URI` | PostgreSQL connection URL |
| `POSTGRES_TABLE` | Table name for CRUD operations |
| `POSTGRES_PRIMARY_KEY` | Primary key column name (default: `id`) |
### File-backed queue (`QueueRepositoryInterface`)
In-process and disk-backed FIFO queues for buffering dict items. No optional extra required.
| Adapter | Role |
| ------------------------ | --------------------------------------------------------------------------------- |
| `MemoryQueueAdapter` | Thread-safe in-memory buffer with optional dedup and age eviction |
| `FileBackedQueueAdapter` | Mirrors memory to a JSONL file; replays on `connect()`, compacts on dequeue/evict |
| Environment variable | Description |
| -------------------------- | ----------------------------------------- |
| `FILE_QUEUE_PATH` | Path to the JSONL queue file |
| `FILE_QUEUE_MAX_AGE_HOURS` | Retention window in hours (default: `24`) |
Dedup keys and the age field name are domain-specific: pass them when constructing `FileQueueConfig` (or as kwargs to `FileQueueConfig.from_env(...)`). Callers choose the file path and item schema.
```python
from pathlib import Path
from python_repositories import FileBackedQueueAdapter, FileQueueConfig
config = FileQueueConfig(
path=Path("/var/lib/my-service/queue/items.jsonl"),
max_age_hours=24,
dedup_keys=("id",),
age_key="created_at",
)
with FileBackedQueueAdapter(config=config) as queue:
queue.enqueue([{"id": 1, "created_at": "2024-01-01T00:00:00+00:00"}])
batch = queue.dequeue_batch(max_items=100)
```
Copy [`.env.example`](.env.example) to `.env` for local development. `RedisConfig.from_env()`, `MinioConfig.from_env()`, `PostgresConfig.from_env()`, and `FileQueueConfig.from_env()` load `.env` automatically when resolving configuration from the environment.
## Configuration injection
Adapters accept optional `config` and `client` keyword arguments for explicit setup and testing:
Adapters accept optional `config` and `client` keyword arguments for explicit setup and testing. Queue adapters accept `config` (and an optional injected `memory` buffer for `FileBackedQueueAdapter`):
```python
from python_repositories import RedisAdapter, RedisConfig, MinioAdapter, MinioConfig
from pathlib import Path
from python_repositories import (
FileBackedQueueAdapter,
FileQueueConfig,
MinioAdapter,
MinioConfig,
PostgresAdapter,
PostgresConfig,
RedisAdapter,
RedisConfig,
)
redis = RedisAdapter(config=RedisConfig(uri="redis://localhost:6379"))
minio = MinioAdapter(
@@ -85,6 +139,20 @@ minio = MinioAdapter(
# create_bucket_if_missing=True, # convenient for local dev
)
)
postgres = PostgresAdapter(
config=PostgresConfig(
uri="postgresql://localhost/mydb",
table="users",
primary_key="user_id",
)
)
queue = FileBackedQueueAdapter(
config=FileQueueConfig(
path=Path("/var/lib/my-service/queue/items.jsonl"),
dedup_keys=("id",),
age_key="created_at",
)
)
```
When both `config` and `client` are provided, `connect()` skips client creation (the caller owns the client lifecycle). `config` is required whenever `client` is injected.
@@ -97,7 +165,7 @@ from python_repositories import load_dotenv
load_dotenv() # optional — from_env() also loads .env by default
```
Calling `RedisAdapter()` or `MinioAdapter()` with no arguments still loads configuration from environment variables (and `.env` if present).
Calling `RedisAdapter()`, `MinioAdapter()`, or `PostgresAdapter()` with no arguments still loads configuration from environment variables (and `.env` if present).
## Quick start
@@ -126,6 +194,17 @@ with ArtifactObjectRepository() as repo:
data = repo.get_artifact("report-1")
```
### Relational rows with Postgres
```python
from python_repositories.examples.user_table_repository import UserTableRepository
with UserTableRepository() as repo:
repo.save_user("alice", {"name": "Alice", "email": "alice@example.com"})
user = repo.get_user("alice")
repo.delete_user("alice")
```
### Subclassing in your own project
```python
@@ -150,12 +229,19 @@ class UserRepository(RedisAdapter):
from python_repositories import (
ConnectionAwareInterface,
ContextAwareInterface,
FileBackedQueueAdapter,
FileQueueConfig,
JsonRepositoryInterface,
MemoryQueueAdapter,
MinioAdapter,
MinioConfig,
ObjectRepositoryInterface,
PostgresAdapter,
PostgresConfig,
QueueRepositoryInterface,
RedisAdapter,
RedisConfig,
TableRepositoryInterface,
load_dotenv,
)
```
@@ -169,10 +255,11 @@ uv run pytest tests/unit/ -v # fast, no Docker
uv run pytest -m "not integration" -v # all non-Docker tests
uv run pytest tests/integration/redis/ -v # Redis container only
uv run pytest tests/integration/minio/ -v # MinIO container only
uv run pytest tests/integration/postgres/ -v # Postgres container only
uv run pytest -v # full suite (requires Docker)
```
Integration tests are marked with `@pytest.mark.integration` and require Docker (testcontainers). Backend-specific markers (`needs_redis`, `needs_minio`) let you run only the containers a test module needs. Run unit tests alone for quick local feedback.
Integration tests are marked with `@pytest.mark.integration` and require Docker (testcontainers). Backend-specific markers (`needs_redis`, `needs_minio`, `needs_postgres`) let you run only the containers a test module needs. Run unit tests alone for quick local feedback.
### Test organization
+5 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "python-repositories"
version = "2.1.0"
version = "2.2.0"
description = "Various python repository interfaces exposed as a python package."
authors = [
{ name = "Brian Bjarke Jensen", email = "schnitzelen@gmail.com" }
@@ -32,6 +32,9 @@ redis = [
minio = [
"minio>=7.2.16",
]
postgres = [
"psycopg[binary]>=3.2.0",
]
[build-system]
requires = ["hatchling"]
@@ -44,6 +47,7 @@ markers = [
"integration: tests requiring Docker containers (deselect with '-m \"not integration\"')",
"needs_redis: integration test requiring a Redis container",
"needs_minio: integration test requiring a MinIO container",
"needs_postgres: integration test requiring a Postgres container",
]
[tool.coverage.run]
+18 -1
View File
@@ -6,26 +6,43 @@ from typing import TYPE_CHECKING
# Interfaces are always available; they have no optional backend dependencies.
from . import adapters
from .config import MinioConfig, RedisConfig, load_dotenv
from .config import (
FileQueueConfig,
MinioConfig,
PostgresConfig,
RedisConfig,
load_dotenv,
)
from .interfaces import (
ConnectionAwareInterface,
ContextAwareInterface,
JsonRepositoryInterface,
ObjectRepositoryInterface,
QueueRepositoryInterface,
TableRepositoryInterface,
)
# Adapters are imported only for static type checkers; runtime loading is delegated below.
if TYPE_CHECKING:
from .adapters.file_backed_queue_adapter import (
FileBackedQueueAdapter as FileBackedQueueAdapter,
)
from .adapters.memory_queue_adapter import MemoryQueueAdapter as MemoryQueueAdapter
from .adapters.minio_adapter import MinioAdapter as MinioAdapter
from .adapters.postgres_adapter import PostgresAdapter as PostgresAdapter
from .adapters.redis_adapter import RedisAdapter as RedisAdapter
__all__ = [
"ConnectionAwareInterface",
"ContextAwareInterface",
"FileQueueConfig",
"JsonRepositoryInterface",
"MinioConfig",
"ObjectRepositoryInterface",
"PostgresConfig",
"QueueRepositoryInterface",
"RedisConfig",
"TableRepositoryInterface",
"load_dotenv",
*adapters.__all__,
]
+12
View File
@@ -11,7 +11,10 @@ from typing import TYPE_CHECKING
# Adapters are imported only for static type checkers; runtime loading is deferred below.
if TYPE_CHECKING:
from .file_backed_queue_adapter import FileBackedQueueAdapter
from .memory_queue_adapter import MemoryQueueAdapter
from .minio_adapter import MinioAdapter
from .postgres_adapter import PostgresAdapter
from .redis_adapter import RedisAdapter
# Map public adapter names to their defining module and class.
@@ -20,11 +23,20 @@ if TYPE_CHECKING:
_LAZY_EXPORTS = {
"RedisAdapter": (".redis_adapter", "RedisAdapter"),
"MinioAdapter": (".minio_adapter", "MinioAdapter"),
"PostgresAdapter": (".postgres_adapter", "PostgresAdapter"),
"MemoryQueueAdapter": (".memory_queue_adapter", "MemoryQueueAdapter"),
"FileBackedQueueAdapter": (
".file_backed_queue_adapter",
"FileBackedQueueAdapter",
),
}
__all__ = [
"RedisAdapter",
"MinioAdapter",
"PostgresAdapter",
"MemoryQueueAdapter",
"FileBackedQueueAdapter",
]
@@ -0,0 +1,215 @@
"""File-backed queue adapter: in-memory hot buffer mirrored to JSONL on disk."""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
import json
import os
from pathlib import Path
import threading
from typing import Any, Self, TextIO
import structlog
from python_repositories.adapters.memory_queue_adapter import MemoryQueueAdapter
from python_repositories.config.file_queue_config import FileQueueConfig
from python_repositories.interfaces.queue_repository_interface import (
QueueRepositoryInterface,
)
class FileBackedQueueAdapter(QueueRepositoryInterface):
"""Queue that mirrors an in-memory buffer to an append-friendly JSONL file.
Does not extend ``ConnectionAwareAdapter``. ``connect()`` replays the JSONL
file into memory; ``disconnect()`` flushes any open handle.
"""
def __init__(
self,
*,
config: FileQueueConfig | None = None,
memory: MemoryQueueAdapter | None = None,
) -> None:
if config is None:
config = FileQueueConfig.from_env()
if memory is None:
memory = MemoryQueueAdapter(
dedup_keys=config.dedup_keys,
age_key=config.age_key,
)
self._config = config
self._memory = memory
self._lock = threading.RLock()
self._connected = False
self._append_handle: TextIO | None = None
self.logger = structlog.get_logger(self.__class__.__name__)
@staticmethod
def _json_default(value: object) -> str:
if isinstance(value, datetime):
return value.isoformat()
raise TypeError(f"Object of type {type(value)!r} is not JSON serializable")
def __enter__(self) -> Self:
self.connect()
return self
def __exit__(
self, exc_type: type | None, exc_val: object | None, exc_tb: object | None
) -> None:
del exc_type, exc_val, exc_tb
self.disconnect()
def connect(self) -> None:
"""Create parent directories and replay the JSONL file into memory."""
with self._lock:
if self._connected:
return
path = self._config.path
path.parent.mkdir(parents=True, exist_ok=True)
self._memory.clear()
if path.is_file():
self._replay_file(path)
self._append_handle = path.open("a", encoding="utf-8")
self._connected = True
def disconnect(self) -> None:
"""Flush and close the append handle if open."""
with self._lock:
if self._append_handle is not None:
self._append_handle.flush()
try:
os.fsync(self._append_handle.fileno())
except OSError:
pass
self._append_handle.close()
self._append_handle = None
self._connected = False
def is_connected(self) -> bool:
"""Return whether ``connect()`` has completed successfully."""
return self._connected
def enqueue(self, items: list[dict[str, Any]]) -> None:
"""Append items to memory and the JSONL file; optionally age-evict."""
self._require_connected()
with self._lock:
added = self._memory.enqueue_and_return_added(items)
if added:
self._append_items(added)
if self._config.age_key is not None:
cutoff = datetime.now(UTC) - timedelta(hours=self._config.max_age_hours)
if self._memory.evict_older_than(cutoff):
self._compact()
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
"""Dequeue from memory and compact the JSONL file to match."""
self._require_connected()
with self._lock:
batch = self._memory.dequeue_batch(max_items=max_items)
if batch:
self._compact()
return batch
def size(self) -> int:
"""Return the number of items currently in the in-memory buffer."""
self._require_connected()
return self._memory.size()
def evict_older_than(self, cutoff: datetime) -> int:
"""Evict aged items from memory and compact the JSONL file."""
self._require_connected()
with self._lock:
removed = self._memory.evict_older_than(cutoff)
if removed:
self._compact()
return removed
def _require_connected(self) -> None:
if not self._connected:
raise RuntimeError("FileBackedQueueAdapter is not connected")
def _replay_file(self, path: Path) -> None:
cutoff: datetime | None = None
if self._config.age_key is not None:
cutoff = datetime.now(UTC) - timedelta(hours=self._config.max_age_hours)
with path.open(encoding="utf-8") as handle:
for line_number, line in enumerate(handle, start=1):
stripped = line.strip()
if not stripped:
continue
try:
payload = json.loads(stripped)
except json.JSONDecodeError:
self.logger.warning(
"Skipping corrupt JSONL line",
path=str(path),
line_number=line_number,
)
continue
if not isinstance(payload, dict):
self.logger.warning(
"Skipping non-object JSONL line",
path=str(path),
line_number=line_number,
)
continue
item: dict[str, Any] = payload
if cutoff is not None and self._config.age_key is not None:
age_value = item.get(self._config.age_key)
if age_value is None:
self.logger.warning(
"Skipping item missing age key on replay",
path=str(path),
line_number=line_number,
age_key=self._config.age_key,
)
continue
try:
if MemoryQueueAdapter._parse_age(age_value) < cutoff:
continue
except ValueError:
self.logger.warning(
"Skipping item with invalid age on replay",
path=str(path),
line_number=line_number,
)
continue
try:
self._memory.enqueue_and_return_added([item])
except ValueError as exc:
self.logger.warning(
"Skipping invalid item on replay",
path=str(path),
line_number=line_number,
error=str(exc),
)
def _append_items(self, items: list[dict[str, Any]]) -> None:
if self._append_handle is None:
raise RuntimeError("append handle is not open")
for item in items:
self._append_handle.write(
json.dumps(item, default=self._json_default, separators=(",", ":"))
)
self._append_handle.write("\n")
self._append_handle.flush()
def _compact(self) -> None:
"""Rewrite the JSONL file from the current in-memory snapshot."""
path = self._config.path
tmp_path = path.with_suffix(path.suffix + ".tmp")
if self._append_handle is not None:
self._append_handle.flush()
self._append_handle.close()
self._append_handle = None
with tmp_path.open("w", encoding="utf-8") as handle:
for item in self._memory.snapshot():
handle.write(
json.dumps(item, default=self._json_default, separators=(",", ":"))
)
handle.write("\n")
handle.flush()
tmp_path.replace(path)
self._append_handle = path.open("a", encoding="utf-8")
@@ -0,0 +1,128 @@
"""In-memory queue adapter with optional dedup and age-based eviction."""
from __future__ import annotations
from collections import OrderedDict
from collections.abc import Hashable
from datetime import UTC, datetime
import threading
from typing import Any
from python_repositories.interfaces.queue_repository_interface import (
QueueRepositoryInterface,
)
class MemoryQueueAdapter(QueueRepositoryInterface):
"""Thread-safe in-process FIFO queue with optional deduplication."""
def __init__(
self,
*,
dedup_keys: tuple[str, ...] = (),
age_key: str | None = None,
) -> None:
self._dedup_keys = dedup_keys
self._age_key = age_key
self._lock = threading.RLock()
self._items: OrderedDict[Hashable, dict[str, Any]] = OrderedDict()
self._seq = 0
@staticmethod
def _parse_age(value: object) -> datetime:
"""Normalize an age field to a timezone-aware UTC datetime."""
if isinstance(value, datetime):
if value.tzinfo is None:
return value.replace(tzinfo=UTC)
return value.astimezone(UTC)
if isinstance(value, str):
normalized = value.replace("Z", "+00:00")
parsed = datetime.fromisoformat(normalized)
if parsed.tzinfo is None:
return parsed.replace(tzinfo=UTC)
return parsed.astimezone(UTC)
raise ValueError(
f"age value must be datetime or ISO-8601 str, got {type(value)!r}"
)
def enqueue(self, items: list[dict[str, Any]]) -> None:
"""Append items; skip duplicates when ``dedup_keys`` is configured."""
self.enqueue_and_return_added(items)
def enqueue_and_return_added(
self, items: list[dict[str, Any]]
) -> list[dict[str, Any]]:
"""Enqueue items and return the subset that was newly stored."""
if not items:
return []
added: list[dict[str, Any]] = []
with self._lock:
for raw in items:
item = self._normalize_item(raw)
key = self._make_key(item)
if self._dedup_keys and key in self._items:
continue
self._items[key] = item
added.append(item)
return added
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
"""Remove and return up to ``max_items`` items in FIFO order."""
if max_items < 0:
raise ValueError("max_items must be >= 0")
with self._lock:
batch: list[dict[str, Any]] = []
for _ in range(min(max_items, len(self._items))):
_key, item = self._items.popitem(last=False)
batch.append(item)
return batch
def size(self) -> int:
"""Return the number of items currently in the queue."""
with self._lock:
return len(self._items)
def evict_older_than(self, cutoff: datetime) -> int:
"""Remove items whose age field is strictly older than ``cutoff``."""
if self._age_key is None:
return 0
cutoff_utc = self._parse_age(cutoff)
removed = 0
with self._lock:
to_remove = [
key
for key, item in self._items.items()
if self._parse_age(item[self._age_key]) < cutoff_utc
]
for key in to_remove:
del self._items[key]
removed += 1
return removed
def clear(self) -> None:
"""Remove all items from the queue."""
with self._lock:
self._items.clear()
def snapshot(self) -> list[dict[str, Any]]:
"""Return a shallow copy of queued items in FIFO order."""
with self._lock:
return [dict(item) for item in self._items.values()]
def _normalize_item(self, raw: dict[str, Any]) -> dict[str, Any]:
item = dict(raw)
if self._dedup_keys:
missing = [key for key in self._dedup_keys if key not in item]
if missing:
raise ValueError(f"item missing dedup key(s): {missing}")
if self._age_key is not None:
if self._age_key not in item:
raise ValueError(f"item missing age key: {self._age_key!r}")
item[self._age_key] = self._parse_age(item[self._age_key])
return item
def _make_key(self, item: dict[str, Any]) -> Hashable:
if not self._dedup_keys:
self._seq += 1
return self._seq
return tuple(item[key] for key in self._dedup_keys)
@@ -0,0 +1,230 @@
"""Definition of PostgresAdapter class."""
from __future__ import annotations
from typing import Any
from python_repositories.adapters.connection_aware_adapter import (
ConnectionAwareAdapter,
)
from python_repositories.config import PostgresConfig
from python_repositories.interfaces import TableRepositoryInterface
try:
import psycopg
from psycopg import sql
from psycopg.errors import UndefinedTable
from psycopg.rows import dict_row
except ImportError as exc:
raise ImportError(
"Postgres support requires the postgres extra. "
"Install with: pip install python-repositories[postgres]"
) from exc
class PostgresAdapter(ConnectionAwareAdapter, TableRepositoryInterface):
"""Postgres adapter exposing basic table CRUD functionality."""
uri_env_var_name: str = "POSTGRES_URI"
table_env_var_name: str = "POSTGRES_TABLE"
primary_key_env_var_name: str = "POSTGRES_PRIMARY_KEY"
connection_name: str = "Postgres"
def __init__(
self,
*,
config: PostgresConfig | None = None,
client: psycopg.Connection[Any] | None = None,
) -> None:
super().__init__()
if client is not None and config is None:
raise ValueError("config is required when client is provided")
if config is None:
config = PostgresConfig.from_env(
self.uri_env_var_name,
self.table_env_var_name,
self.primary_key_env_var_name,
)
self._config = config
self._client_injected = client is not None
self._client: psycopg.Connection[Any] | None = client
if self._client_injected:
self._invalidate_health_cache()
def _is_client_ready(self) -> bool:
return self._client is not None
def _table_identifier(self) -> sql.Identifier:
return sql.Identifier(self._config.table)
def _primary_key_identifier(self) -> sql.Identifier:
return sql.Identifier(self._config.primary_key)
def _validate_injected_client(self) -> None:
if self._client is None:
return
try:
self._run_connection_probe()
except (psycopg.Error, ConnectionError) as exc:
raise ConnectionError(
f"Could not connect to Postgres at {self._config.uri}"
) from exc
def _establish_connection(self) -> None:
uri = self._config.uri
try:
client = psycopg.connect(
uri,
row_factory=dict_row,
connect_timeout=10,
autocommit=True,
)
self._client = client
self._run_connection_probe()
except ConnectionError:
self._client = None
raise
except psycopg.Error as exc:
self._client = None
raise ConnectionError(f"Could not connect to Postgres at {uri}") from exc
def _run_connection_probe(self) -> None:
assert self._client is not None
with self._client.cursor() as cursor:
cursor.execute("SELECT 1")
try:
cursor.execute(
sql.SQL("SELECT 1 FROM {} LIMIT 0").format(self._table_identifier())
)
except UndefinedTable as exc:
raise ConnectionError(
f"Table '{self._config.table}' does not exist on Postgres at "
f"{self._config.uri}"
) from exc
def disconnect(self) -> None:
"""Disconnect from the Postgres server."""
if self._client is not None and not self._client_injected:
self._client.close()
self._client = None
self._invalidate_health_cache()
def _probe_connection(self) -> bool:
assert self._client is not None
try:
self._run_connection_probe()
except (psycopg.Error, ConnectionError):
return False
return True
def _validate_pk(self, pk: Any) -> None:
if pk is None:
raise ValueError("Primary key must not be None")
def _validate_row(self, row: dict[str, Any]) -> None:
if not isinstance(row, dict):
raise ValueError("Row must be a dictionary")
if self._config.primary_key not in row:
raise ValueError(
f"Row must include primary key column '{self._config.primary_key}'"
)
def _validate_limit(self, limit: int | None) -> None:
if limit is not None and (not isinstance(limit, int) or limit < 0):
raise ValueError("Limit must be a non-negative integer or None")
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
"""Fetch a single row by primary key."""
self._validate_pk(pk)
self._require_connected()
assert self._client is not None
query = sql.SQL("SELECT * FROM {} WHERE {} = %s").format(
self._table_identifier(),
self._primary_key_identifier(),
)
with self._client.cursor() as cursor:
cursor.execute(query, (pk,))
row = cursor.fetchone()
self.logger.debug("Fetched row", pk=pk, found=row is not None)
return row
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
"""Fetch all rows from the configured table."""
self._validate_limit(limit)
self._require_connected()
assert self._client is not None
query = sql.SQL("SELECT * FROM {}").format(self._table_identifier())
params: tuple[Any, ...] = ()
if limit is not None:
query = sql.Composed([query, sql.SQL(" LIMIT %s")])
params = (limit,)
with self._client.cursor() as cursor:
cursor.execute(query, params)
rows = cursor.fetchall()
self.logger.debug("Fetched rows", count=len(rows), limit=limit)
return list(rows)
def upsert(self, row: dict[str, Any]) -> None:
"""Insert or update a row by primary key."""
self._validate_row(row)
self._require_connected()
assert self._client is not None
pk_col = self._config.primary_key
columns = list(row.keys())
identifiers = [sql.Identifier(column) for column in columns]
placeholders = sql.SQL(", ").join(sql.Placeholder() * len(columns))
column_list = sql.SQL(", ").join(identifiers)
update_columns = [column for column in columns if column != pk_col]
on_conflict: sql.Composed | sql.SQL
if update_columns:
update_assignments = sql.SQL(", ").join(
sql.SQL("{} = EXCLUDED.{}").format(
sql.Identifier(column),
sql.Identifier(column),
)
for column in update_columns
)
on_conflict = sql.SQL("DO UPDATE SET {}").format(update_assignments)
else:
on_conflict = sql.SQL("DO NOTHING")
query = sql.SQL("INSERT INTO {} ({}) VALUES ({}) ON CONFLICT ({}) {}").format(
self._table_identifier(),
column_list,
placeholders,
self._primary_key_identifier(),
on_conflict,
)
with self._client.cursor() as cursor:
cursor.execute(query, tuple(row[column] for column in columns))
self.logger.debug("Upserted row", pk=row[pk_col], columns=columns)
def delete(self, pk: Any) -> None:
"""Delete a row by primary key."""
self._validate_pk(pk)
self._require_connected()
assert self._client is not None
query = sql.SQL("DELETE FROM {} WHERE {} = %s").format(
self._table_identifier(),
self._primary_key_identifier(),
)
with self._client.cursor() as cursor:
cursor.execute(query, (pk,))
self.logger.debug("Deleted row", pk=pk)
def execute(
self,
sql_text: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
"""Run a read SQL statement and return rows as dicts."""
if not isinstance(sql_text, str) or len(sql_text) == 0:
raise ValueError("SQL must be a non-empty string")
if not isinstance(params, tuple):
raise ValueError("Params must be a tuple")
self._require_connected()
assert self._client is not None
with self._client.cursor() as cursor:
cursor.execute(sql_text, params)
rows = cursor.fetchall()
self.logger.debug("Executed SQL", row_count=len(rows))
return list(rows)
+4
View File
@@ -1,9 +1,13 @@
from .dotenv_loader import load_dotenv as load_dotenv
from .file_queue_config import FileQueueConfig as FileQueueConfig
from .minio_config import MinioConfig as MinioConfig
from .postgres_config import PostgresConfig as PostgresConfig
from .redis_config import RedisConfig as RedisConfig
__all__ = [
"FileQueueConfig",
"MinioConfig",
"PostgresConfig",
"RedisConfig",
"load_dotenv",
]
@@ -0,0 +1,56 @@
"""File-backed queue configuration."""
from __future__ import annotations
from dataclasses import dataclass
import os
from pathlib import Path
from python_utils import check_env
from python_repositories.config.dotenv_loader import load_dotenv
_DEFAULT_MAX_AGE_HOURS = 24
@dataclass(frozen=True)
class FileQueueConfig:
"""Configuration for a JSONL file-backed queue.
Path and key schema are owned by the caller. This package does not assume
any particular directory layout or item field names.
"""
path: Path
max_age_hours: int = _DEFAULT_MAX_AGE_HOURS
dedup_keys: tuple[str, ...] = ()
age_key: str | None = None
@classmethod
def from_env(
cls,
path_env_var_name: str = "FILE_QUEUE_PATH",
*,
max_age_hours_env_var_name: str = "FILE_QUEUE_MAX_AGE_HOURS",
dedup_keys: tuple[str, ...] = (),
age_key: str | None = None,
use_dotenv: bool = True,
) -> FileQueueConfig:
"""Load path and optional max age from environment variables.
``dedup_keys`` and ``age_key`` are domain-specific and must be passed
explicitly; they are not read from the environment.
"""
if use_dotenv:
load_dotenv()
check_env(path_env_var_name)
raw_max_age = os.getenv(max_age_hours_env_var_name)
max_age_hours = (
_DEFAULT_MAX_AGE_HOURS if raw_max_age is None else int(raw_max_age)
)
return cls(
path=Path(str(os.getenv(path_env_var_name))),
max_age_hours=max_age_hours,
dedup_keys=dedup_keys,
age_key=age_key,
)
@@ -0,0 +1,42 @@
"""PostgreSQL connection configuration."""
from __future__ import annotations
from dataclasses import dataclass
import os
from python_utils import check_env
from python_repositories.config.dotenv_loader import load_dotenv
@dataclass(frozen=True)
class PostgresConfig:
"""Configuration for connecting to PostgreSQL."""
uri: str
table: str
primary_key: str = "id"
@classmethod
def from_env(
cls,
uri_env_var_name: str = "POSTGRES_URI",
table_env_var_name: str = "POSTGRES_TABLE",
primary_key_env_var_name: str = "POSTGRES_PRIMARY_KEY",
*,
use_dotenv: bool = True,
) -> PostgresConfig:
"""Load configuration from environment variables."""
if use_dotenv:
load_dotenv()
env_var_names = {uri_env_var_name, table_env_var_name}
check_env(env_var_names)
primary_key = os.getenv(primary_key_env_var_name)
if primary_key is None or primary_key == "":
primary_key = "id"
return cls(
uri=str(os.getenv(uri_env_var_name)),
table=str(os.getenv(table_env_var_name)),
primary_key=primary_key,
)
@@ -0,0 +1,19 @@
"""Example domain repository backed by Postgres tables."""
from typing import Any
from python_repositories.adapters.postgres_adapter import PostgresAdapter
class UserTableRepository(PostgresAdapter):
"""Example: domain repository backed by a Postgres table."""
def get_user(self, user_id: str) -> dict[str, Any] | None:
return self.fetch_one(user_id)
def save_user(self, user_id: str, user: dict[str, Any]) -> None:
row = {self._config.primary_key: user_id, **user}
self.upsert(row)
def delete_user(self, user_id: str) -> None:
self.delete(user_id)
@@ -8,10 +8,18 @@ from .json_repository_interface import (
from .object_repository_interface import (
ObjectRepositoryInterface as ObjectRepositoryInterface,
)
from .queue_repository_interface import (
QueueRepositoryInterface as QueueRepositoryInterface,
)
from .table_repository_interface import (
TableRepositoryInterface as TableRepositoryInterface,
)
__all__ = [
"ConnectionAwareInterface",
"ContextAwareInterface",
"JsonRepositoryInterface",
"ObjectRepositoryInterface",
"QueueRepositoryInterface",
"TableRepositoryInterface",
]
@@ -0,0 +1,41 @@
"""Definition of QueueRepositoryInterface protocol."""
from abc import abstractmethod
from datetime import datetime
from typing import Any, Protocol, runtime_checkable
@runtime_checkable
class QueueRepositoryInterface(Protocol):
"""Interface that defines buffered queue enqueue/dequeue methods."""
@abstractmethod
def enqueue(self, items: list[dict[str, Any]]) -> None:
"""Append items to the queue.
Duplicates may be skipped when the implementation is configured with
dedup keys. An empty list is a no-op.
"""
...
@abstractmethod
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
"""Remove and return up to ``max_items`` items in FIFO order.
Returns an empty list when the queue is empty.
"""
...
@abstractmethod
def size(self) -> int:
"""Return the number of items currently in the queue."""
...
@abstractmethod
def evict_older_than(self, cutoff: datetime) -> int:
"""Remove items older than ``cutoff`` and return how many were removed.
Age is determined by an implementation-specific item field. When no age
field is configured, this is a no-op that returns ``0``.
"""
...
@@ -0,0 +1,55 @@
"""Definition of TableRepositoryInterface protocol."""
from abc import abstractmethod
from typing import Any, Protocol, runtime_checkable
@runtime_checkable
class TableRepositoryInterface(Protocol):
"""Interface that defines relational table CRUD methods."""
@abstractmethod
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
"""Fetch a single row by primary key.
Returns ``None`` when no row matches. Use ``value is not None`` to test
existence; avoid truthiness checks.
"""
...
@abstractmethod
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
"""Fetch all rows from the configured table.
Returns an empty list when the table has no rows.
"""
...
@abstractmethod
def upsert(self, row: dict[str, Any]) -> None:
"""Insert or update a row by primary key.
``row`` must include the configured primary key column. Raises
``ValueError`` for a non-dict row or a missing primary key.
"""
...
@abstractmethod
def delete(self, pk: Any) -> None:
"""Delete a row by primary key.
Idempotent: no error when the row is already absent.
"""
...
@abstractmethod
def execute(
self,
sql: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
"""Run a read SQL statement and return rows as dicts.
Escape hatch for joins, filters, and other queries in subclasses.
"""
...
+4
View File
@@ -103,13 +103,17 @@ changelog_section() {
local summary="$3"
local commits="$4"
# Blank lines after ATX headings match Prettier v4 markdown formatting
# (see mirrors-prettier in .pre-commit-config.yaml).
cat <<EOF
## [${version}] - ${date}
### Summary
${summary}
### Changed
${commits:-- (no commits recorded)}
EOF
}
+6 -1
View File
@@ -1,6 +1,6 @@
"""Shared test configuration constants."""
from python_repositories.config import MinioConfig, RedisConfig
from python_repositories.config import MinioConfig, PostgresConfig, RedisConfig
TEST_REDIS_CONFIG = RedisConfig(uri="redis://localhost:6379")
TEST_MINIO_CONFIG = MinioConfig(
@@ -10,3 +10,8 @@ TEST_MINIO_CONFIG = MinioConfig(
bucket="test-bucket",
secure=False,
)
TEST_POSTGRES_CONFIG = PostgresConfig(
uri="postgresql://localhost/mydb",
table="test_items",
primary_key="id",
)
+26 -1
View File
@@ -6,12 +6,17 @@ from minio import Minio
import pytest
import redis
from python_repositories.config import MinioConfig, RedisConfig
from python_repositories.config import MinioConfig, PostgresConfig, RedisConfig
from tests.integration.minio._containers import (
minio_config_from_env,
minio_env,
raw_minio_client_from_env,
)
from tests.integration.postgres._containers import (
postgres_config_from_container,
postgres_uri,
raw_postgres_client_from_container,
)
from tests.integration.redis._containers import (
raw_redis_client_from_container,
redis_config_from_container,
@@ -53,3 +58,23 @@ def minio_config(minio_container: dict[str, str]) -> MinioConfig:
def raw_minio_client(minio_container: dict[str, str]) -> Generator[Minio, None, None]:
"""Provide a raw Minio client connected to the test Minio container."""
yield from raw_minio_client_from_env(minio_container)
@pytest.fixture(scope="session")
def postgres_container() -> Generator[str, None, None]:
"""Set up a Postgres container for testing and yield the Postgres URI."""
yield from postgres_uri()
@pytest.fixture(scope="session")
def postgres_config(postgres_container: str) -> PostgresConfig:
"""Provide PostgresConfig built from the test container."""
return postgres_config_from_container(postgres_container)
@pytest.fixture(scope="session")
def raw_postgres_client(
postgres_container: str,
) -> Generator[object, None, None]:
"""Provide a raw Postgres client connected to the test Postgres container."""
yield from raw_postgres_client_from_container(postgres_container)
@@ -6,12 +6,15 @@ import os
import random
from minio import Minio
import psycopg
import pytest
from python_repositories.examples.artifact_object_repository import (
ArtifactObjectRepository,
)
from python_repositories.examples.user_json_repository import UserJsonRepository
from python_repositories.examples.user_table_repository import UserTableRepository
from tests.integration.postgres._containers import TEST_TABLE
pytestmark = [
pytest.mark.integration,
@@ -36,6 +39,24 @@ def set_example_env(
_ = os.environ.pop(key, default=None)
@pytest.fixture(scope="module", autouse=True)
def set_postgres_example_env(
postgres_container: str,
raw_postgres_client: psycopg.Connection,
) -> Generator[None, None, None]:
"""Set Postgres env vars for example repositories using from_env() defaults."""
env_vars = {
"POSTGRES_URI": postgres_container,
"POSTGRES_TABLE": TEST_TABLE,
"POSTGRES_PRIMARY_KEY": "id",
}
for key, value in env_vars.items():
os.environ[key] = value
yield
for key in env_vars:
_ = os.environ.pop(key, default=None)
@pytest.fixture(scope="module")
def user_data() -> Generator[dict[str, str], None, None]:
"""Provide sample user data for tests."""
@@ -77,3 +98,27 @@ def test_artifact_object_repository_store_and_get(
assert received is not None
artifact_data.seek(0)
assert received.read() == artifact_data.read()
@pytest.mark.needs_postgres
def test_user_table_repository_save_and_get(
user_data: dict[str, str],
) -> None:
"""Test that UserTableRepository can save and retrieve a user."""
with UserTableRepository() as repo:
repo.save_user("alice", user_data)
user = repo.get_user("alice")
assert user is not None
assert user["name"] == user_data["name"]
assert user["email"] == user_data["email"]
@pytest.mark.needs_postgres
def test_user_table_repository_delete(
user_data: dict[str, str],
) -> None:
"""Test that UserTableRepository can delete a user."""
with UserTableRepository() as repo:
repo.save_user("alice", user_data)
repo.delete_user("alice")
assert repo.get_user("alice") is None
+76
View File
@@ -0,0 +1,76 @@
"""Postgres container session helpers for integration tests."""
from collections.abc import Generator
import psycopg
from psycopg.rows import dict_row
from testcontainers.postgres import PostgresContainer
from python_repositories.config import PostgresConfig
POSTGRES_IMAGE = "postgres:16"
TEST_TABLE = "test_items"
_postgres_uri: str | None = None
_postgres_container: PostgresContainer | None = None
_raw_postgres_client: psycopg.Connection | None = None
_postgres_uri_refs = 0
_raw_postgres_client_refs = 0
def postgres_uri() -> Generator[str, None, None]:
"""Yield a session-scoped Postgres URI, starting the container once."""
global _postgres_uri, _postgres_container, _postgres_uri_refs
if _postgres_uri is None:
_postgres_container = PostgresContainer(POSTGRES_IMAGE, driver=None)
_postgres_container.start()
_postgres_uri = _postgres_container.get_connection_url()
_postgres_uri_refs += 1
yield _postgres_uri
_postgres_uri_refs -= 1
if _postgres_uri_refs == 0 and _postgres_container is not None:
_postgres_container.stop()
_postgres_container = None
_postgres_uri = None
def postgres_config_from_container(uri: str) -> PostgresConfig:
"""Build PostgresConfig from a container URI."""
return PostgresConfig(uri=uri, table=TEST_TABLE, primary_key="id")
def raw_postgres_client_from_container(
uri: str,
) -> Generator[psycopg.Connection, None, None]:
"""Yield a session-scoped raw Postgres client, reusing one client per session."""
global _raw_postgres_client, _raw_postgres_client_refs
if _raw_postgres_client is None:
_raw_postgres_client = psycopg.connect(
uri,
row_factory=dict_row, # type: ignore[arg-type]
autocommit=True,
)
with _raw_postgres_client.cursor() as cursor:
cursor.execute(
f"""
CREATE TABLE IF NOT EXISTS {TEST_TABLE} (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
value INTEGER,
email TEXT
)
"""
)
_raw_postgres_client_refs += 1
yield _raw_postgres_client
_raw_postgres_client_refs -= 1
if _raw_postgres_client_refs == 0 and _raw_postgres_client is not None:
with _raw_postgres_client.cursor() as cursor:
cursor.execute(f"TRUNCATE TABLE {TEST_TABLE}")
_raw_postgres_client.close()
_raw_postgres_client = None
+33
View File
@@ -0,0 +1,33 @@
"""Postgres integration test fixtures."""
from collections.abc import Generator
import psycopg
import pytest
from python_repositories.config import PostgresConfig
from tests.integration.postgres._containers import (
postgres_config_from_container,
postgres_uri,
raw_postgres_client_from_container,
)
@pytest.fixture(scope="session")
def postgres_container() -> Generator[str, None, None]:
"""Set up a Postgres container for testing and yield the Postgres URI."""
yield from postgres_uri()
@pytest.fixture(scope="session")
def postgres_config(postgres_container: str) -> PostgresConfig:
"""Provide PostgresConfig built from the test container."""
return postgres_config_from_container(postgres_container)
@pytest.fixture(scope="session")
def raw_postgres_client(
postgres_container: str,
) -> Generator[psycopg.Connection, None, None]:
"""Provide a raw Postgres client connected to the test container."""
yield from raw_postgres_client_from_container(postgres_container)
@@ -0,0 +1,318 @@
"""Integration tests for the PostgresAdapter."""
from collections.abc import Generator
import logging
import psycopg
import pytest
from python_repositories.adapters.postgres_adapter import PostgresAdapter
from python_repositories.config import PostgresConfig
from tests.integration.postgres._containers import TEST_TABLE
pytestmark = [pytest.mark.integration, pytest.mark.needs_postgres]
@pytest.fixture(scope="module")
def row() -> Generator[dict[str, object], None, None]:
"""Provide a sample row for tests."""
yield {"id": "test-id", "name": "Alice", "value": 42}
@pytest.fixture(scope="function")
def row_in_postgres(
raw_postgres_client: psycopg.Connection,
row: dict[str, object],
) -> Generator[tuple[str, dict[str, object]], None, None]:
"""Fixture to set up a known row in Postgres before each test."""
with raw_postgres_client.cursor() as cursor:
cursor.execute(
f"INSERT INTO {TEST_TABLE} (id, name, value) VALUES (%s, %s, %s)",
(row["id"], row["name"], row["value"]),
)
yield str(row["id"]), row
with raw_postgres_client.cursor() as cursor:
cursor.execute(f"DELETE FROM {TEST_TABLE} WHERE id = %s", (row["id"],))
@pytest.fixture(scope="module")
def postgres_adapter(
postgres_config: PostgresConfig,
raw_postgres_client: psycopg.Connection,
) -> Generator[PostgresAdapter, None, None]:
"""Fixture to provide a connected PostgresAdapter instance."""
_ = raw_postgres_client # ensure test table exists before connect probe
adapter = PostgresAdapter(config=postgres_config)
adapter.connect()
yield adapter
adapter.disconnect()
@pytest.fixture(scope="function", autouse=True)
def clear_postgres(raw_postgres_client: psycopg.Connection) -> None:
"""Fixture to clear all rows before each test."""
with raw_postgres_client.cursor() as cursor:
cursor.execute(f"TRUNCATE TABLE {TEST_TABLE}")
def test_should_log_info_when_already_connected(
postgres_adapter: PostgresAdapter,
caplog: pytest.LogCaptureFixture,
) -> None:
"""Test that PostgresAdapter logs info when connect is called while connected."""
with caplog.at_level(logging.INFO):
postgres_adapter.connect()
assert "Already connected" in caplog.text
assert "Postgres" in caplog.text
def test_should_raise_connection_error_when_unable_to_connect() -> None:
"""Test that PostgresAdapter raises ConnectionError when unable to connect."""
adapter = PostgresAdapter(
config=PostgresConfig(
uri="postgresql://invalid:5432/nodb",
table=TEST_TABLE,
)
)
with pytest.raises(ConnectionError):
adapter.connect()
assert adapter._client is None
assert not adapter.is_connected()
def test_should_raise_connection_error_when_table_missing(
postgres_config: PostgresConfig,
) -> None:
"""Test that PostgresAdapter raises ConnectionError when the table is missing."""
config = PostgresConfig(
uri=postgres_config.uri,
table="missing_table",
primary_key="id",
)
adapter = PostgresAdapter(config=config)
with pytest.raises(ConnectionError, match="does not exist"):
adapter.connect()
def test_should_log_error_on_exception_during_exit(
postgres_config: PostgresConfig,
caplog: pytest.LogCaptureFixture,
) -> None:
"""Test that PostgresAdapter logs an error when an exception occurs during exit."""
try:
with PostgresAdapter(config=postgres_config) as adapter:
assert adapter.is_connected()
raise ValueError("Simulated error")
except ValueError:
pass
assert "Error while exiting context" in caplog.text
def test_should_have_context_manager(postgres_config: PostgresConfig) -> None:
"""Test that PostgresAdapter can be used as a context manager."""
with PostgresAdapter(config=postgres_config) as adapter:
assert adapter._client is not None
assert adapter._client is None
def test_should_fetch_one(
row_in_postgres: tuple[str, dict[str, object]],
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter can fetch a row by primary key."""
pk, row = row_in_postgres
fetched = postgres_adapter.fetch_one(pk)
assert fetched is not None
assert fetched["id"] == row["id"]
assert fetched["name"] == row["name"]
assert fetched["value"] == row["value"]
def test_should_fetch_none_for_missing_row(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that fetching a non-existent row returns None."""
assert postgres_adapter.fetch_one("missing-id") is None
def test_should_raise_value_error_on_invalid_fetch_one_pk(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter raises ValueError for an invalid primary key."""
with pytest.raises(ValueError):
postgres_adapter.fetch_one(None)
def test_should_raise_connection_error_on_fetch_one_when_not_connected(
postgres_config: PostgresConfig,
) -> None:
"""Test that fetch_one raises ConnectionError when not connected."""
adapter = PostgresAdapter(config=postgres_config)
with pytest.raises(ConnectionError):
adapter.fetch_one("some-id")
def test_should_upsert_row(
row: dict[str, object],
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter can insert a row."""
assert postgres_adapter.fetch_one(row["id"]) is None
postgres_adapter.upsert(row)
fetched = postgres_adapter.fetch_one(row["id"])
assert fetched is not None
assert fetched["name"] == row["name"]
def test_should_update_row(
row_in_postgres: tuple[str, dict[str, object]],
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter can update an existing row."""
pk, _ = row_in_postgres
new_row = {"id": pk, "name": "Bob", "value": 99}
postgres_adapter.upsert(new_row)
fetched = postgres_adapter.fetch_one(pk)
assert fetched is not None
assert fetched["name"] == "Bob"
assert fetched["value"] == 99
def test_should_raise_value_error_on_invalid_upsert_row(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter raises ValueError for invalid upsert data."""
with pytest.raises(ValueError):
postgres_adapter.upsert({"name": "Alice"}) # missing primary key
def test_should_raise_value_error_on_non_dict_upsert_row(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter raises ValueError when upsert row is not a dict."""
with pytest.raises(ValueError):
postgres_adapter.upsert("not-a-dict") # type: ignore[arg-type]
def test_should_raise_connection_error_on_upsert_when_not_connected(
postgres_config: PostgresConfig,
row: dict[str, object],
) -> None:
"""Test that upsert raises ConnectionError when not connected."""
adapter = PostgresAdapter(config=postgres_config)
with pytest.raises(ConnectionError):
adapter.upsert(row)
def test_should_fetch_all_rows(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter can fetch all rows."""
postgres_adapter.upsert({"id": "a", "name": "Alice", "value": 1})
postgres_adapter.upsert({"id": "b", "name": "Bob", "value": 2})
rows = postgres_adapter.fetch_all()
assert len(rows) == 2
ids = {row["id"] for row in rows}
assert ids == {"a", "b"}
def test_should_fetch_all_with_limit(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter respects fetch_all limit."""
postgres_adapter.upsert({"id": "a", "name": "Alice", "value": 1})
postgres_adapter.upsert({"id": "b", "name": "Bob", "value": 2})
rows = postgres_adapter.fetch_all(limit=1)
assert len(rows) == 1
def test_should_return_empty_list_for_empty_table(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that fetch_all returns an empty list for an empty table."""
assert postgres_adapter.fetch_all() == []
def test_should_raise_value_error_on_invalid_fetch_all_limit(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter raises ValueError for an invalid limit."""
with pytest.raises(ValueError):
postgres_adapter.fetch_all(limit=-1)
def test_should_raise_connection_error_on_fetch_all_when_not_connected(
postgres_config: PostgresConfig,
) -> None:
"""Test that fetch_all raises ConnectionError when not connected."""
adapter = PostgresAdapter(config=postgres_config)
with pytest.raises(ConnectionError):
adapter.fetch_all()
def test_should_delete_row(
row_in_postgres: tuple[str, dict[str, object]],
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that deleting a row removes it from the table."""
pk, _ = row_in_postgres
assert postgres_adapter.fetch_one(pk) is not None
postgres_adapter.delete(pk)
assert postgres_adapter.fetch_one(pk) is None
def test_should_delete_idempotently(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that deleting a missing row does not raise."""
postgres_adapter.delete("missing-id")
def test_should_raise_value_error_on_invalid_delete_pk(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter raises ValueError for an invalid delete pk."""
with pytest.raises(ValueError):
postgres_adapter.delete(None)
def test_should_raise_connection_error_on_delete_when_not_connected(
postgres_config: PostgresConfig,
) -> None:
"""Test that delete raises ConnectionError when not connected."""
adapter = PostgresAdapter(config=postgres_config)
with pytest.raises(ConnectionError):
adapter.delete("some-id")
def test_should_execute_sql(
row_in_postgres: tuple[str, dict[str, object]],
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter can execute read SQL."""
pk, _ = row_in_postgres
rows = postgres_adapter.execute(
f"SELECT * FROM {TEST_TABLE} WHERE id = %s",
(pk,),
)
assert len(rows) == 1
assert rows[0]["id"] == pk
def test_should_raise_value_error_on_invalid_execute_sql(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter raises ValueError for invalid SQL."""
with pytest.raises(ValueError):
postgres_adapter.execute("")
def test_should_raise_connection_error_on_execute_when_not_connected(
postgres_config: PostgresConfig,
) -> None:
"""Test that execute raises ConnectionError when not connected."""
adapter = PostgresAdapter(config=postgres_config)
with pytest.raises(ConnectionError):
adapter.execute("SELECT 1")
+47 -2
View File
@@ -45,6 +45,9 @@ from python_repositories import JsonRepositoryInterface
assert JsonRepositoryInterface is not None
assert "python_repositories.adapters.redis_adapter" not in sys.modules
assert "python_repositories.adapters.minio_adapter" not in sys.modules
assert "python_repositories.adapters.postgres_adapter" not in sys.modules
assert "python_repositories.adapters.memory_queue_adapter" not in sys.modules
assert "python_repositories.adapters.file_backed_queue_adapter" not in sys.modules
"""
result = subprocess.run(
[sys.executable, "-c", script],
@@ -58,10 +61,19 @@ assert "python_repositories.adapters.minio_adapter" not in sys.modules
def test_lazy_adapter_load_succeeds_when_extra_present() -> None:
"""Adapters load when their optional dependencies are installed."""
from python_repositories import MinioAdapter, RedisAdapter
from python_repositories import (
FileBackedQueueAdapter,
MemoryQueueAdapter,
MinioAdapter,
PostgresAdapter,
RedisAdapter,
)
assert RedisAdapter.__name__ == "RedisAdapter"
assert MinioAdapter.__name__ == "MinioAdapter"
assert PostgresAdapter.__name__ == "PostgresAdapter"
assert MemoryQueueAdapter.__name__ == "MemoryQueueAdapter"
assert FileBackedQueueAdapter.__name__ == "FileBackedQueueAdapter"
def test_redis_adapter_import_error_without_extra() -> None:
@@ -86,6 +98,17 @@ def test_minio_adapter_import_error_without_extra() -> None:
importlib.reload(minio_adapter_module)
def test_postgres_adapter_import_error_without_extra() -> None:
"""Missing postgres extra raises ImportError with install hint."""
import python_repositories.adapters.postgres_adapter as postgres_adapter_module
with patch.object(builtins, "__import__", new=_block_backend_import("psycopg")):
with pytest.raises(ImportError, match=r"python-repositories\[postgres\]"):
importlib.reload(postgres_adapter_module)
importlib.reload(postgres_adapter_module)
def test_top_level_lazy_import_propagates_redis_import_error() -> None:
"""Top-level RedisAdapter access surfaces adapter import errors."""
with patch(
@@ -112,6 +135,19 @@ def test_top_level_lazy_import_propagates_minio_import_error() -> None:
_ = python_repositories.MinioAdapter
def test_top_level_lazy_import_propagates_postgres_import_error() -> None:
"""Top-level PostgresAdapter access surfaces adapter import errors."""
with patch(
"importlib.import_module",
side_effect=ImportError(
"Postgres support requires the postgres extra. "
"Install with: pip install python-repositories[postgres]"
),
):
with pytest.raises(ImportError, match=r"python-repositories\[postgres\]"):
_ = python_repositories.PostgresAdapter
def test_adapters_subpackage_lazy_import_succeeds() -> None:
"""Adapter subpackage imports delegate to the same lazy loader."""
from python_repositories.adapters import RedisAdapter
@@ -123,7 +159,13 @@ def test_adapters_dir_exposes_lazy_exports() -> None:
"""dir(adapters) includes lazy adapter names for tab completion."""
import python_repositories.adapters as adapters
assert {"RedisAdapter", "MinioAdapter"}.issubset(set(dir(adapters)))
assert {
"RedisAdapter",
"MinioAdapter",
"PostgresAdapter",
"MemoryQueueAdapter",
"FileBackedQueueAdapter",
}.issubset(set(dir(adapters)))
def test_adapters_getattr_raises_for_unknown() -> None:
@@ -138,3 +180,6 @@ def test_top_level_dir_exposes_lazy_exports() -> None:
"""dir(python_repositories) includes lazy adapter names for tab completion."""
assert "RedisAdapter" in dir(python_repositories)
assert "MinioAdapter" in dir(python_repositories)
assert "PostgresAdapter" in dir(python_repositories)
assert "MemoryQueueAdapter" in dir(python_repositories)
assert "FileBackedQueueAdapter" in dir(python_repositories)
+14 -1
View File
@@ -5,12 +5,14 @@ from __future__ import annotations
from unittest.mock import MagicMock
from minio import Minio
import psycopg
import pytest
import redis
from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.adapters.postgres_adapter import PostgresAdapter
from python_repositories.adapters.redis_adapter import RedisAdapter
from tests.conftest import TEST_MINIO_CONFIG, TEST_REDIS_CONFIG
from tests.conftest import TEST_MINIO_CONFIG, TEST_POSTGRES_CONFIG, TEST_REDIS_CONFIG
@pytest.fixture
@@ -25,3 +27,14 @@ def minio_adapter() -> MinioAdapter:
"""Provide a MinioAdapter with an injected mock client."""
mock_client = MagicMock(spec=Minio)
return MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
@pytest.fixture
def postgres_adapter() -> PostgresAdapter:
"""Provide a PostgresAdapter with an injected mock client."""
mock_client = MagicMock(spec=psycopg.Connection)
mock_cursor = MagicMock()
mock_cursor.__enter__ = MagicMock(return_value=mock_cursor)
mock_cursor.__exit__ = MagicMock(return_value=False)
mock_client.cursor.return_value = mock_cursor
return PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
+74 -1
View File
@@ -5,11 +5,13 @@ from __future__ import annotations
from typing import cast
from unittest.mock import MagicMock, patch
import psycopg
import redis
from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.adapters.postgres_adapter import PostgresAdapter
from python_repositories.adapters.redis_adapter import RedisAdapter
from tests.conftest import TEST_MINIO_CONFIG, TEST_REDIS_CONFIG
from tests.conftest import TEST_MINIO_CONFIG, TEST_POSTGRES_CONFIG, TEST_REDIS_CONFIG
class TestRedisConnectionHealth:
@@ -161,3 +163,74 @@ class TestMinioConnectionHealth:
assert reinjected.is_connected()
assert mock_client.bucket_exists.call_count == 2
class TestPostgresConnectionHealth:
def test_not_connected_when_no_client(self) -> None:
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
assert not adapter.is_connected()
def test_connected_when_probe_succeeds(
self, postgres_adapter: PostgresAdapter
) -> None:
assert postgres_adapter.is_connected()
cast(MagicMock, postgres_adapter._client).cursor.assert_called()
def test_stale_connection_when_probe_fails(
self, postgres_adapter: PostgresAdapter
) -> None:
mock_cursor = cast(MagicMock, postgres_adapter._client).cursor.return_value
mock_cursor.execute.side_effect = psycopg.OperationalError("connection lost")
assert not postgres_adapter.is_connected()
def test_cache_hit_avoids_second_probe(
self, postgres_adapter: PostgresAdapter
) -> None:
mock_client = cast(MagicMock, postgres_adapter._client)
with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic",
return_value=100.0,
):
assert postgres_adapter.is_connected()
assert postgres_adapter.is_connected()
assert mock_client.cursor.call_count == 1
def test_cache_miss_runs_probe_again(
self, postgres_adapter: PostgresAdapter
) -> None:
mock_client = cast(MagicMock, postgres_adapter._client)
with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic",
side_effect=[100.0, 102.0],
):
assert postgres_adapter.is_connected()
assert postgres_adapter.is_connected()
assert mock_client.cursor.call_count == 2
def test_disconnect_clears_cache(self, postgres_adapter: PostgresAdapter) -> None:
mock_client = cast(MagicMock, postgres_adapter._client)
with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic",
return_value=100.0,
):
assert postgres_adapter.is_connected()
postgres_adapter.disconnect()
reinjected = PostgresAdapter(
config=postgres_adapter._config,
client=mock_client,
)
with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic",
return_value=100.0,
):
assert reinjected.is_connected()
assert mock_client.cursor.call_count == 2
@@ -0,0 +1,239 @@
"""Unit tests for FileBackedQueueAdapter."""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from pathlib import Path
import pytest
from python_repositories.adapters.file_backed_queue_adapter import (
FileBackedQueueAdapter,
)
from python_repositories.adapters.memory_queue_adapter import MemoryQueueAdapter
from python_repositories.config.file_queue_config import FileQueueConfig
from python_repositories.interfaces.queue_repository_interface import (
QueueRepositoryInterface,
)
def _config(path: Path, **kwargs: object) -> FileQueueConfig:
return FileQueueConfig(path=path, **kwargs) # type: ignore[arg-type]
def test_implements_interface() -> None:
assert issubclass(FileBackedQueueAdapter, QueueRepositoryInterface)
def test_ops_require_connect(tmp_path: Path) -> None:
queue = FileBackedQueueAdapter(config=_config(tmp_path / "q.jsonl"))
with pytest.raises(RuntimeError, match="not connected"):
queue.enqueue([{"id": 1}])
with pytest.raises(RuntimeError, match="not connected"):
queue.dequeue_batch()
with pytest.raises(RuntimeError, match="not connected"):
queue.size()
with pytest.raises(RuntimeError, match="not connected"):
queue.evict_older_than(datetime.now(UTC))
def test_enqueue_dequeue_persists_and_compacts(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
with FileBackedQueueAdapter(config=_config(path, dedup_keys=("id",))) as queue:
queue.enqueue([{"id": 1}, {"id": 2}, {"id": 1}])
assert queue.size() == 2
assert path.is_file()
lines = path.read_text(encoding="utf-8").strip().splitlines()
assert len(lines) == 2
batch = queue.dequeue_batch(max_items=1)
assert batch == [{"id": 1}]
remaining = path.read_text(encoding="utf-8").strip().splitlines()
assert len(remaining) == 1
assert queue.size() == 1
def test_replay_on_connect(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
config = _config(path, dedup_keys=("id",), age_key="created_at", max_age_hours=24)
now = datetime.now(UTC)
with FileBackedQueueAdapter(config=config) as queue:
queue.enqueue(
[
{"id": 1, "created_at": now - timedelta(hours=1)},
{"id": 2, "created_at": now - timedelta(minutes=10)},
]
)
with FileBackedQueueAdapter(config=config) as queue:
assert queue.size() == 2
assert [item["id"] for item in queue.dequeue_batch(max_items=10)] == [1, 2]
def test_replay_filters_by_max_age(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
now = datetime.now(UTC)
path.write_text(
"\n".join(
[
f'{{"id": "old", "created_at": "{(now - timedelta(hours=30)).isoformat()}"}}',
f'{{"id": "new", "created_at": "{(now - timedelta(hours=1)).isoformat()}"}}',
]
)
+ "\n",
encoding="utf-8",
)
config = _config(path, dedup_keys=("id",), age_key="created_at", max_age_hours=24)
with FileBackedQueueAdapter(config=config) as queue:
assert queue.size() == 1
assert queue.dequeue_batch(max_items=10)[0]["id"] == "new"
def test_connect_skips_corrupt_and_invalid_lines(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
path.write_text(
"\n".join(
[
"",
"not-json",
"[1, 2]",
'{"id": 1}',
'{"name": "missing-id"}',
]
)
+ "\n",
encoding="utf-8",
)
config = _config(path, dedup_keys=("id",))
with FileBackedQueueAdapter(config=config) as queue:
assert queue.size() == 1
assert queue.dequeue_batch(max_items=10) == [{"id": 1}]
def test_connect_skips_missing_or_invalid_age_on_replay(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
now = datetime.now(UTC)
path.write_text(
"\n".join(
[
'{"id": 1}',
'{"id": 2, "created_at": "not-a-date"}',
f'{{"id": 3, "created_at": "{now.isoformat()}"}}',
]
)
+ "\n",
encoding="utf-8",
)
config = _config(path, dedup_keys=("id",), age_key="created_at")
with FileBackedQueueAdapter(config=config) as queue:
assert queue.size() == 1
assert queue.dequeue_batch(max_items=10)[0]["id"] == 3
def test_connect_is_idempotent(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
queue = FileBackedQueueAdapter(config=_config(path))
queue.connect()
queue.enqueue([{"id": 1}])
queue.connect()
assert queue.size() == 1
queue.disconnect()
assert not queue.is_connected()
def test_evict_compacts_file(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
config = _config(path, dedup_keys=("id",), age_key="created_at", max_age_hours=24)
now = datetime.now(UTC)
with FileBackedQueueAdapter(config=config) as queue:
queue.enqueue(
[
{"id": 1, "created_at": now - timedelta(hours=2)},
{"id": 2, "created_at": now - timedelta(minutes=5)},
]
)
removed = queue.evict_older_than(now - timedelta(hours=1))
assert removed == 1
assert queue.size() == 1
lines = [line for line in path.read_text(encoding="utf-8").splitlines() if line]
assert len(lines) == 1
def test_auto_evict_on_enqueue(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
config = _config(path, dedup_keys=("id",), age_key="created_at", max_age_hours=1)
now = datetime.now(UTC)
with FileBackedQueueAdapter(config=config) as queue:
queue.enqueue([{"id": 1, "created_at": now - timedelta(hours=2)}])
# Item is enqueued then immediately age-evicted.
assert queue.size() == 0
def test_from_env_when_config_omitted(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
path = tmp_path / "from-env.jsonl"
monkeypatch.setenv("FILE_QUEUE_PATH", str(path))
with FileBackedQueueAdapter() as queue:
queue.enqueue([{"id": 1}])
assert queue.size() == 1
def test_injected_memory(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
memory = MemoryQueueAdapter(dedup_keys=("id",))
config = _config(path, dedup_keys=("id",))
with FileBackedQueueAdapter(config=config, memory=memory) as queue:
queue.enqueue([{"id": 1}])
assert memory.size() == 1
def test_empty_enqueue_and_dequeue_skip_compact(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
with FileBackedQueueAdapter(config=_config(path)) as queue:
queue.enqueue([])
assert queue.dequeue_batch(max_items=10) == []
assert queue.evict_older_than(datetime.now(UTC)) == 0
assert not path.exists() or path.read_text(encoding="utf-8") == ""
def test_reconnect_replays_after_disconnect(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
queue = FileBackedQueueAdapter(config=_config(path, dedup_keys=("id",)))
queue.connect()
queue.enqueue([{"id": 1}])
queue.disconnect()
queue.connect()
assert queue.size() == 1
queue.disconnect()
def test_json_default_rejects_unsupported() -> None:
with pytest.raises(TypeError):
FileBackedQueueAdapter._json_default(object())
def test_disconnect_ignores_fsync_oserror(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
path = tmp_path / "items.jsonl"
queue = FileBackedQueueAdapter(config=_config(path))
queue.connect()
def boom(_fd: int) -> None:
raise OSError("fsync failed")
monkeypatch.setattr(
"python_repositories.adapters.file_backed_queue_adapter.os.fsync",
boom,
)
queue.disconnect()
assert not queue.is_connected()
def test_append_items_requires_open_handle(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
queue = FileBackedQueueAdapter(config=_config(path))
queue.connect()
queue._append_handle = None # noqa: SLF001
with pytest.raises(RuntimeError, match="append handle is not open"):
queue._append_items([{"id": 1}]) # noqa: SLF001
queue._connected = False # noqa: SLF001
+56
View File
@@ -0,0 +1,56 @@
"""Unit tests for FileQueueConfig."""
from __future__ import annotations
from pathlib import Path
import pytest
from python_repositories.config import FileQueueConfig
def test_defaults() -> None:
config = FileQueueConfig(path=Path("/tmp/queue.jsonl"))
assert config.path == Path("/tmp/queue.jsonl")
assert config.max_age_hours == 24
assert config.dedup_keys == ()
assert config.age_key is None
def test_from_env_loads_path(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("FILE_QUEUE_PATH", "/data/items.jsonl")
monkeypatch.delenv("FILE_QUEUE_MAX_AGE_HOURS", raising=False)
config = FileQueueConfig.from_env(use_dotenv=False)
assert config.path == Path("/data/items.jsonl")
assert config.max_age_hours == 24
def test_from_env_loads_max_age_hours(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("FILE_QUEUE_PATH", "/data/items.jsonl")
monkeypatch.setenv("FILE_QUEUE_MAX_AGE_HOURS", "48")
config = FileQueueConfig.from_env(use_dotenv=False)
assert config.max_age_hours == 48
def test_from_env_raises_when_path_missing(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("FILE_QUEUE_PATH", raising=False)
with pytest.raises(Exception):
FileQueueConfig.from_env(use_dotenv=False)
def test_from_env_respects_custom_env_var_names(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv("CUSTOM_QUEUE_PATH", "/custom/q.jsonl")
monkeypatch.setenv("CUSTOM_MAX_AGE", "12")
config = FileQueueConfig.from_env(
"CUSTOM_QUEUE_PATH",
max_age_hours_env_var_name="CUSTOM_MAX_AGE",
dedup_keys=("id",),
age_key="created_at",
use_dotenv=False,
)
assert config.path == Path("/custom/q.jsonl")
assert config.max_age_hours == 12
assert config.dedup_keys == ("id",)
assert config.age_key == "created_at"
+149
View File
@@ -0,0 +1,149 @@
"""Unit tests for MemoryQueueAdapter."""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
import threading
import pytest
from python_repositories.adapters.memory_queue_adapter import MemoryQueueAdapter
from python_repositories.interfaces.queue_repository_interface import (
QueueRepositoryInterface,
)
def test_implements_interface() -> None:
assert issubclass(MemoryQueueAdapter, QueueRepositoryInterface)
def test_fifo_without_dedup() -> None:
queue = MemoryQueueAdapter()
queue.enqueue([{"id": 1}, {"id": 2}, {"id": 3}])
assert queue.size() == 3
assert queue.dequeue_batch(max_items=2) == [{"id": 1}, {"id": 2}]
assert queue.dequeue_batch(max_items=10) == [{"id": 3}]
assert queue.dequeue_batch() == []
assert queue.size() == 0
def test_empty_enqueue_is_noop() -> None:
queue = MemoryQueueAdapter()
queue.enqueue([])
assert queue.size() == 0
assert queue.enqueue_and_return_added([]) == []
def test_dedup_keeps_first() -> None:
queue = MemoryQueueAdapter(dedup_keys=("id",))
queue.enqueue([{"id": 1, "v": "a"}, {"id": 1, "v": "b"}, {"id": 2, "v": "c"}])
assert queue.size() == 2
assert queue.dequeue_batch(max_items=10) == [
{"id": 1, "v": "a"},
{"id": 2, "v": "c"},
]
def test_missing_dedup_key_raises() -> None:
queue = MemoryQueueAdapter(dedup_keys=("id",))
with pytest.raises(ValueError, match="missing dedup key"):
queue.enqueue([{"name": "x"}])
def test_missing_age_key_raises() -> None:
queue = MemoryQueueAdapter(age_key="created_at")
with pytest.raises(ValueError, match="missing age key"):
queue.enqueue([{"id": 1}])
def test_evict_older_than() -> None:
queue = MemoryQueueAdapter(age_key="created_at", dedup_keys=("id",))
now = datetime.now(UTC)
queue.enqueue(
[
{"id": 1, "created_at": now - timedelta(hours=2)},
{"id": 2, "created_at": now - timedelta(minutes=30)},
{"id": 3, "created_at": (now - timedelta(hours=3)).isoformat()},
]
)
removed = queue.evict_older_than(now - timedelta(hours=1))
assert removed == 2
assert queue.size() == 1
remaining = queue.dequeue_batch(max_items=10)
assert remaining[0]["id"] == 2
def test_evict_without_age_key_is_noop() -> None:
queue = MemoryQueueAdapter()
queue.enqueue([{"id": 1}])
assert queue.evict_older_than(datetime.now(UTC)) == 0
assert queue.size() == 1
def test_parse_age_rejects_unsupported_type() -> None:
with pytest.raises(ValueError, match="age value must be"):
MemoryQueueAdapter._parse_age(123)
def test_parse_age_aware_datetime() -> None:
from datetime import timezone
eastern = timezone(timedelta(hours=-5))
aware = datetime(2024, 1, 1, 12, 0, 0, tzinfo=eastern)
assert MemoryQueueAdapter._parse_age(aware) == datetime(
2024, 1, 1, 17, 0, 0, tzinfo=UTC
)
def test_parse_age_naive_datetime() -> None:
naive = datetime(2024, 1, 1, 12, 0, 0)
assert MemoryQueueAdapter._parse_age(naive) == datetime(
2024, 1, 1, 12, 0, 0, tzinfo=UTC
)
def test_parse_age_zulu_string() -> None:
assert MemoryQueueAdapter._parse_age("2024-01-01T00:00:00Z") == datetime(
2024, 1, 1, 0, 0, 0, tzinfo=UTC
)
def test_parse_age_naive_string() -> None:
assert MemoryQueueAdapter._parse_age("2024-01-01T00:00:00") == datetime(
2024, 1, 1, 0, 0, 0, tzinfo=UTC
)
def test_dequeue_rejects_negative_max_items() -> None:
queue = MemoryQueueAdapter()
with pytest.raises(ValueError, match="max_items"):
queue.dequeue_batch(max_items=-1)
def test_clear_and_snapshot() -> None:
queue = MemoryQueueAdapter()
queue.enqueue([{"id": 1}, {"id": 2}])
assert queue.snapshot() == [{"id": 1}, {"id": 2}]
queue.clear()
assert queue.size() == 0
assert queue.snapshot() == []
def test_thread_safety_smoke() -> None:
queue = MemoryQueueAdapter(dedup_keys=("id",))
errors: list[BaseException] = []
def worker(start: int) -> None:
try:
for i in range(start, start + 50):
queue.enqueue([{"id": i}])
except BaseException as exc: # noqa: BLE001
errors.append(exc)
threads = [threading.Thread(target=worker, args=(i * 50,)) for i in range(4)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
assert errors == []
assert queue.size() == 200
+237
View File
@@ -0,0 +1,237 @@
"""Unit tests for PostgresAdapter instantiation and injection."""
from __future__ import annotations
from unittest.mock import MagicMock
import psycopg
import pytest
from python_repositories.adapters.postgres_adapter import PostgresAdapter
from python_repositories.interfaces import TableRepositoryInterface
from tests.conftest import TEST_POSTGRES_CONFIG
def _mock_cursor() -> MagicMock:
mock_cursor = MagicMock()
mock_cursor.__enter__ = MagicMock(return_value=mock_cursor)
mock_cursor.__exit__ = MagicMock(return_value=False)
return mock_cursor
def _mock_client(*, probe_raises: Exception | None = None) -> MagicMock:
mock_client = MagicMock(spec=psycopg.Connection)
mock_cursor = _mock_cursor()
if probe_raises is not None:
mock_cursor.execute.side_effect = probe_raises
mock_client.cursor.return_value = mock_cursor
return mock_client
def test_should_adhere_to_interface() -> None:
assert issubclass(PostgresAdapter, TableRepositoryInterface)
_ = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
def test_should_have_logger_when_instantiated() -> None:
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
assert hasattr(adapter, "logger")
assert adapter.logger is not None
def test_should_not_be_connected_when_instantiated() -> None:
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
assert adapter._client is None
assert not adapter.is_connected()
def test_constructs_with_injected_config_without_env(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.delenv("POSTGRES_URI", raising=False)
monkeypatch.delenv("POSTGRES_TABLE", raising=False)
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
assert adapter._config == TEST_POSTGRES_CONFIG
def test_constructs_with_injected_client_without_env(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.delenv("POSTGRES_URI", raising=False)
monkeypatch.delenv("POSTGRES_TABLE", raising=False)
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
assert adapter._client is mock_client
assert adapter._client_injected is True
def test_raises_when_client_provided_without_config() -> None:
mock_client = MagicMock(spec=psycopg.Connection)
with pytest.raises(ValueError, match="config is required"):
PostgresAdapter(client=mock_client)
def test_disconnect_does_not_close_injected_client() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
adapter.disconnect()
mock_client.close.assert_not_called()
assert adapter._client is None
class CustomEnvPostgresAdapter(PostgresAdapter):
uri_env_var_name = "CUSTOM_POSTGRES_URI"
table_env_var_name = "CUSTOM_POSTGRES_TABLE"
primary_key_env_var_name = "CUSTOM_POSTGRES_PRIMARY_KEY"
def test_subclass_custom_env_var_names(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("CUSTOM_POSTGRES_URI", "postgresql://custom/mydb")
monkeypatch.setenv("CUSTOM_POSTGRES_TABLE", "items")
monkeypatch.setenv("CUSTOM_POSTGRES_PRIMARY_KEY", "item_id")
adapter = CustomEnvPostgresAdapter()
assert adapter._config.uri == "postgresql://custom/mydb"
assert adapter._config.table == "items"
assert adapter._config.primary_key == "item_id"
def test_connect_with_injected_client_succeeds_when_probe_ok() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
adapter.connect()
assert mock_client.cursor.return_value.execute.call_count == 2
def test_connect_with_injected_client_raises_on_probe_failure() -> None:
mock_client = _mock_client(probe_raises=psycopg.OperationalError("connection lost"))
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ConnectionError, match="Could not connect to Postgres"):
adapter.connect()
def test_connect_with_injected_client_skips_validation_when_client_cleared() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
adapter.disconnect()
adapter.connect()
mock_client.cursor.assert_not_called()
def test_connect_reconnects_when_existing_client_unhealthy(
monkeypatch: pytest.MonkeyPatch,
) -> None:
stale_client = _mock_client(
probe_raises=psycopg.OperationalError("connection lost")
)
new_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
adapter._client = stale_client
monkeypatch.setattr("psycopg.connect", lambda *args, **kwargs: new_client)
adapter.connect()
stale_client.close.assert_called_once()
assert adapter._client is new_client
def test_connect_skips_reconnect_when_already_connected(
monkeypatch: pytest.MonkeyPatch,
) -> None:
healthy_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
adapter._client = healthy_client
connect = MagicMock()
monkeypatch.setattr("psycopg.connect", connect)
adapter.connect()
healthy_client.close.assert_not_called()
connect.assert_not_called()
assert adapter._client is healthy_client
def test_fetch_one_raises_value_error_on_invalid_pk() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ValueError, match="Primary key must not be None"):
adapter.fetch_one(None)
def test_fetch_one_raises_connection_error_when_not_connected() -> None:
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
with pytest.raises(ConnectionError):
adapter.fetch_one("some-id")
def test_upsert_raises_value_error_on_invalid_row() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ValueError, match="Row must be a dictionary"):
adapter.upsert("not-a-dict") # type: ignore[arg-type]
def test_upsert_raises_value_error_when_primary_key_missing() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ValueError, match="Row must include primary key column 'id'"):
adapter.upsert({"name": "Alice"})
def test_upsert_raises_connection_error_when_not_connected() -> None:
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
with pytest.raises(ConnectionError):
adapter.upsert({"id": "alice", "name": "Alice"})
def test_fetch_all_raises_value_error_on_invalid_limit() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ValueError, match="Limit must be a non-negative integer"):
adapter.fetch_all(limit=-1)
def test_execute_raises_value_error_on_invalid_sql() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ValueError, match="SQL must be a non-empty string"):
adapter.execute("")
def test_execute_raises_value_error_on_invalid_params() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ValueError, match="Params must be a tuple"):
adapter.execute("SELECT 1", []) # type: ignore[arg-type]
def test_upsert_with_primary_key_only_uses_do_nothing() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
adapter.upsert({"id": "pk-only"})
mock_cursor = mock_client.cursor.return_value.__enter__.return_value
insert_call = mock_cursor.execute.call_args_list[-1]
assert insert_call[0][1] == ("pk-only",)
def test_execute_raises_connection_error_when_not_connected() -> None:
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
with pytest.raises(ConnectionError):
adapter.execute("SELECT 1")
+69
View File
@@ -0,0 +1,69 @@
"""Unit tests for PostgresConfig."""
from __future__ import annotations
import pytest
from python_repositories.config import PostgresConfig
def _set_required_postgres_env(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("POSTGRES_URI", "postgresql://localhost/mydb")
monkeypatch.setenv("POSTGRES_TABLE", "users")
def test_from_env_loads_all_fields(monkeypatch: pytest.MonkeyPatch) -> None:
_set_required_postgres_env(monkeypatch)
config = PostgresConfig.from_env(use_dotenv=False)
assert config.uri == "postgresql://localhost/mydb"
assert config.table == "users"
assert config.primary_key == "id"
def test_from_env_reads_primary_key_from_env(
monkeypatch: pytest.MonkeyPatch,
) -> None:
_set_required_postgres_env(monkeypatch)
monkeypatch.setenv("POSTGRES_PRIMARY_KEY", "user_id")
config = PostgresConfig.from_env(use_dotenv=False)
assert config.primary_key == "user_id"
def test_from_env_defaults_primary_key_to_id_when_unset(
monkeypatch: pytest.MonkeyPatch,
) -> None:
_set_required_postgres_env(monkeypatch)
monkeypatch.delenv("POSTGRES_PRIMARY_KEY", raising=False)
config = PostgresConfig.from_env(use_dotenv=False)
assert config.primary_key == "id"
def test_from_env_raises_when_uri_missing(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("POSTGRES_URI", raising=False)
monkeypatch.setenv("POSTGRES_TABLE", "users")
with pytest.raises(Exception):
PostgresConfig.from_env(use_dotenv=False)
def test_from_env_raises_when_table_missing(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("POSTGRES_URI", "postgresql://localhost/mydb")
monkeypatch.delenv("POSTGRES_TABLE", raising=False)
with pytest.raises(Exception):
PostgresConfig.from_env(use_dotenv=False)
def test_from_env_respects_custom_env_var_names(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv("CUSTOM_POSTGRES_URI", "postgresql://custom/mydb")
monkeypatch.setenv("CUSTOM_POSTGRES_TABLE", "items")
monkeypatch.setenv("CUSTOM_POSTGRES_PRIMARY_KEY", "item_id")
config = PostgresConfig.from_env(
"CUSTOM_POSTGRES_URI",
"CUSTOM_POSTGRES_TABLE",
"CUSTOM_POSTGRES_PRIMARY_KEY",
use_dotenv=False,
)
assert config.uri == "postgresql://custom/mydb"
assert config.table == "items"
assert config.primary_key == "item_id"
@@ -0,0 +1,100 @@
"""Unit tests for QueueRepositoryInterface."""
from datetime import UTC, datetime
from typing import Any
import pytest
from python_repositories.interfaces.queue_repository_interface import (
QueueRepositoryInterface,
)
class InMemoryQueueRepo:
"""Plain class that satisfies QueueRepositoryInterface without inheritance."""
def enqueue(self, items: list[dict[str, Any]]) -> None:
pass
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
del max_items
return []
def size(self) -> int:
return 0
def evict_older_than(self, cutoff: datetime) -> int:
del cutoff
return 0
def accepts_queue_repo(repo: QueueRepositoryInterface) -> None:
"""Type-checking hook for QueueRepositoryInterface structural subtyping."""
repo.size()
def test_instantiation_fails_when_enqueue_not_implemented() -> None:
class Incomplete(QueueRepositoryInterface):
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
return []
def size(self) -> int:
return 0
def evict_older_than(self, cutoff: datetime) -> int:
return 0
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_dequeue_batch_not_implemented() -> None:
class Incomplete(QueueRepositoryInterface):
def enqueue(self, items: list[dict[str, Any]]) -> None:
pass
def size(self) -> int:
return 0
def evict_older_than(self, cutoff: datetime) -> int:
return 0
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_size_not_implemented() -> None:
class Incomplete(QueueRepositoryInterface):
def enqueue(self, items: list[dict[str, Any]]) -> None:
pass
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
return []
def evict_older_than(self, cutoff: datetime) -> int:
return 0
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_evict_older_than_not_implemented() -> None:
class Incomplete(QueueRepositoryInterface):
def enqueue(self, items: list[dict[str, Any]]) -> None:
pass
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
return []
def size(self) -> int:
return 0
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_structural_subtyping() -> None:
repo: QueueRepositoryInterface = InMemoryQueueRepo()
accepts_queue_repo(repo)
assert isinstance(repo, QueueRepositoryInterface)
assert repo.evict_older_than(datetime.now(UTC)) == 0
@@ -0,0 +1,162 @@
"""Unit tests for TableRepositoryInterface."""
from typing import Any
import pytest
from python_repositories.interfaces.table_repository_interface import (
TableRepositoryInterface,
)
class InMemoryTableRepo:
"""Plain class that satisfies TableRepositoryInterface without inheritance."""
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
return None
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
del limit
return []
def upsert(self, row: dict[str, Any]) -> None:
pass
def delete(self, pk: Any) -> None:
pass
def execute(
self,
sql: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
del sql, params
return []
def accepts_table_repo(repo: TableRepositoryInterface) -> None:
"""Type-checking hook for TableRepositoryInterface structural subtyping."""
repo.fetch_one("pk")
def test_instantiation_fails_when_fetch_one_not_implemented() -> None:
"""Test that instantiation fails if fetch_one is not implemented."""
class Incomplete(TableRepositoryInterface):
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
return []
def upsert(self, row: dict[str, Any]) -> None:
pass
def delete(self, pk: Any) -> None:
pass
def execute(
self,
sql: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_fetch_all_not_implemented() -> None:
"""Test that instantiation fails if fetch_all is not implemented."""
class Incomplete(TableRepositoryInterface):
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
return None
def upsert(self, row: dict[str, Any]) -> None:
pass
def delete(self, pk: Any) -> None:
pass
def execute(
self,
sql: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_upsert_not_implemented() -> None:
"""Test that instantiation fails if upsert is not implemented."""
class Incomplete(TableRepositoryInterface):
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
return None
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
return []
def delete(self, pk: Any) -> None:
pass
def execute(
self,
sql: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_delete_not_implemented() -> None:
"""Test that instantiation fails if delete is not implemented."""
class Incomplete(TableRepositoryInterface):
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
return None
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
return []
def upsert(self, row: dict[str, Any]) -> None:
pass
def execute(
self,
sql: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_execute_not_implemented() -> None:
"""Test that instantiation fails if execute is not implemented."""
class Incomplete(TableRepositoryInterface):
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
return None
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
return []
def upsert(self, row: dict[str, Any]) -> None:
pass
def delete(self, pk: Any) -> None:
pass
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_structural_subtyping() -> None:
"""Test that a plain class satisfies TableRepositoryInterface structurally."""
repo: TableRepositoryInterface = InMemoryTableRepo()
accepts_table_repo(repo)
assert isinstance(repo, TableRepositoryInterface)
Generated
+496 -398
View File
File diff suppressed because it is too large Load Diff