storage

storage

Storage usage aggregation across the package stack.

Orchestrates queries from three layers: - terok-sandbox — container overlay sizes (podman) - terok-executor — task workspace and shared mount sizes (filesystem) - terok itself — image knowledge (L0/L1/L2 classification)

Two entry points mirror two levels of detail:

• get_storage_overview — fast global summary with per-project one-liners (no per-container podman --size queries)
• get_project_storage_detail — per-task breakdown for one project, including the expensive overlay size computation

ORPHAN_PROJECT_ID = '(orphans)' module-attribute

Synthetic project id for the overview row that collects L2 images whose project no longer exists in the config.

Parentheses are forbidden in real project ids (validated against [a-z0-9][a-z0-9_-]* in project_model), so this value can never collide with a configured project.

ProjectSummary(project_id, image_bytes, workspace_bytes, task_count) dataclass

One-line digest of a project's storage footprint.

project_id instance-attribute

image_bytes instance-attribute

workspace_bytes instance-attribute

task_count instance-attribute

total_bytes property

Sum of images and workspaces (overlays excluded in overview mode).

StorageOverview(global_images, shared_mounts, projects) dataclass

Fast global summary: globals expanded, projects as one-liners.

global_images instance-attribute

shared_mounts instance-attribute

projects instance-attribute

global_images_bytes property

Total size of global images.

shared_mounts_bytes property

Total size of shared mount directories.

projects_bytes property

Total size across all projects.

grand_total property

Everything combined.

ProjectDetail(project_id, images, tasks, overlays) dataclass

Full per-task storage breakdown for a single project.

project_id instance-attribute

images instance-attribute

tasks instance-attribute

overlays instance-attribute

images_bytes property

Total size of project images.

workspace_bytes property

Total workspace size across all tasks.

overlay_bytes property

Total overlay size across all running containers.

total_bytes property

Everything for this project.

parse_image_size(text)

Best-effort parse of podman's human-readable image size strings.

Returns 0 for unparseable input — storage reporting should never crash on a formatting surprise from podman.

Source code in src/terok/lib/domain/storage.py

def parse_image_size(text: str) -> int:
    """Best-effort parse of podman's human-readable image size strings.

    Returns 0 for unparseable input — storage reporting should never crash
    on a formatting surprise from podman.
    """
    m = _SIZE_RE.search(text)
    if not m:
        return 0
    try:
        multiplier = _UNITS.get(m.group(2).upper())
        if multiplier is None:
            return 0
        return int(float(m.group(1)) * multiplier)
    except (ValueError, OverflowError):
        return 0

format_bytes(n)

Format bytes as a right-aligned human-readable string.

Uses SI units (1000-based) to match podman's output convention. Always returns a fixed-width string suitable for column alignment.

Source code in src/terok/lib/domain/storage.py

def format_bytes(n: int) -> str:
    """Format bytes as a right-aligned human-readable string.

    Uses SI units (1000-based) to match podman's output convention.
    Always returns a fixed-width string suitable for column alignment.
    """
    for unit, threshold in (("TB", 1e12), ("GB", 1e9), ("MB", 1e6), ("KB", 1e3)):
        if n >= threshold:
            return f"{n / threshold:.1f} {unit}"
    return f"{n} B"

get_storage_overview()

Gather global summary — fast, no per-container podman queries.

Iterates all projects, sums workspace sizes via terok-executor, and classifies images into global vs per-project.

Source code in src/terok/lib/domain/storage.py

def get_storage_overview() -> StorageOverview:
    """Gather global summary — fast, no per-container podman queries.

    Iterates all projects, sums workspace sizes via terok-executor, and
    classifies images into global vs per-project.
    """
    all_images = list_images()
    global_images = [img for img in all_images if _is_global_image(img)]

    shared_mounts = SharedMountStorageInfo.measure_all(sandbox_live_mounts_dir())

    # Per-project: sum image sizes + workspace sizes
    projects_conf = list_projects()
    project_image_bytes: dict[str, int] = {}
    for img in all_images:
        pid = _image_project_id(img)
        if pid:
            project_image_bytes[pid] = project_image_bytes.get(pid, 0) + parse_image_size(img.size)

    summaries = []
    for proj in projects_conf:
        tasks = TaskStorageInfo.measure_all(proj.tasks_root)
        summaries.append(
            ProjectSummary(
                project_id=proj.id,
                image_bytes=project_image_bytes.get(proj.id, 0),
                workspace_bytes=sum(t.workspace_bytes for t in tasks),
                task_count=len(tasks),
            )
        )

    # Surface L2 images whose project is no longer configured so they roll up
    # into the grand total instead of vanishing from the overview.
    known_ids = {p.id for p in projects_conf}
    orphan_bytes = sum(b for pid, b in project_image_bytes.items() if pid not in known_ids)
    if orphan_bytes:
        summaries.append(
            ProjectSummary(
                project_id=ORPHAN_PROJECT_ID,
                image_bytes=orphan_bytes,
                workspace_bytes=0,
                task_count=0,
            )
        )

    return StorageOverview(
        global_images=global_images,
        shared_mounts=shared_mounts,
        projects=summaries,
    )

get_project_storage_detail(project_id)

Detailed view for one project, including overlay sizes.

This triggers podman ps --size for the project's containers — expect a brief pause while podman computes overlay diffs.

Source code in src/terok/lib/domain/storage.py

def get_project_storage_detail(project_id: str) -> ProjectDetail:
    """Detailed view for one project, including overlay sizes.

    This triggers ``podman ps --size`` for the project's containers —
    expect a brief pause while podman computes overlay diffs.
    """
    from ..core import runtime as _rt
    from ..core.projects import load_project

    project = load_project(project_id)
    project_images = [img for img in list_images(project_id) if not _is_global_image(img)]
    tasks = TaskStorageInfo.measure_all(project.tasks_root)
    runtime = _rt.resolve_runtime(project)
    # ``container_rw_sizes`` is podman-specific; not every backend exposes it.
    overlays = (
        runtime.container_rw_sizes(project_id) if hasattr(runtime, "container_rw_sizes") else {}
    )

    return ProjectDetail(
        project_id=project_id,
        images=project_images,
        tasks=tasks,
        overlays=overlays,
    )