> ## Documentation Index > Fetch the complete documentation index at: https://piloop.autoresearch.inc/llms.txt > Use this file to discover all available pages before exploring further. # The artifact store > How piloop archives the shared workspace when it is done with it, and how to get files back. Every project has one shared workspace, `$PILOOP_HOME/projects//workspace`. It is linked into every agent's worktree as `./workspace/`. Agents leave notes, logs, evaluation outputs and analysis arrays there. On a long campaign it grows to hundreds of gigabytes, much of it copies. piloop stores workspace files in a content-addressed **artifact store** once nothing needs them any more. It removes the local copies later, and `piloop restore` brings them back. ## What is archived, and when The daemon archives in the background, one job at a time. Its scheduling loop never waits for an archiving. * **When an idea is done** (its last run is judged), the entries that belong to it are archived: `workspace/idea…` (`idea12/`, `idea12-canary.log`). Workers are told to put their output there. * **When a campaign stops** and none of its agents is alive any more, everything left in its workspace is archived. * **Never archived**: `NOTES.md` and `research/` at the top of the workspace. These small knowledge files stay where research sweeps read them. An archiving stores every file and symlink, stores a manifest that lists them, and checks that the store holds all of it. Only then does it record the archiving in the `archives` table. It removes nothing: the local copies stay, so an agent or a human that still reads a file, for example to answer an escalation, finds it. ## When local copies go The daemon removes the workspace copies of an archiving later, and records when in `archives.pruned_at`: * **By age**: 7 days after the archiving. * **Under disk pressure**: while the disk of the store has less than `min_free_gb` (`[storage]`, 20 GB by default) free, the oldest archivings first. It removes only a file that still holds what was stored (same size and hash), and a symlink that still points where it did. A file that an agent wrote since stays, and so does its directory. It checks age and pressure every hour, before the next archiving, and checks pressure again before it stores anything. Storing a file writes its compressed copy first, so archiving needs room on the disk of the store. A workspace larger than the free space above `min_free_gb` is stored in batches: before each batch, the daemon removes older local copies while the disk is short, then measures the free space again. When not one more file fits, the rest is not stored, and the daemon writes an `anomaly.disk` event on the project, with the bytes needed, the bytes free and the reserve. Only the last batch of a job that stored everything marks the job as done (`archives.complete`). A job that stops short, because of the disk, a failure or a stop of the daemon, stays due, and its next run stores the rest. After a failed or deferred archiving the daemon waits 10 minutes before the next one. Which archivings are due is computed from the database. A daemon that stops in the middle of a backlog picks it up where it was when it starts again. Content that was already stored is found by its hash and is not stored twice. The first daemon start after an upgrade archives every done idea and stopped campaign that is already there. `piloop store status` shows how many are still due. Workers are told to put their output under `workspace/idea/`, so it is archived with their idea. They are also told not to copy the code tree into the workspace (every candidate's tree is in git) and to leave evaluator output in the project's own cache. ## Getting files back ```bash theme={null} piloop restore my-project 12 # the files of idea#12 piloop restore my-project idea12/plots # one path, or everything under it piloop restore my-project # everything ``` `piloop restore` reads the files from the local store, or downloads them from the remote. A file that is already in the workspace with other content is kept and listed, never overwritten. Agents may run it too. ## How the store works One store serves every project of a `PILOOP_HOME`, in `$PILOOP_HOME/store`. * **Content addressing.** A blob is addressed by the BLAKE3 hash of its whole content. A file that appears in many places, or in two campaigns, is stored once. * **Compression.** Blobs are kept zstd-compressed. JSON and logs shrink about tenfold; arrays that are already compressed (`.npz`) barely shrink. * **Streaming.** Files of any size are read and written in 1 MiB pieces, never held in memory. * **Manifests.** A manifest lists the paths of one archiving, each with its blob, size and mode (or a symlink's target). It is a blob too, and the `archives` table records its address. Nothing is ever deleted from the store. There is no garbage collection. ## A remote Set `[storage.remote]` in `$PILOOP_HOME/config.toml` to mirror the store. Its `kind` picks the backend, and the backend owns the other keys of the table. `s3` and `gcs` are built in; `gcs` needs the `gcs` extra (`uv tool install --reinstall 'piloop[gcs]'`). A package can add a backend: see [storage backends](/architecture/extension-points#storage-backends). The keys of `[storage]` itself hold for every backend. The [configuration reference](/reference/configuration) lists every key and shows the same store on S3 and on GCS. ```toml theme={null} [storage] evict_after_days = 30 local_max_gb = 200 [storage.remote] kind = "s3" url = "s3://my-bucket/piloop" storage_class = "GLACIER_IR" # the default standard_below = 131072 # compressed bytes; the default (128 KB) ``` piloop checks `[storage.remote]` against its backend when it opens the store: `piloop serve`, `piloop restore` and `piloop store` fail on a key the backend does not know. * **Mirror.** With a remote, every blob is uploaded as it is stored. For S3, boto3 finds the credentials (environment, profile or instance role). * **Storage class.** Each object gets Glacier Instant Retrieval (millisecond reads at a fraction of the price of STANDARD), except objects under `standard_below`: Glacier IR bills each object as at least 128 KB, so small ones are cheaper as STANDARD. * **Eviction.** Every hour, before the next archiving, the daemon drops the local copies of blobs that the remote holds: those older than `evict_after_days`, then the oldest ones until the local store fits `local_max_gb`. `piloop store evict` does the same on demand. It first uploads any blob that was stored before you set the remote. * **Remote only.** With `evict_after_days = 0`, the local store is only a staging area: each blob's local copy is dropped as soon as it is uploaded, and `piloop restore` downloads it. * **One remote per store.** The first upload writes the remote's URL to `$PILOOP_HOME/store/remote.url`. A store whose blobs went to one remote refuses to open with another remote, or with none: the blobs it marked as uploaded are only in the first one. To move to a new remote, copy the objects there, then write its URL into that file. `piloop store verify` checks that every blob of every archive is still there. It hashes the local copies again and asks the remote for the evicted ones. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.