Data, security, and recovery

Scientific Agent stores research locally by default. Durable data, cache, and application configuration are separated so important results can be backed up while regenerable content can be cleared safely.

Default locations

Research data defaults to ~/Documents/ScientificAgent on every platform. Cache and configuration follow platform conventions:

The CLI can override them explicitly:

scientific-agent \
  --data-dir /data/my-study \
  --cache-dir /scratch/scientific-agent-cache \
  --config-dir /data/my-settings \
  status RUN_ID

These three global options must appear before the subcommand.

What is in the data directory?

Main directories include:

An individual run commonly contains events/, artifacts/, workspace/, logs/, interactions/, and state/. This is an internal layout. Read it through Desktop, CLI, or the Control API; do not edit it manually.

Why keep both events and artifacts?

• Events record what happened: lifecycle, actions, budgets, interventions, and completion decisions.
• Artifacts record what supports it: inputs, data, code, predictions, evaluations, analyses, and reports.
• Projections combine them into Control Room, Timeline, and Lineage views.

On startup, recovery, or inspection, the Runtime replays events to reconstruct research state. Artifacts carry SHA-256 digests and are verified when read, so replaced content is not silently accepted as original evidence.

Manually changing events.jsonl, state files, or artifact content therefore breaks recovery and auditability. Correct a study by creating a new run or using a supported intervention that adds a new record.

Move the data directory

Settings → Storage shows the current location, size, and free space, and lets you choose another location. Before migration:

1. Pause or finish all runs.
2. Create a Runtime backup.
3. Choose a persistently mounted destination with enough space.
4. Let the app verify the destination after copying or moving.

Do not move the directory in a file manager while a run is active. Network and sync drives may change atomic-write and locking behavior; if one is required, first test pause, recovery, and artifact verification with a small toy run.

Runtime backups

Choose Create backup in Desktop, or call:

curl -sS -X POST http://127.0.0.1:8000/api/v1/storage/backups | jq

The ZIP is placed under <data-dir>/exports/backups/. It contains runs, artifacts, models, and reports, plus backup.json with a digest for every file. Cache and ordinary configuration are excluded because they are not research facts.

The backup file is still inside the data root. Real disaster recovery also copies it to another disk or controlled backup service and periodically tests extraction and hashes.

Scientific exports

A successful run can create a reviewer-oriented scientific export. It selects the final report, model/inference files, and necessary metadata under exports/.

Exports are suitable for paper supplements, colleague review, and deployment handoff. They are not a mirror of every historical run. Use a Runtime backup for full recovery.

Interruption and recovery

After an app or machine crash, a run normally appears as INTERRUPTED. Recovery:

1. verifies the event sequence and existing artifacts;
2. finds a safe recovery point;
3. marks unfinished jobs instead of assuming they succeeded;
4. continues with the saved contract and remaining budget.

Choose Recover in Control Room, or run:

scientific-agent resume RUN_ID

If the run is not in a recoverable state, the operation is rejected. Inspect the last failure or lifecycle event with status RUN_ID --events first.

Credentials

Desktop delegates provider credentials to the operating-system credential store and does not write them into research contracts, events, or model profiles. The CLI uses environment variables:

export LAB_MODEL_API_KEY="..."
scientific-agent run study.yaml

A config contains only the environment variable name, such as auth_env: LAB_MODEL_API_KEY. Do not commit .env files, shell history, logs, or configurations containing credentials.

The codex-cli provider uses the local Codex login state. It is a separate authentication path from API-key providers.

Control API and Desktop security

• Desktop's local API uses an app-generated ephemeral token and restricts accepted local origins.
• scientific-agent serve is an unauthenticated convenience mode for development and automation. It defaults to 127.0.0.1; do not expose it directly on a public address.
• A remote or multi-user service needs TLS, authentication, access control, request-size limits, and a dedicated operating-system account in front of it.
• The API returns controlled read models; never publish the data directory through a static file server.

Plugins and local execution

An enabled Python plugin has the same permissions as the app process. ZIP installation blocks common archive path attacks but does not prove that plugin business logic is safe. Enable only trusted code.

Local Executor constrains the workspace and declared inputs/outputs, but it is not a security sandbox for malicious programs. Run the entire Scientific Agent in a container, VM, or isolated account when code is untrusted.

Evaluator-private data

Artifacts with evaluator_private visibility are supplied only to trusted evaluators. They do not enter:

• Scientific Reasoner context;
• ordinary research-plugin inputs;
• researcher-facing artifact lists or Research Flow;
• public parts of scientific exports.

This prevents the ordinary training loop from directly reading answers. File permissions and isolated accounts should still protect the underlying disk.

Retention and deletion

The current UI emphasizes opening, moving, backing up, and exporting rather than treating historical facts as cache. To reclaim space:

1. create and externally retain a backup;
2. confirm that related reports and models were exported;
3. stop the app and API service;
4. archive or remove an explicitly identified whole run, never only its events or artifacts.

Cache can be regenerated; research data usually cannot. Never combine them in one broad cleanup command.