Plugin SDK reference

This chapter is for developers who have completed the plugin tutorial. It describes SDK v1 contracts, lifecycle, and boundaries. Examples use the flat plugin.yaml form used by built-in repository plugins.

Loading sequence

A directory plugin enters the system in this order:

1. The Registry discovers and parses plugin.yaml without importing code.
2. It validates plugin ID, SDK version, capabilities, dependency declarations, and entry-point shape.
3. Install stores the validated package; Enable imports its entrypoint.
4. It creates the ScientificPlugin instance and binds the on-disk manifest as authoritative for that loaded plugin.
5. start() runs before first execution; stop() runs during Runtime shutdown.
6. Each capability call enters execute() after input and artifact-reference validation.

When a CLI configuration directly names a directory under plugins, discovery and loading happen during run creation. A Desktop custom ZIP deliberately separates Install from Enable.

plugin.yaml

Plugin-level fields

Plugin and capability IDs must begin with a letter and contain only letters, digits, underscore, dot, colon, or hyphen. Once published, do not reuse an ID for an incompatible meaning.

dependencies is a declaration for display and availability checks, not an installation script. Plugin code must never invoke a package manager during import, start(), or execute().

Capability fields

Simple schema types are:

string  number  integer  boolean  object  array

This is a capability-boundary check, not full JSON Schema. Put nested fields, units, ranges, and mutual-exclusion rules in input_guidance and example, then enforce domain validation in code. Unknown ordinary inputs are rejected, and Python bool is not accepted as number or integer.

Artifact kind versus ArtifactType

They serve different purposes:

• ArtifactType is a Runtime storage classification such as DATASET, TRAINED_MODEL, PREDICTOR_OUTPUT, ANALYSIS_RESULT, or SCIENTIFIC_REPORT.
• An artifact kind is domain semantics such as three_body_trajectory, evaluation_result, or oscillator_assessment, normally carried in scientific metadata and declared by capabilities.

An analysis file can use ArtifactType.ANALYSIS_RESULT while carrying the domain kind oscillator_assessment. Do not extend the storage enum for every domain concept.

ScientificPlugin

A minimal implementation is:

class MyPlugin(ScientificPlugin):
    @property
    def manifest(self) -> PluginManifest:
        ...

    async def execute(
        self,
        capability: str,
        inputs: JsonObject,
        context: PluginExecutionContext,
    ) -> PluginResult:
        ...

Optional hooks are:

async def start(self) -> None: ...
async def stop(self) -> None: ...
def create_reasoner(self) -> Reasoner | None: ...
def estimate_cost(
    self, capability: str, inputs: JsonObject
) -> dict[str, int | float]: ...

Lifecycle guidance

• The constructor and manifest should be fast, deterministic, and free of network side effects.
• start() may load models, establish local connections, or prepare read-only resources; it should be safe to invoke once.
• execute() must not rely on global state concurrently mutated by another call.
• stop() should safely release resources even after partial initialization.
• No hook may edit research contracts, event files, or Runtime artifact directories.

estimate_cost()

This method runs before execution and can reject an obviously over-budget action. It must be fast and deterministic and must not perform the experiment itself. For example:

def estimate_cost(self, capability, inputs):
    if capability == "lab.solver.generate_dataset":
        return {
            "training_samples": int(inputs["num_samples"]),
            "solver_call": 1,
        }
    return {}

An estimate is not actual usage. Report measurable consumption in PluginResult.resource_usage afterward. Training samples are a commonly recognized direct usage field; action types also account for experiments, training runs, and solver calls.

PluginExecutionContext

Prefer resolved_artifacts["logical_name"] for direct inputs declared in the manifest. Use artifact_graph and read_artifact() only when lineage traversal is necessary.

Do not:

• infer disk paths from artifact URIs;
• scan the entire data directory;
• open another run's directory;
• escape with workspace.parent;
• attempt to read private fields from an ordinary research plugin.

The Runtime enforces same-run scope, visibility, and hashes. The plugin must still validate file format, size, encoding, and domain schema.

PluginResult

PluginResult(
    success=True,
    metrics={"rmse": 0.04},
    observations={"samples": 512},
    artifacts=(...),
    resource_usage={"training_samples": 512},
)

Rules:

• A successful result cannot also have error; a failed result must have one.
• Metric names must be unique ignoring case, with values persistable as numbers.
• Observations, metadata, and resource usage must be JSON-serializable.
• Resource usage cannot contain negative values or booleans.
• Artifact file names in one result must be unique.

Use a failed PluginResult for expected invalid input, solver failure, or external-tool failure. A genuine plugin bug or unrecoverable initialization failure may throw; the Runtime records it as an execution failure. Error text must not contain credentials, private labels, or large input payloads.

PluginArtifact

A binary artifact:

PluginArtifact(
    name="predictions.csv",
    artifact_type=ArtifactType.PREDICTOR_OUTPUT,
    content=csv_bytes,
    metadata={"schema": "predictions-v1"},
)

The text helper:

PluginArtifact.text(
    name="report.md",
    artifact_type=ArtifactType.SCIENTIFIC_REPORT,
    content=markdown,
    metadata={"format": "markdown"},
)

name must be a plain file name, never a path. Link several new artifacts from one result with parent_names; each parent must appear before its child:

artifacts=(
    PluginArtifact.text("data.json", ArtifactType.DATASET, data),
    PluginArtifact.text(
        "analysis.json",
        ArtifactType.ANALYSIS_RESULT,
        analysis,
        parent_names=("data.json",),
    ),
)

The default is visibility=ArtifactVisibility.RESEARCH. EVALUATOR_PRIVATE is only for a designed and reviewed trusted-evaluation flow; it is not a way for an ordinary plugin to hide inconvenient results from users.

Trusted evaluators

Relevant manifest fields are:

type: evaluator
access_role: trusted_evaluator
trusted_metrics: true

A trusted evaluator needs stricter design:

• predictions and public evaluation features arrive through ordinary artifact inputs;
• the Runtime resolves private targets from an immutable evaluation_id;
• sample IDs, shape, schema, and evaluation identity are matched exactly;
• outputs contain only aggregate metrics, allowed diagnostics, and evaluation artifacts;
• private labels never leak through observations, logs, or research-visible artifacts;
• tests prove that ordinary plugins and the Reasoner cannot access private content.

Self-declaring trusted fields in a manifest does not replace code review or deployment trust. Review a third-party evaluator as privileged code before enabling it.

Executor plugins

An execution backend inherits ExecutorPlugin. Its core method is:

async def execute_job(
    self,
    job: JobSpec,
    context: PluginExecutionContext,
) -> JobResult:
    ...

Asynchronous or remote backends can also implement submit(), poll(), cancel(), collect_outputs(), collect_usage(), and cleanup(). The generic execute() already adapts a job_spec input to these lifecycle methods.

An Executor must honor the command, input, and output protocol in JobSpec; preserve stdout/stderr and exit status; support cancellation; and keep outputs in the workspace. A remote scheduler must persist external job IDs so polling can resume after process restart.

Do not confuse an Executor with a Domain plugin. A Domain plugin describes what scientific operation to perform; an Executor describes where and how to run an already-declared job.

Versioning and compatibility

• sdk_version changes when the SDK contract changes; the current value is 1.
• Use a patch release for a bug fix that preserves inputs and outputs.
• A minor release may add backward-compatible optional fields or capabilities.
• Removing a capability, renaming a metric, changing a required input, or changing artifact schema requires a major release.
• Persisted runs depend on old capability identity; never change an old version's meaning in place.

When migration is necessary, introduce a new capability ID and provide a configuration conversion example in release notes. Capability aliases are for deliberate compatibility routing, not for hiding a breaking change.

ZIP package limits

A custom package must have:

• a .zip extension;
• at most 10 MiB compressed content;
• at most 50 MiB expanded content;
• at most 512 files;
• one unambiguous plugin.yaml, plugin.yml, or plugin.json;
• no absolute path, .. traversal, symlink, encrypted, duplicate, or conflicting entries.

Do not package virtual environments, model caches, Git history, test output, or credentials. Large models and datasets should be separately deployed, verifiable resources rather than content embedded in the plugin ZIP.

Recommended test layers

1. Manifest tests parse the manifest and check capability IDs, metric producers, and entry point.
2. Pure unit tests call execute() directly for normal, boundary, and invalid inputs.
3. Artifact tests verify content, metadata, parent relationships, and hash-backed reads.
4. Registry tests discover and load from a real directory without a test-only import shortcut.
5. ZIP tests install, enable, and disable; malicious paths and oversized archives must fail.
6. End-to-end run creates a small-budget contract and reaches a real acceptance criterion.
7. Recovery tests cover pause, process interruption, recovery, and cancellation for long jobs.
8. Authority tests prove ordinary plugins cannot read evaluator-private or other-run artifacts.

Start code validation with:

ruff check path/to/plugin
pytest -q path/to/plugin/tests
scientific-agent --data-dir /tmp/plugin-e2e run plugin-example.yaml