Shared Modules¶

This page is the release procedure for the published libraries Algol Cloud consumes. It is the one operator-facing document that names those libraries' import paths, because the paths are the contract.

Some of this repository's libraries are consumed by algol-cloud. They are the device/cloud message contract: the addressing and envelope logic both sides must implement identically or misroute. Before this, cloud kept a hand-copied fork of them, and the fork drifted - see RFC-026 for what that cost and why versioned modules replace it.

A module in this list is published, which means it has a version, a consumer outside this repository, and rules about what may change in it. Everything else in lib/ and app/ is internal and has none of that.

Module	Import path	What cloud uses it for
`lib/component`	`github.com/parametric-sds/algol-app/lib/component`	Component reference parsing, attributes, filters (ADR-002)
`lib/messaging`	`github.com/parametric-sds/algol-app/lib/messaging`	The wire types: component status, telemetry measurements, subjects and context types, content digests (RFC-026 steps 3 and 5)
`lib/configuration`	`github.com/parametric-sds/algol-app/lib/configuration`	The config model: every config type and its pattern constants, the validator (`validate`, a subpackage so that importing the types alone links no `go-playground/validator`), the schema document reader (`configschema`), property binding (`jsonbind`) and the RFC-009 migration registry (`migrate`), for authoring configs and bundles (RFC-026 step 5). Unlike the two above it changes shape on purpose, versioned by `SchemaVersion()` rather than by additive-only fields, so it stays on a `v0` line.

The list lives in scripts/shared-modules.sh and is explicit: being published is a decision, not a property of the code.

How a subdirectory module is published¶

There is no root go.mod in this repository, so a shared module is published on its own version line and the repo's vX.Y.Z release tags mean nothing to Go.

Go resolves github.com/parametric-sds/algol-app/lib/component by fetching the algol-app repository and reading lib/component/go.mod inside it. The version comes from a tag carrying the module's directory as a prefix:

lib/component/v1.0.0

Publishing is creating and pushing that tag. Nothing is uploaded anywhere, and the repository stays private.

A pushed tag is immutable. A consumer that resolved it recorded the module's hash in its own go.sum, so moving the tag later reads as corruption rather than as an update. There is no fixing a bad release in place - publish the next patch version.

A major version above 1 is a path change, not a tag. v2.0.0 of lib/component would have to live at lib/component/v2 with a matching module line, and every consumer edits its imports. That expense is the point: RFC-026 exists because a breaking change to a shared type used to be invisible, and this is what makes it visible work.

Within a major version the rule is the same one RFC-025 S3 states for event schemas: add only. A fleet does not update atomically, so a device in the field is always running an older contract than the cloud - a removed field or a retyped one breaks a device that is still deployed, not just a build.

Releasing¶

./scripts/tag-shared-module.sh lib/component v1.0.0
git push origin lib/component/v1.0.0

The script refuses anything that would produce an unusable tag: an unlisted module, a malformed version, a tag that already exists locally or on origin, a HEAD that is not on a remote branch, modified tracked files, or a module that fails scripts/verify-shared-modules.sh. It creates the tag and stops - pushing it is the release and stays a deliberate step.

Then note what changed. There is no CHANGELOG.md in this repository (GitHub Releases is the source of truth, see the release notes template), so a shared-module release is a GitHub Release on its tag, and one line saying what a consumer gains or must change is enough.

Why the verification script exists¶

Every app in this repository reaches lib/component through a replace directive:

github.com/parametric-sds/algol-app/lib/component => ../../lib/component

so nothing in our own build ever resolves it as a published module. A shared module can break for an external consumer - a new intra-repo dependency, a leftover replace, a failing test - while every app here stays green. Until this was added, lib/component's own tests never ran in CI at all: the matrix in workflow_go_app.yml is built from ./app only.

scripts/verify-shared-modules.sh closes that gap and runs on every PR and every push to main. It checks that each shared module:

has no replace directives - a replace in a dependency module is ignored by the consumer, so the require it redirects resolves against the real, unpublished path
requires no intra-repo module that is not itself published
carries no v0.0.0 placeholder version
has a module path matching its directory
is gofmt clean, vets, and passes its tests

When a shared module depends on a shared module¶

lib/messaging requires lib/component, and lib/configuration requires both, so the no-replace rule above bites differently there than it does in an app. The require names a published version:

require github.com/parametric-sds/algol-app/lib/component v1.0.0

Three things follow, and none of them is optional.

Intra-repo replace directives must not pin a version on the left. A replace with a version applies to that version only. Once lib/messaging requires v1.0.0, MVS selects v1.0.0 in every app that depends on both, and a ... v0.0.0 => ../../lib/component rule stops matching - the build then resolves lib/component over the network instead of from the tree it is standing in. The version-less form applies to every version and keeps the working tree authoritative.

A local lib/component edit is invisible to lib/messaging's own tests. When lib/messaging is the main module - its own go test, and verify-shared-modules.sh - it has no replace, so it builds against the published v1.0.0. Apps are unaffected, because a main module's replace wins over anything a dependency declares. So a change spanning both modules is a two-step release: tag lib/component, then bump the require in lib/messaging.

A Go workspace lifts that restriction while you are working, and only while you are working:

go work init && go work use -r .   # resolves every local module from the tree

go.work and go.work.sum are gitignored, deliberately. A committed workspace would put every build in the tree into workspace mode, where two errors stop being errors: a missing or wrong replace still builds, and a shared module that has drifted ahead of the version it requires - using a symbol that exists here but not in the tag - compiles clean. The container build has no root go.work (its context is the app directory) so it stays in module mode and fails on both, which is the wrong place to find out.

For the same reason verify-shared-modules.sh sets GOWORK=off itself. Its whole claim is "this is what an external consumer gets", and a workspace makes that claim false while leaving the output green.

CI needs read access to the module path, not just a checkout. The go command clones this repository to read lib/component out of it at the tag, so workflow_shared_module.yml sets GOPRIVATE and grants git an org-scoped insteadOf using the job's own GITHUB_TOKEN. That token is enough because the module lives in the same repository, and it expires with the job.

Consuming from algol-cloud¶

The repository is private, so GOPRIVATE is required - and not only for convenience. Without it the go command asks proxy.golang.org and sum.golang.org for the module. Those requests fail to authenticate, but only after the module path and version have been sent to a third party.

export GOPRIVATE=github.com/parametric-sds/*

GOPRIVATE puts the fetch in direct mode, which means the go command shells out to git, so the credential that matters is git's, not Go's. GOAUTH (default netrc) covers the go command's own HTTPS requests - a module proxy, or a go-import meta page - and github.com needs neither, because the go command recognises it as a VCS host without asking.

Authenticating¶

Actions' built-in GITHUB_TOKEN cannot be used: it is scoped to the repository running the workflow, so in an algol-cloud workflow it has no read access to algol-app. That gap is the reason this section exists.

Access is over SSH, on both sides. A token embedded in a git config file is what this deliberately avoids: it writes the secret to disk, where it outlives the step, appears in any git config --list in the job log, and persists on a self-hosted runner. Neither of these is used:

# not used here
git config --global url."https://x-access-token:${TOKEN}@github.com/".insteadOf "https://github.com/"
git config --global http."https://github.com/".extraheader "AUTHORIZATION: basic ${B64}"

Import paths are https:// URLs regardless, so both setups below rewrite them to SSH, scoped to the organisation rather than to all of github.com:

git config --global url."ssh://git@github.com/parametric-sds/".insteadOf "https://github.com/parametric-sds/"

This covers every private module from the organisation, not just this one, and leaves other hosts and other orgs alone.

Developer machines¶

The rewrite above plus the SSH key the developer already has. Nothing else, and no credential is stored by it.

CI: a read-only deploy key¶

algol-cloud's Actions workflows authenticate to algol-app with a deploy key. A deploy key grants access to exactly one repository, which makes the grant repo-scoped by construction - the closest available fit to "read algol-app, nothing else".

Set it up once:

Generate a keypair. It belongs to CI, not to a person:

ssh-keygen -t ed25519 -N "" -C "algol-cloud CI -> algol-app (read-only)" -f algol-app-deploy-key

In algol-app: Settings -> Deploy keys -> Add deploy key. Paste algol-app-deploy-key.pub, title it for the consumer, and leave Allow write access unchecked. A shared module is read-only to its consumers.
In algol-cloud: Settings -> Secrets and variables -> Actions -> New repository secret, named ALGOL_APP_DEPLOY_KEY, holding the private half.
Delete both local files.

Then in the workflow, load the key into an ssh-agent so the private key is never written to the runner's filesystem:

- name: Authenticate to algol-app
  env:
    DEPLOY_KEY: ${{ secrets.ALGOL_APP_DEPLOY_KEY }}
  run: |
    eval "$(ssh-agent -s)"
    echo "$DEPLOY_KEY" | ssh-add -
    echo "SSH_AUTH_SOCK=$SSH_AUTH_SOCK" >> "$GITHUB_ENV"
    echo "SSH_AGENT_PID=$SSH_AGENT_PID" >> "$GITHUB_ENV"
    mkdir -p ~/.ssh
    ssh-keyscan -t ed25519 github.com >> ~/.ssh/known_hosts
    git config --global url."ssh://git@github.com/parametric-sds/".insteadOf "https://github.com/parametric-sds/"

- name: Build
  run: go build ./...
  env:
    GOPRIVATE: github.com/parametric-sds/*

Notes on that step, each of which is a way it silently fails otherwise:

SSH_AUTH_SOCK must be exported to $GITHUB_ENV. Every run: is a fresh shell, so an agent started in one step is invisible to the next unless its address is handed over.
ssh-add - reads the key from stdin, so it never becomes a file. Passing it through the step env: rather than interpolating ${{ secrets.* }} into the script keeps it out of the rendered command line too.
GOPRIVATE must be set on the step that builds, not only on the one that authenticates. Without it the go command asks the public proxy first.
The key must be a deploy key on algol-app, not on algol-cloud. It authenticates the repository being read.

The cost of a deploy key is that it is long-lived: rotation is manual, and nothing expires it. If that becomes uncomfortable, the alternative is a GitHub App installation token scoped to algol-app and minted per run (actions/create-github-app-token), handed to git through a credential helper that reads the value from the environment rather than from a config file.

Using it¶

go get github.com/parametric-sds/algol-app/lib/component@v1.0.0

import "github.com/parametric-sds/algol-app/lib/component"

Three consequences worth knowing before the first go get:

The consumer needs read access to the whole algol-app repository, not just to the shared module. Go fetches the repository and reads the module out of it; there is no narrower grant. This is the one real cost of publishing from this repository rather than a dedicated one (RFC-026 OQ1).
Integrity is still checked. Bypassing the public checksum database does not mean bypassing go.sum: the first resolution records the hash in the consumer's go.sum and every later build verifies against it.
The go directive is a constraint on the consumer. lib/component declares go 1.27, so a consumer on an older toolchain cannot build it.

Adding a module to the list¶

Publishing a module is a contract, so it is not just an edit to scripts/shared-modules.sh:

The module must pass scripts/verify-shared-modules.sh - in particular it must not depend on any unpublished module in this repository. For the wire types (messaging, messaging/service/{status,telemetry}) that is RFC-026 step 3 and involves moving them out of lib/flow, whose dependency graph is exactly what must not reach cloud.
Add it to scripts/shared-modules.sh and to the table at the top of this document.
Tag it, and say in the PR that a new module became public. From that tag on, a breaking change to it costs a major version.