Packages and PDKs (design)¶
Status: design, decided on 2026-11-05 (Decisions). Phases 0, 1 (the store and lock format 2), 2 (manifests, transitive dependencies, PDK compatibility and the trust rule) and 3 (packages from an index, the remote-host policy, licences and vendoring, the library browser's Packages section) are done; phase 4 (changesets and agents) is in progress; phases 5 and 6 are not implemented. Phases says what each phase delivers and, per phase, what exists today; what a user can do with PDKs now is in the PDKs guide, and with dependencies and packages in Libraries and dependencies.
The document has several baselines. Today describes the code when this design was written, and its line numbers are from then. In Phase 0 status, the items done in the first part of phase 0 were checked against the code right after that part, and the others against the code when phase 0 was finished; their line numbers are from those points. Phase 1 status describes the code of phase 1 by module and function, not by line, and Phase 2 status does the same for phase 2.
Goal¶
Libraries and PDKs are shared like cargo crates or npm packages: a project asks for analog_ip ^1.2 on sky130A,
Undine finds a version that fits, fetches it once into a shared store, pins exactly what it got, and every checkout of
the project gets the same files, checked by hash. Publishing a library is a pull request to an index repository.
Decided with the user:
- The registry is a git index repository (like crates.io-index or a Homebrew tap): name + version → git URL, commit, content hash. Publishing is a PR to the index. Private indexes are possible. It builds on today's git dependencies.
- PDKs: Undine wraps ciel. The PDK is a special root dependency: ciel
fetches and installs it, Undine pins the full commit and a tree hash, verifies it, and offers to fetch it (after the
person confirms) when it is missing. The process knowledge in
pdkprofile.pybecomes PDK profile packages.
Not goals: hosting a registry service (an index is a git repository, files stay in git repositories; a tapeout keeps
its own copy with pkg vendor, see Fetch and store), installing EDA tools (pixi does that),
running a package's code outside simulation (a package is data: cells and a manifest; simulating its cells runs what
they tell the simulator, see Security), several versions of one package in one project (see
Resolution), signing (later, see Decisions).
Today¶
The table describes the code when this design was written. Rows changed since say so: in the first part of phase 0 the
PDK row (OSDI is no longer compiled into the PDK tree) and the locking row (the CLI and the server now take a
project lock); in the rest of phase 0 the git dependencies row (format checked, leftovers, --dep-root) and the
netlist row (a name clash refused). Phase 1 changed the git dependencies, PDK and tools rows again; they are
left as they were, and Phase 1 status says what they are now. Phase 2 changed the git
dependencies row once more (manifests, transitive packages, local names), the symbol lookup row (a package's own
dependencies are installed for it, under the names its schematics use) and the netlist row (each line is attributed
to the cell body and the symbol it came from); Phase 2 status says how.
| Where | What it does / lacks | |
|---|---|---|
| git dependencies | store/deps.py |
{git, rev, subdir} in project.json libraries; undine.lock (format: 1) pins commit and sha (SHA-256 over every file's path and SHA-256, tree_sha, of the files git archive gave); installed read-only into .undine/deps/<name>/; add / update / remove / sync / lock / status. No manifest in the dependency, no version ranges, no transitive dependencies, no cache (a temporary bare repo per fetch). read_lock does not look at format. Since phase 0: read_lock refuses a format other than 1 (a newer one with "made by a newer Undine"), sync removes a dependency gone from project.json and status reports it as leftover, and HTTP takes a local repository only under a --dep-root |
| symbol lookup | xschem.py:311-324, project.py:219-234 |
C {<lib>/<cell>.sym} by the consumer's local library name: the schematic's own directory first, then the first hit on the search path. A dependency that places another library's symbols works only if the consumer added that library under the same name. The Xschem netlister backend (schematic.xschem_netlist) and Xschem opened outside Undine use the same path list (XSCHEM_LIBRARY_PATH) |
| netlist | schematic.py:589, xschem.py:482 |
a subcircuit is named by the symbol's file stem alone (subckts[symname]); two different cells of one name in one netlist: the first wins, silently (only a stand-in clash is refused, schematic.py:527-530). Since phase 0: two cells of one name that netlist to other lines are refused, naming both; the same lines are written once |
| layout | layout.py:1340-1377 |
placing a cell of another library copies it. A subcell of the same name with other content becomes name$1; a cell the layout already has under the placed name is kept as it is, whatever its content (_copy_cell_from returns nothing). The operation log keeps src: {lib} and the content sha, not the dependency's commit |
| PDK | pdks/pdk.py |
found once at start in $PDK_ROOT, ~/.ciel, ~/.volare (or --pdk-root, which replaces them); Pdk.version is the full commit of the PDK's source (open_pdks for sky130 and gf180mcu, IHP-Open-PDK for ihp-sg13), but Pdk.id (sky130A@8afc8346a57f, 12 characters) is what project.json, undine.lock and provenance record. No tree hash, no install, no rescan. A missing PDK opens the project with no PDK and status says only "locked X, here None". Installed trees are written to after install: sky130's libs.tech/klayout/python/cells/__pycache__ (when the PCell code is imported), IHP's libs.tech/ngspice/osdi/*.osdi (from scripts/osdi.sh; since phase 0 it compiles into Undine's cache, ~/.cache/undine/osdi/<PDK id>/, and leaves the PDK as installed) |
| process knowledge | pdks/pdkprofile.py |
one Python Profile per family (layers, decks, models, PCell library, stack rules); not versioned with the PDK. Supported commits are written in three places (pcell.SUPPORTED, pixi.toml, setup.sh) |
| tools | tools.py:43-63, tools.py:116-131 |
the lock compares version banners as text; which binary answers depends on the search order ($UNDINE_TOOLS or ~/undine-tools/bin before PATH), so CLI and server can disagree. Here klayout answers 0.28.17 from ~/undine-tools/bin while /usr/bin has 0.30.12, and neither is pixi's |
| provenance | store/provenance.py |
job inputs record cell content hashes (the netlist hash covers generated subcircuits, the layout hash copied cells), the PDK id, tool banners and the KLayout deck's SHA-256 (a changed deck makes results outdated). Model files, PCell code and the rest of the PDK are not hashed |
| changesets, agents | api.py (_no_preview), agent.py:20 |
dependency changes are written straight into the project, refused in a changeset preview; agents cannot add dependencies. project.json and undine.lock merge as text (merge.md) |
| locking | server/api.py, store/deps.py |
the server holds the server-wide srv.lock around dependency changes; the CLI holds no lock when it writes project.json and undine.lock, so a CLI and a server writing one project race. Since phase 0: a fetch holds only the project's .undine/deps.lock, and every writer of the two files (requests, the CLI, adopt and revert) takes meta_lock, an flock on .undine/project.lock; srv.lock is held only to swap the files in |
Problems found on the way, fixed in phase 0: read_lock accepts any format; set-pdk writes the lock
only when one exists; sync leaves the files and lock entry of a dependency removed from project.json (and the search
path still finds them); HTTP add / sync / update hold the server-wide lock during a git fetch of up to 600 s; the
CLI writes project.json and the lock with no lock at all; HTTP accepts file:// and absolute paths, so any
repository on the server's disk can be read; the netlister lets the first of two same-named cells win silently;
scripts/osdi.sh imports a module that no longer exists. The first part of phase 0 fixed the fetch under srv.lock, the CLI with no
lock and osdi.sh, and the rest of phase 0 the others; Phase 0 status says where each fix is and
which tests cover it.
Packages¶
A package is one library (a directory of cells, as today) plus a manifest undine-package.toml at its top. A git
repository may hold several packages in subdirectories. TOML because people write it by hand; Undine reads it with
Python's tomllib.
[package]
name = "analog_ip" # unique in its index; the default local library name; [a-z][a-z0-9_]*
version = "1.2.0" # semver
description = "Bandgap, LDO and comparator for sky130"
license = "Apache-2.0" # SPDX expression; required to publish
authors = ["Kadomoto <k@example.org>"]
repository = "https://github.com/me/analog_ip.git"
[pdk]
family = "sky130"
variants = ["sky130A"] # sky130A and sky130B have other metal stacks
compat = ["sky130/2025"] # compatibility tags (defined by the PDK profile package), or:
# commits = ["8afc8346a57fe1ab7934ba5a6056ea8b43078e71"]
built_with = "8afc8346a57fe1ab7934ba5a6056ea8b43078e71" # the PDK the layouts were drawn and checked on
[dependencies]
sar_common = "^1.2" # from the index this package is published in
bias = { package = "bias_gen", version = "~0.3" } # placed as bias/<cell>.sym in this package's schematics
tech_cells = { version = "^2", index = "acme" } # another index
[exports]
cells = ["bandgap", "ldo", "comparator"] # the public cells; the others are installed and netlisted too, but internal
[files]
include = ["*.sch", "*.sym", "*.oas", "*.netlist.yaml", "*.intent.yaml", "rtl/**", "README.md", "LICENSE"]
[checks] # how `pkg check` runs DRC / LVS on this package's cells
drc = { deck = "sky130A_mr.drc", options = { feol = true, beol = true, density = false } }
waive = [{ cell = "unit_cap", rules = ["m1.2"], reason = "abutment cell: clean once arrayed" }]
[renames] # old cell name -> new, per version that renamed it
"1.2.0" = { comp = "comparator" }
- PDK compatibility is a hard constraint. PDK commits have no order, so there are no ranges: a package lists
commits, or compatibility tags. A tag (
sky130/2025) is a promise, made by the PDK profile package (not by the index), that the commits under it give the same layers, rules, PCell geometry and device models (model files and corners). Tags have one level: a commit that changes a model, even with the same geometry, gets another tag, since the offsets, bandwidths and spec results of a published cell would change under it; there is no layout-only tag. The profile's[[pdk]]entries carry per-library tree hashes (see PDK profiles), so the review of an update or ofset-pdklists the commits whose model files differ. PCell geometry also depends on the generator: the PDK's PCell code run in Undine'spcellpixi environment (gdsfactory, kfactory), whichGEN_VERSIONrecords (it is in every variant's name).built_withis always one exact commit. - Binary layouts stay one
.oasfile per cell, as in a project library, so a new version shares the unchanged cells with the old one (see Store). No git LFS: an LFS pointer file in a package is refused with a message. - Checks (
[checks]): the deck and its options for the package's DRC (unit and abutment cells are often not clean alone, and density and antenna rules apply only at the top level), and waived rules per cell, each with a reason.pkg checkand the review show the waivers. - A published package depends only on index packages. A project may still use plain git dependencies (as today).
Implemented (phase 2): store/manifest.py reads this file (with tomllib, or tomli on Python 3.10) and refuses
it on the first problem, naming the key path and what to write (<where>: undine-package.toml: <key path>: <what>:
<fix>); unknown keys are refused, [checks] and [renames] are accepted as tables and not used yet. With no index
yet, [dependencies] takes a git form, and the index forms above are parsed but refused at resolution: see
Phase 2 status.
Implemented (phase 5): [checks] and [renames] are typed (manifest.Checks, Waiver, Manifest.renames), each
refusal naming its key path: drc takes deck (a deck file name, no directory) and options (true, false, a number
or a string each); a waiver needs a cell, at least one rule and a reason that is not empty, and its cell must be a
top-level cell of the include set; a [renames] key is a version at most [package].version, its old and new names
cell names, never one name to itself and never two old names to one new name in one version. These rules are
publishing's (pkg init, pack, check): a consumer (a fetch, lib sync, the lock, the library list) reads both
tables as before, of any content, so a package published or locked earlier keeps installing; the typed fields are
then empty when the tables break the rules. manifest.render writes a manifest back as TOML that parses to the same
manifest. See Phase 5 status.
Index¶
An index is a git repository:
config.json {"name": "undine", "format": 1}
an/al/analog_ip.jsonl one line per version, appended, never rewritten (except "yanked")
pr/of/profile_sky130.jsonl
3/i/inv.jsonl names of three letters: 3/<first letter>/
2/op.jsonl names of one or two letters: 1/ or 2/
One file per package, one JSON line per version. The path follows crates.io-index: a name of four letters or more
goes under its first two and next two letters, a name of three under 3/<first letter>/, of one or two under 1/ or
2/.
{"name": "analog_ip", "vers": "1.2.0", "git": "https://github.com/me/analog_ip.git", "subdir": "lib",
"commit": "229914698b1fb16a629bb73109b39eb1aba40eae", "tree": "sha256:e6d28176...",
"pdk": {"family": "sky130", "variants": ["sky130A"], "compat": ["sky130/2025"], "built_with": "8afc8346a57f..."},
"deps": [{"name": "sar_common", "req": "^1.2"}, {"name": "bias", "package": "bias_gen", "req": "~0.3"},
{"name": "tech_cells", "req": "^2", "index": "git@github.com:acme/undine-index.git"}],
"exports": ["bandgap", "ldo", "comparator"], "license": "Apache-2.0", "size": 1843200,
"check": {"url": "https://github.com/me/analog_ip/releases/download/v1.2.0/undine-check.json", "sha256": "5f2a..."},
"yanked": false}
tree is a hash over the commit's own bytes, the same on every machine: for each file the manifest includes (the
manifest too), in path order, its path below subdir, its mode (100644 or 100755) and the SHA-256 of its blob as
git stores it, read with git ls-tree -r and git cat-file. No git archive: archive applies .gitattributes
(export-subst, export-ignore, eol) and the machine's core.autocrlf, so a publisher, the index CI and a consumer
could hash one commit differently. A symlink or submodule entry is refused.
The encoding, exactly (phase 1; a golden test, test_tree_hash_golden_and_order_by_utf8_bytes in test_cas.py, pins
it):
- A file's object hash is the SHA-256 of its raw bytes: the blob's content, with no git header (
git hash-objectwould addblob <size>\0). It is written as 64 lower-case hex characters. - The tree hash is
"sha256:"+ the SHA-256 of the concatenation, for each file in the order of the UTF-8 bytes of its POSIX path belowsubdir, of the UTF-8 bytes off"{mode} {hex} {path}\0":modeis100644, or100755for an executable;hexthe object hash;pathrelative,/-separated, with no.or..part. - The store records a tree as
trees/sha256/<hex>.json,{path: [mode, hex]}, named by the hex of its tree hash. - A PDK part hash (Lockfile) runs the same encoder over the files on disk, with the owner's execute bit as the mode.
The commit and the tree are what pins a
version's content; there is no signature (signing is later). A version's commit and tree never change;
yanking sets yanked: true (a PR too): resolution skips it, an existing lock keeps it and status warns. check
is the link to the publisher's check report and its SHA-256 (see Publishing).
Implemented (phase 5): index.line_text writes a line as above: compact JSON, UTF-8 as it is, the keys in the
example's order with description after license, an absent value left out, deps and yanked always; it reads
back through parse_line to the same line. check must be exactly {url, sha256}: an https URL with no user or
password, and 64 lower-case hex characters. index.yank_rewrite marks one version yanked (or not) by rewriting its
line in place, every other byte of the file kept.
The default index is undine: a repository in the kadomoto-lab GitHub organisation,
https://github.com/kadomoto-lab/undine-index.git (created on 2026-11-20, empty so far). Its URL is in Undine's
configuration. For now kadomoto reviews its PRs.
Several indexes. More indexes are named in
$UNDINE_HOME/config.toml (UNDINE_HOME defaults to ~/.undine) and, so that a clone of the project works, in
project.json:
"indexes": {"acme": "git@github.com:acme/undine-index.git"}
A requirement with index is resolved in that index. One without it means the index of the package that declares it
(cargo's rule), and for the project's own requirements the default index. So a package from acme that depends on
sar_common gets acme's sar_common, never one someone published under that name in the public index. An index is
known by its URL: the nickname is local, pkg pack writes the URL into the index line, and the lock records the URL.
One package name coming from two indexes in one project is an error that names both. Access is git's: ssh keys or a
credential helper (GIT_TERMINAL_PROMPT=0 stays, there is no password prompt). Each index is cloned once into
$UNDINE_HOME/store/index/<sha256 of URL>/ and refreshed only on request (pkg index update, the Refresh button)
or when a resolution needs a version it does not have and the person agrees.
Resolution¶
- Start from the project's dependencies (
project.jsonlibraries, now also{"package": ..., "version": "^1.2"}) and the PDK. - For each package, the highest non-yanked version that satisfies every requirement on it, is compatible with the project's PDK commit and variant, and whose own dependencies resolve (backtracking; the graphs are small).
- One version per package name per project. Two requirements with no common version are an error that names both
chains (
app -> analog_ip 1.2 -> sar_common ^1.2vsapp -> sar_common ~1.0). Reason: SPICE subcircuits and layout cells are named by cell name, not by library, so two versions ofcomparatorin one netlist collide. Several major versions by prefixing cell names are left for later (Decisions 3). - Cell names are disjoint. The same reason applies across packages: the cells of all resolved packages (exported
and internal, since internal cells are installed and netlisted all the same), of the project's libraries and of the
PDK libraries must have different names. A clash is a resolution error that names the cell and both packages
(
comparator: analog_ip 1.2.0 and sar_common 1.3.1). One package placed under two local names is one package, not a clash. Prefixing subcircuit and layout cell names by library instead was not chosen (Decisions 4). - Local names. Each library declares the local names its schematics use (
bias/x.sym) and the package each one means; the lock keeps that table. Local names are project-wide: each resolved package is installed under every local name it is placed by (.undine/deps/<local name>/, hard links into the store, so a second name costs nothing), and the symbol search path stays one list of directories. That is what the Xschem netlister backend and Xschem opened outside Undine need:XSCHEM_LIBRARY_PATHis one global search by path and cannot express a per-library table. So a dependency's own dependencies work without the consumer adding them, a package can be placed under another local name (bias→bias_gen), and a diamond (A -> C,B -> C) installsConce per local name. Two libraries that give one local name different packages are refused, naming both. A local name may not be a PDK library name (asdeps.addchecks today) or the name of a project library. Per-library tables (one local name meaning two packages) were not chosen (Decisions 5). - PDK compatibility is checked for every package, not just direct ones. A project that changes its PDK (
set-pdk) is refused when a locked package is not compatible, with the list.
Implemented (phase 2), for git requirements only: with no index, each requirement names one git commit, so there is
one candidate per requirement and no backtracking. store/resolve.py walks the graph from project.json's
libraries: a version is (package name, tree), the manifest's version a label checked against an optional ^,
~, =, bare or * requirement. Every problem of the walk is collected and refused once, ending "nothing was
changed": two requirements on one package that do not agree (both chains, with version, rev and commit), a package
name other than the one asked for, a manifest-less target (only project.json may name a plain git dependency), an
index form, a local URL named by a remote requirer, a bad manifest, a local name claimed for two packages, a reserved
local name, a cell clash and a PDK the package does not accept. add and update re-resolve only the named
requirement's subtree; every other locked package is held as locked, read from the store or its install and never
fetched (store/locked.py). Cell names (rule 4) are compared casefolded among logic views (.sch, .sym, .sv,
.v): a project library contributes only those, so a layout copy placed from a package never clashes, and a
package's layout-only cell is excused against its own closure and the PDK. Local names (rule 5) are compared
casefolded, and besides project and PDK library names they may not be devices or a subdirectory of Xschem's library
or of a project.json symbol_paths entry, since .undine/deps precedes them on the one symbol path. PDK
compatibility (rule 6) is pdks/compat.py: family, variants, full commits and built_with; tags are not known yet
(phase 6). Two of the phase 2 changes for plain git dependencies come from here: status reports a
project-library logic cell named like a dependency's logic cell (an issue against a package with a manifest, a note
against a plain dependency; resolution refuses a clash only when a package with a manifest is on one side), and
lib add refuses a new local name that is reserved as above, while a name already in project.json is only noted.
Lockfile¶
undine.lock becomes format: 2.
{
"format": 2,
"pdk": {"id": "sky130A@8afc8346a57f", "family": "sky130", "variant": "sky130A",
"commit": "8afc8346a57fe1ab7934ba5a6056ea8b43078e71",
"source": "ciel static-web:https://fossi-foundation.github.io/ciel-releases",
"libraries": ["sky130_fd_pr", "sky130_fd_sc_hd"],
"tree": {"libs.tech": "sha256:41c0...", "libs.ref/sky130_fd_pr": "sha256:9a7e...",
"libs.ref/sky130_fd_sc_hd": "sha256:0f3b..."},
"verified": "profile_sky130 1.0.0"},
"profile": {"package": "profile_sky130", "index": "https://github.com/kadomoto-lab/undine-index.git", "version": "1.0.0",
"commit": "5b1c...", "tree": "sha256:77aa..."},
"packages": {
"analog_ip": {"package": "analog_ip", "index": "https://github.com/kadomoto-lab/undine-index.git", "version": "1.2.0",
"git": "https://github.com/me/analog_ip.git", "subdir": "lib",
"commit": "229914698b1fb16a629bb73109b39eb1aba40eae", "tree": "sha256:e6d28176...",
"names": ["analog_ip"], "deps": {"sar_common": "sar_common", "bias": "bias_gen"}},
"bias_gen": {"...": "..."}, "sar_common": {"...": "..."}
},
"libraries": {"old_ip": {"git": "...", "rev": "v1", "commit": "...", "tree": "sha256:..."}},
"tools": {"undine": "0.1.0 (3f9c2e1)", "pixi_lock": "sha256:13be...",
"klayout": {"version": "0.30.12", "from": "PATH"}, "xschem": {"version": "3.4.5", "from": "pixi"},
"ngspice": {"version": "47", "from": "pixi"}, "magic": {"version": "8.3.678", "from": "pixi"},
"netgen": {"version": "1.5.272", "from": "pixi"}}
}
- Format. Phase 0 makes Undine refuse a lock whose
formatit does not know, with "made by a newer Undine". Before that, an older Undine reading format 2 would report thepdkobject as an issue, and itsset-pdkwould overwritepdkwith a string. So format 2 is written only by an explicit command that needs it, never as a side effect of another change, and those commands come from phase 1, released after phase 0's check has been out: a collaborator on an older Undine gets a refusal, not a corrupted lock. In phase 1 the writers arepdk verify PROJECT --pinandlib lock PROJECT --format 2, both through one function (pdkops.upgrade);pkg addandpkg migratewrite it too once they exist, in phase 3. Withoutlib lock --format 2, a project with no PDK could not reach format 2 in phase 1 (both commands refuse a PDK that is not installed here:pdk fetchfirst). A format 1 lock is read and, by the commands of today (lib add,set-pdk, ...), still written as format 1. Implemented (phase 0):read_lockrefuses a larger integerformatwith "made by a newer Undine", and any other value but 1 (a string,true,1.0,0) as one "no Undine writes"; a lock withoutformatis read as format 1. Every command that changes the project reads the lock first, so a refused lock leavesproject.json,undine.lockand.undine/deps/as they were. Implemented (phase 1):read_lockreads formats 1 and 2 and refuses 3 or more the same way ("this one reads formats 1 and 2"). A format 2 lock stays format 2 under every command, and a command that changes it (lib add,update,set-pdk,lib lock) keeps what it does not change. - Packages are keyed by package name, which is unique in a project because one name from two indexes is refused;
an entry is (index URL, name) and
syncchecks both.namesare the local names it is installed under. - PDK: the full commit (open_pdks for sky130 and gf180mcu, IHP-Open-PDK for ihp-sg13), ciel's source, the
libraries fetched, and one tree hash per part, so a PDK installed with more libraries still verifies. The hash
leaves out what gets written into a PDK tree after install:
__pycache__/and*.pyc(from importing the PCell code; the PCell worker also runs withPYTHONDONTWRITEBYTECODE=1from phase 1) andlibs.tech/ngspice/osdi/(compiled there byscripts/osdi.shon installs set up before phase 0). The expected hashes come from the profile package's[[pdk]]entry for that commit (verifiednames it), so a PDK is checked against published hashes, not against whatever the first machine had. Hashing one sky130 variant (1.1 GB) takes seconds once; Undine keeps the result with file stamps in$UNDINE_HOME/store/pdk/and re-reads a file only when its stamp changed.project.jsonpdkkeeps the id (the requirement); the lock is the pin, andstatusreports them disagreeing.set-pdkalways writes both. - Tools (decided): tools stay out of the package system; pixi pins them (Undine's
pixi.lock). The lock records Undine's version and commit (0.1.0 (3f9c2e1),+dirtyafter the commit when tracked files are changed; another version warns, another commit of one version is only a note), the SHA-256 of itspixi.lock(nullwhen Undine was installed without pixi;statusthen says the tools are not pinned), and per tool its version number and where the binary came from (pixi,UNDINE_TOOLS,undine-toolsfor~/undine-tools/bin,envfor the server's own conda or virtual environment that is not pixi's,PATH):tools.findlooks in$UNDINE_TOOLSor~/undine-tools/binbeforePATH, so thepixi.lockhash alone says nothing about which binary answered. Another number or origin is a warning instatus, never a refusal. A format 1 lock records the banners only: it has noundineorpixi_lock, its tools no origin, andstatusnotes that it cannot compare them. The staletools.klayoutofproject.jsonis dropped (no longer written or read; an existing key is left on disk). The number is parsed from today's banners, one rule per tool (a banner no rule matches is recorded as text with no number, andstatussays so):
| Tool | Banner today | Rule |
|---|---|---|
klayout (-v) |
KLayout 0.28.17 |
KLayout (\d+(\.\d+)+) |
xschem (-v) |
XSCHEM V3.4.5 |
XSCHEM V(\d+(\.\d+)+) |
ngspice (-v) |
ngspice-47 : Circuit level simulation program |
ngspice-(\d+(\.\d+)*) |
magic (--version) |
8.3.678 |
^(\d+(\.\d+)+)$ |
netgen (-batch quit) |
Netgen 1.5.272 (after the build date is cut, as today) |
Netgen (\d+(\.\d+)+) |
librarieskeeps plain git dependencies as today. In format 2 theirtreeis computed as an index tree is (from git's blobs, over every file of the subdir, or over the manifest'sincludewhen the dependency has a manifest); a format 1sha(overgit archiveoutput) is kept until that entry next changes.- Implemented (phase 2): with no index, the lock's new keys are fewer than the example above shows.
librarieskeeps every directproject.jsongit dependency: a manifest-less one, or a phase 1 one not resolved again, is exactly the phase 1 entry; one resolved in phase 2 with a manifest addsfiles: "include",package,version,names(every local name, its key first) anddeps({local name: package}).packages[package]holds the packages reached only through manifests:{package, version, git, rev, subdir?, commit, tree, files: "include", names, deps}, with noindex(phase 3); a sibling records its requirer'sgitandrev. One package name appears once across thepackageskeys andlibraries[*].package. A tree covers the manifest'sincludeset only when the entry saysfiles: "include"; with nofileskey it covers every file of the subdir, as in phase 1, andsyncnever adds or removes the key. A format 1 lock never gets these keys: a manifest that names dependencies, or a package name other than its key, is refused there with "needs lock format 2: runlib lock PROJECT --format 2first", and that command converts and resolves in one write, or leaves the format 1 lock as it was.syncstays frozen over the whole graph: it installs everylibrariesandpackagesentry under each of itsnames, from the store when it can, and refuses an incomplete graph (a missing package, a name claimed twice or reserved) before anything is read. A lock edited by hand is still trusted as it stands, see the trust note under Security.
Fetch and store¶
A shared, content-addressed store, $UNDINE_HOME/store/ (~/.undine/store/ by default; UNDINE_STORE moves the store
alone):
objects/sha256/e6/d28176... every file, by its SHA-256, read-only
trees/sha256/e6d28176....json a package version: {path: [mode, object hash]}
trees/archive/<sha>.json the same for a format 1 entry, named by its `sha` (git archive's files)
git/<sha256 of URL>.git bare repositories fetched from, kept for the next fetch
index/<sha256 of URL>/ index clones
pdk/<commit>-<variant>-<root>.json PDK tree hashes and file stamps; <root>: the first 12 hex of the SHA-256 of the
PDK's real path, so one commit under two roots keeps two records
(nocommit-<root>-<name>.json for a PDK with no version)
build/<commit>/<variant>/osdi/ compiled OSDI models (no longer written into the PDK tree; local-<root> in place of
<commit> for a PDK with no version)
projects/<sha256 of path>.json the projects that added, updated, synced or locked a dependency: {path, synced}
tmp/ every write lands here first and is renamed into place
.lock an flock: shared for a dependency operation, exclusive for `store gc`
The store is $UNDINE_STORE when set, else $UNDINE_HOME/store; UNDINE_HOME (default ~/.undine) also holds the
connections. The variables, and UNDINE_OFFLINE, UNDINE_STORE_LINK, UNDINE_CIEL and UNDINE_FETCH_MAX, are in
Data layout.
- Fetch:
git fetch --depth 1of the commit into the kept bare repository (all refs only when the server will not serve a commit by itself, as today), then the files are read straight from git's objects (git ls-tree -r -z,git cat-file), as the tree hash is defined; only regular files are taken. Every file goes intoobjects/and the tree is checked against the lock or the index entry before anything is installed. Unchanged.oascells of a new version are already in the store: per-file dedup. Implemented (phase 1,store/gitsrc.py): as described, for plain git dependencies; the file count (100 000) and the size (UNDINE_FETCH_MAX, 1 GiB) are checked fromgit ls-tree -lbefore any blob is read. A format 1 entry keeps itsshaovergit archiveoutput (phase 0's definition,.gitattributesandcore.autocrlfapplied): the archive runs in a throwaway repository that holds what phase 0's depth-1 fetch into an empty repository held, the commit fetched at depth 1 from the kept repository (a local read) and none of its refs, so thatexport-subst(%d,%D,%(describe)) expands as it did then,%hhas the same length (git sizes an abbreviation by the objects it can see, which a kept repository that grew would change), and theshacomes out the same. A path with a.gitpart (any case,.git.,git~1), which git itself refuses in a tree, is refused byblob_treeand by every install. - Install: each
.undine/deps/<local name>/is made from the tree by hard links intoobjects/(copies across file systems, or withUNDINE_STORE_LINK=copy). The files are read-only;store verifyre-hashes the objects,statusre-hashes the installed trees as today. Implemented (phase 1): a copy is also made where the file system refuses a link (EPERM,EMLINKat its link limit,ENOTSUP); copies are read-only too. Every object is re-hashed before it is linked, and an object that fails is read again from git. A changeset's worktrees hard-link the installed dependencies too (Changesets on git branches). - Sync is frozen:
syncinstalls exactly what the lock says and refuses a lock that no longer satisfiesproject.json(after a pull that changed a requirement), naming the entry and saying to runpkg lockorpkg update. Resolution happens only inadd,update,lockand changeset operations, never as a side effect of a sync, so a sync never changes what evidence or an agent sees. Today'ssyncre-resolves a changed or unlocked dependency; that changes in phase 1 (lib lock, which exists, andpkg lockre-resolve on request). Implemented (phase 1):lib syncrefuses before anything is read or fetched, naming each entry (analog_ip: project.json asks for rev v2, undine.lock locks v1, ornot in undine.lock) and saying to runlib lock PROJECTorlib update PROJECT NAME;lib lockre-resolves what the lock does not pin asproject.jsonnames it (fetching it), and only that. A sync never rewrites a lock entry: a format 1 entry keeps itsshauntiladd,updateorlib lockwrites the entry into a format 2 lock, where it gets atree. - Offline (
--offline,UNDINE_OFFLINE=1, or no network): resolution uses the index clones as they are;syncworks when every locked tree is in the store, else it names what is missing. A clone or pull of a project is followed bysyncfrom the store, which needs no network for versions fetched before. Implemented (phase 1):lib sync --offline(orUNDINE_OFFLINE=1, or{"offline": true}over HTTP) installs from the store's records and objects, then from a kept repository that has the commit, and refuses naming every entry it has neither for (offline, and not in the store: analog_ip @ 229914698b1f (sha e6d28176...)); nothing is changed. A dependency installed before the store existed is imported into it by the first sync whose files match the lock, so it then syncs offline too. - PDK:
pdk fetchrunsciel fetch --pdk-family <family> <commit> -l <lib>...with the lock's source, into the first PDK root Undine scans that is a ciel root (--pdk-rootwhen given, else$PDK_ROOT, else~/.ciel; ciel's--pdk-rootis set to it), and refuses when none of the scanned roots is one (only~/.volare, say), since a PDK fetched elsewhere would not be found. ciel installs into its own store there (not copied); Undine then hashes it, compares with the lock and rescans the PDKs (new: rescan without a restart). ciel stays the installer; Undine never edits a PDK tree. Implemented (phase 1): the ciel family is taken from the variant (ihp-sg13g2→ihp-sg13); a root qualifies when it hasciel/, does not exist yet or is empty. A format 1 id, or a pin from a ciel install, assumes ciel's default source,static-web:https://fossi-foundation.github.io/ciel-releases; a pin from volare is refused. A 12-character id is expanded from the installed PDKs, else withciel ls-remotewhen the person allows the network (--network, or the dialog's checkbox). The CLI asks before it runs ciel; over HTTP the fetch needs the confirmation token (Security) and runs as a job. The CLI cannot rescan a running server: it says to use Rescan PDKs orPOST /api/pdks/rescan. store gcremoves objects no project on this machine locks (projects are found from the server's--rootand a list in the store of projects that synced). Implemented (phase 1): the projects are those under--root, the locks in their changesets' worktrees and the store'sprojects/list; what each one locks and what is installed in its.undine/deps/stay. A lock it cannot read stops it before anything is removed. It removes trees and objects only (andtmp/entries older than a day):git/,pdk/andbuild/stay.- Vendoring (
pkg vendor). Files stay in the publishers' repositories, and a repository that is deleted, made private or force-pushed (its commit then garbage-collected) loses that version for every machine whose store lacks it. So a tapeout keeps its own copy:pkg vendor PROJECT DIRwrites, from the store, the pinned tree of every locked package, plain git dependency and the profile package intoDIR/vendor/<package>@<version>/(git dependencies as<name>@<commit>), withvendor/vendor.json(per entry the index URL, git URL, subdir, commit and tree hash, as in the lock) andvendor/LICENSES.jsonandvendor/LICENSES.mdfrompkg licenses. Each written tree is hashed again and compared with the lock; a tree not in the store is fetched first (asks first) or, offline, named and refused. The tapeout export runs it into the bundle (<project>/tapeout/<target>/<top>-<time>/vendor/), andreport.jsonlists the vendored trees. The PDK is recorded invendor.json(commit, ciel source, per-library tree hashes, licence) but not copied: ciel's releases keep it, and a sky130 variant is 1.1 GB.pkg sync --from-vendor DIRreads a vendored directory back into the store, checking every tree against the lock, so the project's dependencies can be installed again with no repository and no network. Implemented (phase 3, store/vendor.py): the trees come from the store, else from an install that is exactly the locked tree; the CLI asks before it fetches one in neither (--yes,--offline), the tapeout export never fetches. The trees are written as plain files (0o755 for mode 100755) intoDIR/.vendor.new-<pid>, each hashed again, and the directory renamed tovendor/only when all match: avendor/that exists is whole, and one that exists already is refused.vendor.jsonalso records the lock's format and SHA-256.--from-vendortakes the bundle or itsvendor/, matches each lock entry by commit and hash, hashes every directory (a link or special file refused) before any object enters the store, then runslib sync --offline; an entry the bundle lacks is left to the store. When the export cannot vendor, the bundle is written withoutvendor/,report.jsonhasvendor.errorand avendorcheck warns. The profile package is not vendored yet (phase 6).
Publishing¶
undine pkg init libs/analog_ip # writes undine-package.toml from the library (cells, PDK of the project)
undine pkg check libs/analog_ip # what a publish requires, below
undine pkg pack libs/analog_ip # tree hash at the tagged commit; prints the index line
undine pkg publish libs/analog_ip --index undine # branch + commit in a clone of the index, opens the PR
check requires:
- a clean worktree at a tag that the
repositoryURL serves:v<version>, or<name>-v<version>when the repository holds more than one package; - every exported cell has a symbol that passes
check_symboland a schematic that netlists; - every cell with a layout passes DRC and LVS against
built_with, with the deck, options and waivers of[checks]; - the intent specs of exported cells that have them are run (
check_specs); for analog IP they are the contract; - dependencies resolve from the index; a license; no file over a size limit (default 50 MB) and no LFS pointer;
- the package's code blocks (
.control,shell,.include/.libpaths) and RTL are listed, and the ones the trust rule would refuse are flagged.
Against the previous version it reports semver changes: an exported cell removed or renamed without a
[renames] entry, a symbol pin removed or renamed, or a change to an exported cell's layout bounding box, pin
positions or pin layers (it breaks a consumer's placement and routing) needs a major version (minor before 1.0); a new
cell needs a minor version; another layout change needs at least a patch. A spec result that changed is listed too.
The check report. check writes undine-check.json: the package name, version, commit and tree hash it ran on,
the PDK commit (built_with) and the tools (as in the lock's tools), per cell the DRC and LVS results with the
waivers applied, the spec results, the semver report, the flagged code blocks and RTL, and the job ids of the runs in
the publisher's project. A report is valid only for the tree it names.
Publishing. publish pushes no commit or tag to the package's repository. It fetches the commit back from
repository and checks the tree, then requires a passing report for that same tree (it runs check again when there is
none), and puts the report where the index PR can link it: as an asset of the tag's GitHub release (gh release
upload, after the publisher confirms) when the repository is on GitHub, or at a URL the publisher gives with
--check-url. It writes the index line with check: {url, sha256}, then opens the PR with gh when it is installed,
otherwise it prints the branch and the URL to open it. The PR body has a summary of the report (DRC / LVS per cell,
waivers with their reasons, spec results, semver report, flagged code blocks and RTL) and the link. The reviewer of
the default index (kadomoto, for now) reads that summary; the evidence is the publisher's run, said as such in the PR.
The index's own CI (a workflow file Undine provides) repeats what needs no PDK: the fetch and tree check, the name rules, the code-block and RTL listing, and the report check (the linked file's SHA-256 equals the line's, and it names the line's commit and tree and passes). DRC and LVS in the index CI, which needs the PDK there (1.1 GB per sky130 variant), come later (Decisions 9).
Implemented (phase 5, so far): pkg init, pkg pack and pkg check --no-run (store/pkgsource.py,
store/checkreport.py), which writes the report with the checks that need no PDK; the runs are not made yet, so the
report does not pass. See Phase 5 status.
Cells across versions¶
A cell is known by its name. [renames] maps old names to new per version, so an update can say "comp is now
comparator" instead of "removed + added", and rewrite the consumer's C {analog_ip/comp.sym} references when the
person agrees (in the changeset the update is). Stable cell ids were considered and not chosen: every cell file would
need one, and names are already the identity in schematics, layouts and netlists.
pkg update (and the review of an update) reports per exported cell: added / removed / renamed / changed, which
views changed (symbol pins, schematic, layout, intent) and which spec results changed; the consumer's cells that place
it (today's used_by); layout copies that still have the old geometry, found by the operation log's origin, which now
records src: {"lib": "analog_ip", "package": "analog_ip@1.2.0", "tree": "sha256:..."} (copies made before that are
matched by their content sha against the old version's cell); and the results that become outdated.
Refreshing layout copies. Placing a package cell copies its layout into the consumer, and placing or syncing it
again does not replace a cell the layout already has under that name. So dep_update also refreshes the copies: each
copied cell whose origin is the old version is replaced in the changeset by the new version's cell under the same
name, with its subcells (a subcell of the same name with other content becomes name$1, as today). A copy the consumer
has edited since (its content no longer equals the old version's cell) is not replaced: it is listed for the person.
The review shows each refresh as a layout diff; the DRC and LVS of the consumer cells that place it are outdated
(their layout hash changed) and are re-run in the changeset. Without this, the schematic and the LVS reference would
come from the new version while the layout kept the old geometry.
PCell variants and PDK versions¶
A generated PCell cell is named by the generator generation and its parameters (pcell.cell_name), not by PDK
commit; all of a project's cells are generated under its one PDK. A package ships its variant cells as generated under
built_with. Under a compatible tag those are taken as they are (the tag promises the same geometry; once the index
CI has a PDK, later, it can regenerate them under each listed commit, in the same pcell environment, and compare).
When a copied cell brings a variant subcell whose name the consumer's layout already has with other content, the
copy's subcell becomes name$1 (as today for subcells) and the update report lists it. PDK commits stay out of
variant names: putting them in would rename every existing variant for nothing within one compatibility tag.
PDK profiles as packages¶
pdkprofile.py becomes data: one package per ciel family (profile_sky130, profile_gf180mcu, profile_ihp_sg13)
with profile.toml holding what Profile holds today (layers, metals, decks and options, Magic / Netgen files, models
and corners, PCell library, stack rules, LVS elements, limits) and:
schema = 1 # the profile format; an Undine that does not know it refuses the profile
undine = ">=0.1.0" # the Undine versions that read this profile
[[pdk]]
commit = "8afc8346a57fe1ab7934ba5a6056ea8b43078e71"
compat = ["sky130/2025"]
tested = true # replaces pcell.SUPPORTED, pixi.toml's and setup.sh's copies
license = "Apache-2.0"
tree = { "libs.tech" = "sha256:41c0...", "libs.ref/sky130_fd_pr" = "sha256:9a7e...",
"libs.ref/sky130_fd_sc_hd" = "sha256:0f3b..." } # expected PDK tree hashes, per part
No code: the one computed field (LvsElement.params, sky130's MIM area and perimeter) becomes a named formula that
Undine implements (params = "mim_area_perimeter"). Unknown fields are refused, not ignored: Profile grows with
Undine's features (straps, rings, fill, stacks, erc_tap, ...), and an Undine that silently dropped a field would run
DRC or LVS differently. verified and magic_drc stay with Undine, in a table keyed by family and profile version:
they record what Undine's own tests have run, not facts about the PDK. Undine ships its current profiles built in as
the fallback with no index. A profile change (a deck option, a model path) is an update of the profile package,
reviewed like any other; the profile fields a job uses are in its inputs (see evidence),
so results that used a changed field become outdated, and the others do not.
Implemented (phase 6, pdks/profiledata.py, pdks/formulas.py, pdks/tested.py): one package per ciel family, not per
process (decided): profile_ihp_sg13's profile.toml has a [[profile]] for SG13G2 and one for SG13CMOS5L, each
naming the PDK variants it describes. Undine's built-in copies are in undine-eda at
server/undine/data/profiles/<package>/: an undine-package.toml (the package's name and version, 1.0.0 now, and
[files] include = ["profile.toml"]) and the profile.toml, so that an index line's subdir can point at them and the
built-in fallback is the published package byte for byte. A key is the Profile field of its name (name is
Profile.family) and is converted by the field's annotation: a list becomes a tuple, a table a dataclass, an integer
in a float field a float; an untyped table (drc_options, lvs_options, rename) keeps the TOML types, since a deck
gets str(value). Refused, each with its key path (<where>: profile.toml: profile[0].stack.cuts[2].name: ...): an
unknown key (verified and magic_drc too, with a hint), a schema above 1, an undine range that leaves this Undine
out, a formula or stack generator Undine does not have, an unknown drc_switches, a template placeholder the field
does not fill ({name.attr} included), a regex that does not compile, a path that is absolute or has a .. part, a
PCell library whose module, function or library is not a dotted name or whose environment is not one of Undine's
(pcell, pcell-gf180), a variant in two [[profile]] tables or of another family, a [[pdk]] commit listed twice,
a tag of another family and a tree value that is not sha256:<64 hex>. Besides the named formula, a stack may come
from a named, versioned generator (decided), as GF180's and IHP's do: [profile.stack] generator = "gf180_stack@1"
with its arguments beside it (thin, top, plate; ihp_stack@1 takes thin and tops); a changed rule is a new
version of the generator. verified (Undine's tests ran the profile) and magic_drc come from Undine's table
(pdks/tested.py) by family, profile version and profile name; a version it does not list is not verified and takes
magic_drc from the newest version listed (decided). The built-in [[pdk]] entries are the tested commits (sky130
8afc8346a57f, gf180mcu 792ef67037d7, ihp-sg13 89d42ee6d2fd) with their licence; compat and tree are read and
checked, and no built-in profile has them yet. The TOML files were written from the Python profiles, and
test_profile_data.py compares the loaded profiles field by field, types included, with golden data saved from the
Python before it was removed (server/tests/data/profiles_golden.json, rewritten by profile_golden.py after a
deliberate change). pcell.SUPPORTED is gone: profiledata.tested_commits(family) gives a family's tested commits;
pick_pdk prefers one of them for a bare PDK name (for GF180 and IHP too now), and pixi run pdk (python -m undine
pdk enable sky130) and setup.sh (python -m undine pdk tested FAMILY) name no commit of their own.
Licences¶
license (SPDX) is required to publish; PDK licences come from the profile package. pkg licenses PROJECT lists the
licence of every locked package and the PDK; pkg vendor writes that list into the tapeout bundle next to the
vendored trees (see Fetch and store). Third-party IP and reference data are fetched as
packages, not copied into Undine or into a project (for example server/undine/data/tt_sky130A_analog.json is a
candidate).
Implemented (phase 3): a package's licence is its manifest's license, else the local copy of its index's line
(read only, no fetch); a plain git dependency records none, and its top-level LICENSE, COPYING, NOTICE or
COPYRIGHT files are listed. Until profile packages exist the PDK licences are a table in store/vendor.py (sky130,
gf180mcu and ihp-sg13: Apache-2.0).
Implemented (phase 5): store/spdx.py checks an SPDX expression by its syntax only (SPDX 2.3, Annex D: identifiers,
+, WITH, AND, OR in upper case, parentheses, LicenseRef- and DocumentRef-). No licence list is shipped, so
a new identifier needs no Undine release and a misspelt one is for the reviewer to see. pkg init --license and pkg
check use it; the manifest parser does not, so a manifest that parsed before still parses.
Changesets and evidence¶
A dependency change is a changeset operation, like any other edit, reviewed and adopted:
dep_add/dep_update/dep_remove/pdk_setchangeproject.jsonandundine.lockin the changeset's working copy and install into its.undine/depsfrom the store (hard links, cheap). The changeset can then place the new cells and run checks. The preview no longer refuses dependency changes.dep_updaterefreshes layout copies (see Cells across versions). Implemented (phase 4, store/changeset.py: Changesets.dep_op, pdk_set): the operation runs deps.add / update / remove (Project.set_pdk) on the working copy, never on the project nor the baseline (refused before anything is installed). It fetches under the working copy's own.undine/deps.lockand commits under a guard (_DepGuard) that takes the changesets' lock and then the server's, checks the changeset again (open, no interrupted merge, the write lease the caller's) and appends the record to the working copy's project loghistory/@project.jsonl(a name no library can have, as@is not in a library name):{op: dep_add | dep_update | dep_remove | pdk_set, args, actor, reason, cell: null, view: null, effects: [{k: "d", do: add | mod | del, name, before: {project, lock}, after: {project, lock}, state: "installed", pdk?: [old, new]}]}, both files parsed whole. The readers of a changeset's records (the timeline, the review's changes, restore, the adopt's records and logs, the agent's write limit) take that log besides the libraries' without making it a library. Restore takes the operations back newest first (project.json key by key, undine.lock as it was), records that as onerestorerecord of the project log and installs from the store (an offline sync of the working copy); while an operation of the changeset still fetches, a restore is Busy rather than waiting (the operation waits for the changesets' lock that the restore holds).dep_update's refresh of layout copies is not built yet.- The review shows the lock diff, the update report above, licences, the download size, and the package's code blocks and RTL files (see Security).
- Adopting merges
project.json(libraries,indexes,pdk) andundine.lockentry by entry, then resolves again; when the merged requirements do not resolve, or the merged lock does not satisfy them, the adopt is refused with the reason. Then it installs in the project from the store; nothing is fetched at adopt. This is new: today adopt is a git merge and both files merge as text (merge.md), so two changesets that each change a dependency would make line conflicts, or merge cleanly into a lock that does not match the installed trees. Phase 4 adds a merge rule for the two files to Undine's git merge driver. Implemented (phase 4, vcs/depmerge.py): both files merge as JSON entry by entry, in the git merge driver (git-setupregisters it for them) and in a changeset's adopt (branches.plan_merge); the rule per key is in merge.md. "Resolves again" is, at adopt, a check from the merged files alone (deps.merged_problems: every requirement pinned as named, the locked graph complete): it fetches nothing and chooses no version, and a problem the project's files did not have before is a conflict ofundine.lock, the adopt refused with the reason. Implemented (phase 4, the install): a lock entry the merge brings or changes whose files are neither in the store nor installed in the project as locked is a conflict ofundine.locktoo (not in the store: fetch it first, deps.not_stored), so the adopt stays all or nothing with nothing fetched. Once the merged files are written (meta_lock let go, before the netlists are read again), a merge that changedundine.lockor project.json's dependencies installs them in the project with an offline deps.sync; a failure is said (deps_warningin the record'sapplied/reverted), not a failed adopt. merging.json notes the install, so a server stopped before it installs at its next start. The install never waits for the project's dependency lock (it runs with the changesets' lock and the project's editors, layouts and logs held): while another operation holds it, the install is left pending (deps_pending) forlib sync. A revert goes through the same merge, and takes anundine.lockthe changeset created back out: the dependency the changeset added is uninstalled, and the cells of the project that still place its cells are named (deps_dropped,deps_warning; Decisions 19). - A person's own Add in the library browser makes the same operation in a small changeset and adopts it in one
click, so there is one path.
Implemented (phase 4, POST /api/p/{p}/deps/changeset): the browser's Add (the git form and the Packages dialog),
Update, Remove and its move to another PDK each open a small changeset (git-setup first in a project in no
repository), run the operation in it and adopt it. An operation refused discards the changeset; an adopt refused
keeps it open for review (the 409's body names it) and the browser says so. The
/depsroutes and the CLI change the project directly, as before. - Agents may propose
dep_add/dep_update/dep_removein their changeset, using the index clones as they are. They never fetch: when a tree is not in the store, the operation is recorded as needs fetch (URL, size) and the review shows a Fetch button. Only a person's confirmation fetches (see the token under HTTP), and the same goes for refreshing an index and for fetching a PDK. Until the fetch, the review marks the operation unchecked (no cell of the package was placed or checked in the changeset). When the person's fetch ends, the agent's changeset gets a notice (an agent waiting withwait_forsees it), the operation becomes installed, and the agent can place the cells and run checks. - Evidence already hashes
project.jsonandundine.lock. Job inputs additionally record, outsidekey(for information, so evidence names exact contents): eachpackage@versionand tree used by the job's cells, the PDK commit, and the profile package and version. Outdated detection stays on what the job reads: the cells' content (the netlist hash covers generated subcircuits, the layout hash copied cells), the deck SHA, and, new, the model files the netlist includes (.lib/.includetargets and OSDI files, by hash) and the profile fields the job uses (deck options, deck variables, model paths). A changed README or an unrelated cell in a package, or a layer colour in a profile, makes nothing outdated. Implemented (phase 4, store/provenance.py): insidekey, a simulation's inputs gainmodels(every file its netlist reads through.lib/.include, transitively, a.libline by its section's files, and the OSDI files, as{path, sha}of the original files, not the reduced copies;modellib.files_read; the hashes cached by path, time and size) andprofile(model_lib,model_includes,osdi,corners); KLayout DRC and LVS gainprofile(drc_optionsof the deck orlvs_options, anddeck_varsof the PDK). Outsidekey, every job's status hasabout:packages({name, package, version, tree | sha, index}of each lock entry one of whose installs holds a file the job read: its netlist's schematics and symbols, its model files, its layout's library),pdk({id, commit, family, variant, tree}, the tree being the lock's pin) andprofile({family, source: "built-in", undine}; profile packages are phase 6).recomputegivesmodelsandprofileonly to a job that recorded them, so a job from before stays fresh rather than unknown;diffsays "model file X changed" and "profile field Y: a → b". A changeset's evidence manifest lists each side'spackagesandpdkfrom its archivedundine.lock, and each job row carries the job'sabout(Job provenance). Magic DRC, Netgen LVS and extraction record no profile fields (their setup files are not profile fields).
UI, CLI, HTTP, agent tools¶
- Library browser: a Packages section searches the index clones; a package page shows versions, licence,
description, exported cells with symbol previews, dependencies, and whether it fits this project's PDK (and why
not). Dependency rows show
analog_ip 1.2.0and, when there is one, the newer version. Implemented (phase 3, store/packages.py, web/src/PackagesDialog.tsx): a Packages dialog from the library browser's header: an index picker, a search, the list (newest live version, licence, a PDK warning) and a package's page (every version with yanked ones struck through and the PDK fit of each, repository, commit, exports, needs) with Add (local name, requirement; clicking a version requires exactly it). The first read of an index fetches it, under the --dep-root / --dep-host rule; refresh fetches it again. A dependency row from an index shows→ VERSIONfrom the local copy (never fetched for it), and its Update takes a new requirement. Exported cells are listed by name: symbol previews need the package's files, fetched only on add, and are left for later. - Lock badge:
locked/N lock issuesas today, plus the PDK (verified / unverified / missing → Fetch...), available updates and yanked versions. Implemented (phase 1): the PDK's state is on the PDK badge, whose menu has a hint for the state, Fetch... (when the PDK is missing or modified) and Rescan PDKs; the lock dialog shows the lock's format, how each dependency is pinned (shaortree, and not in the store), the PDK row, and the warnings and notes in a lighter style under the issues. - CLI (
undine pkg ...;lib ...stays for plain git dependencies):
| Command | |
|---|---|
pkg search TEXT, pkg info NAME[@VER] |
from the index clones |
pkg add PROJECT NAME[@REQ] [--as LOCAL] [--index I] |
resolve, fetch (asks first), lock, install |
pkg update PROJECT [NAME] [--to VER] [--dry-run], pkg lock, pkg outdated, pkg tree, pkg why NAME |
lock re-resolves without fetching |
pkg remove, pkg sync [--offline] [--from-vendor DIR], pkg licenses |
sync installs the lock as it is |
pkg vendor PROJECT DIR |
the locked trees and their licences, for a tapeout bundle |
pkg init, pkg check, pkg pack, pkg publish, pkg yank NAME@VER |
publishing |
pkg index list / add NAME URL / update [NAME] |
indexes |
pdk list, pdk fetch PROJECT, pdk verify PROJECT [--pin] |
PDKs through ciel |
store verify, store gc |
the store |
- HTTP:
GET /api/packages?q=,GET /api/packages/{name},POST /api/indexes/{name}/refresh, changeset operationsPOST /api/p/{p}/changesets/{id}/deps,POST /api/p/{p}/deps/fetchandPOST /api/p/{p}/pdk/fetch(jobs with progress),POST /api/pdks/rescan; the existing/depsendpoints keep working. All of them go intodocs/reference/api.md(today/deps,POST /libs,GET /api/pdksandPOST /api/p/{p}/pdkare missing there). Implemented (phase 1):POST /api/pdks/rescan,GET /api/p/{p}/pdk/fetch-plan(what the fetch dialog shows),POST /api/p/{p}/pdk/fetchandGET .../pdk/fetch/{job};/deps/synctakes{offline}and/deps/lockre-resolves; all in the HTTP API. There is no route forlib lock --format 2orpdk verify --pin: the command line only, in phase 1. - Confirmation token. A fetch, an index refresh and a PDK fetch over HTTP need one. The UI asks for it with
POST /api/confirmations {kind, url, commit, tree}only after the person clicks in a dialog that shows the URL, size and licence; the token is single-use, lives five minutes, is bound to that exact (URL, commit, tree) (for a PDK: commit, variant and libraries), and is returned only in that response. No MCP tool,undine callcommand, Python client method or built-in agent tool asks for one, carries one or returns one. Said plainly: Undine has no authentication yet, and the actor of a request is what the request says (as for adopt and evidence prune today), so a process with a shell and the person's access to the server can do what the person's browser does. The token stops agents that work through Undine's tools, and mistakes; it is not a defence against a hostile process running as the person. Implemented (phase 1), for PDK fetches only (decided: phase 4 generalises it to the other kinds):POST /api/confirmations {kind: "pdk", project, commit, variant, libraries, allow_network?}checks the request against the fetch plan the server makes now (commitmay be the plan's first 7 or more characters, as a format 1 id has 12) and binds the token to the project and the whole plan (family, variant, full commit, libraries, source, root); any other kind, a changeset preview and a source other than ciel's default are refused.POST /api/p/{p}/pdk/fetch {token}uses the token up, makes the plan again and refuses one that changed since; no token, an unknown, used or expired one, or one for another project is a 403. The fetch runs as an in-memory job (GET /api/p/{p}/pdk/fetch/{job}), lost on a restart. A test intest_api_phase1.pychecks that no tool,undine callcommand or client method reaches either route. Implemented (phase 2): a second kind,trust:POST /api/confirmations {kind: "trust", project, name}returns the findings with the token, which is bound to the project, the local name, the trust key and the installed tree;POST /api/p/{p}/trust {name, token}uses it up and refuses a key or tree that changed since. Taking trust back ({name, untrust: true}) needs no token. A test intest_agent_phase2.pychecks that no tool,undine callcommand or client method reaches either. - Agent tools (MCP and the built-in agent):
search_packages,get_package,get_dependency_status,propose_dependency(add / update / remove in the agent's changeset). Package and index text (descriptions, READMEs, cell names, intent notes, index lines) is data for agents: the tools return it underagent.py's DATA rule, never as instructions.agent.py's system prompt anddocs/guide/changesets.mdchange from "cannot add dependencies" to "can propose them"; the skill gets a short reference on dependencies (today it does not mention them).
Migration¶
- Git dependencies keep working unchanged.
pkg migrate PROJECT NAMEturns one into an index dependency when an index version's files (those its manifest includes) equal the installed ones, compared file by file. The commit a git dependency pins usually predates the manifest (and a commit without one cannot be published), and today'sshacovers the whole subdir while the indextreecovers only the included files plus the manifest, so the two hashes are not compared. Migrate reinstalls with the manifest's file set and reports the change: the manifest that comes and the files outsideincludethat go. Lock format 1 is read; format 2 is written bypkg migrate(see Lockfile). Implemented (phase 3, deps.migrate): the version compared is the one installed (its manifest's), looked up in the index's local copy, which is fetched again once when it does not list it; the requirement written is=VERSION(pkg update --version ^VERSIONloosens it), and the dependency is installed again throughlib update's path, so the trust of the entry goes as for any update. A dependency with no manifest, a version the index does not list or any included file that differs is refused with nothing changed. A format 1 lock is converted first (pdkops.upgrade, aslib lock --format 2). - PDK ids: a 12-character id (
sky130A@8afc8346a57f) is expanded to the full commit from the installed PDKs (a unique prefix), else fromciel ls-remotewhen the person allows the network. A PDK is verified against the tree hashes its profile package lists for that commit. A commit the profile does not list is locked by the hashes found and shown as unverified instatusand the badge (trust on first use, said as such) until a profile lists it. A PDK with no version (a plain<root>/<name>/directory) is locked by tree hash only, with a warning.pdk_historystays. Implemented (phase 1): all of it, with no profile package yet, so a pinned PDK that matches is unverified, a note and not an issue (decided). A prefix that matches several commits is refused, listing them. - Projects without a lock (made before locks) get one on the first dependency change, as today, but now
statussays it was created. Implemented (phase 1) as the result of the operation that created it, sincestatuscannot tell afterwards: every CLI command that madeundine.lockprintscreated undine.lock (format N)(new,set-pdk, thelibcommands,pdk verify --pin), andrelock,pdkops.upgradeandset-pdkreturncreated. - Profiles: phase 6 moves each family's
Profiletoprofile.tomland checks that the loaded data equals today's Python object for every family before the Python is removed.
Security¶
-
Trust. A package is data, but simulating it runs what its cells tell the simulator. A schematic's code blocks go to ngspice, where
.controlcan runshelland.includecan read any file on the server; a package'srtl/goes through Verilator, where$c(...)and`systemc_*blocks are C++. So, from phase 2, when a netlist includes a cell from a dependency, a.controlblock, ashellcommand, or an.include/.lib/ OSDI path that is absolute or leaves the package (..) in that cell is refused with the cell named; and Verilator refuses package RTL with$cor`systemc_*. A person can mark a package version trusted in the project ("trusted": ["analog_ip@1.2.0"]inproject.json); an agent can propose that but the review shows it, and an update drops it. The review of a dependency change lists the package's code blocks and RTL files, andpkg checkand the index CI flag them. The PDK's own model files and PCell code are trusted as the PDK, as today. Implemented (phase 2):store/trust.pyfinds the content, and every dependency installed under.undine/depsthat the project does not trust is checked when a job is about to run. A plain git dependency is checked too: that is the first of the phase 2 changes for plain git dependencies, so a simulation whose dependency cell has.controlor an absolute.includenow needslib trust. What is refused, exactly:- ngspice (
start_sim, before a run is reused): every.controlblock (with or withoutshell, also inside a.subckt; matched by prefix as ngspice does, so.controlxand.control;too), every*#line, and every file an.include/.libcard or an XSPICEfile=,input_file=orstate_file=parameter reads that, after the netlister's variable substitution, still holds$or~, is absolute outsidepdk.root, or is relative with a..part.+continuations are joined first; a card the scanner cannot read is refused as unparsed. The only absolute root allowed is the PDK's own (pdk.root), not the dependency's install directory: an.includeinto it would read a file nothing scanned. A line is charged to both the cell body it sits in and the symbol it was formed from (schematic.Source), so a trusted symbol's@valueset in an untrusted cell, and an untrusted symbol's template default placed in a project cell, are both caught; the generated corner.libline passes. A.subcktline is a line too (its pins are the placed symbol's), and a pin or port name that holds a space or a line break is refused when the netlist is formed. - LVS (KLayout and netgen): the same paths rule only, since they run no control code, checked when the run
launches (
Run.launch, through a gate), so sign-off status and changeset reuse still show results. - Verilator (
rtl.py, lint and simulation of a dependency library):$c(and$c8...$c64, not$clog2and the other system functions) and$systemwherever they appear (a macro argument becomes the call once expanded),`systemc_*,import/export "DPI-C",`definebodies holding these,import,exportorDPI, or pasting tokens (``), and a path that is absolute, holds~or a..part, or is not a literal, in`include,$fopen,$dumpfile(a simulation is built with--trace),$dumpports,$readmem*and$writemem*. Comments are skipped. Lint, which compiles nothing, checks`includepaths only. - LibreLane (
librelane.submit, therun_librelanetool and the run route): a flow whose top library is a dependency is checked with Verilator's full rule (its Verilator lint and Yosys read the RTL,$readmem*included), the schematic's gate-level export too, before anything is copied or started. - Xschem's netlister backend (the netlist route with
backend=xschem):tcleval(in any.schor.symof an untrusted install (read with every backslash dropped, as Xschem unescapes a value before it looks:tcl\eval(counts), andexpr(whose argument holds[or$, whatever Undine's own pre-pass made of the cell. Only Tcl: that netlist is for display, and Xschem evaluates nothing else when it netlists.
A refusal names the cell, the dependency and each finding with its line, and says that a person can trust the version (
undine lib trust PROJECT NAME, or Trust... in Undine) and an agent cannot. The key is<package>@<version>, or<local name>@<commit12>for a dependency without a manifest;update,removeandlib lockdrop it whenever the entry's commit or tree changes,syncdrops the keys of what it removes, and anadd(orupdate,lib lock) that resolves a package at content the lock did not have drops any key of that name left from earlier content, since any repository can claim a name and version. A changeset's worktree reads trust from the real project only (its ownproject.jsonis a copy from when it opened), so trust a person gives, or takes back, after an agent opened its changeset reaches it. Not covered: a package's key names its version label, not its tree, so anundine.lockedited by hand (or pulled) to another commit of the same version keeps the trust throughsync, which never resolves; binding the key to the tree would close that and is left open. Xschem or ngspice run outside Undine is not protected, nor is aproject.jsontrustedlist that a process with a shell edits: as for the confirmation token, the rule stops agents working through Undine's tools, not a hostile process running as the person. - ngspice (
-
No other code from a package, an index or a profile runs: profiles have named formulas only. PCell code stays the PDK's, run in the PCell worker as today.
- Files: only regular files from git's objects (no symlinks, no submodules), no absolute paths or
.., size and file-count limits, LFS pointers refused. No checkout and no hooks (bare repository). Plain git dependencies with a format 1 lock keep today'sgit archiveand itsdatatar filter until their entry next changes. Implemented (phase 1): installing from complete store records and objects reads no repository and runs no git (each object is re-hashed before it is linked); the kept repositories and the archive view are made withgit init --template=, so no hook comes from the user'sinit.templateDir. - git:
protocol.ext.allow=neveras today.file://and absolute paths are allowed from the CLI; from HTTP only under roots given with--dep-root(like--import-root), so a browser cannot read any repository on the server's disk. Implemented (phase 0):serve --dep-root DIR, repeatable, with no default (unlike--import-root, whose default is home): without it HTTP takes no local repository. The check applies to a URL from the request and to one aproject.jsonnames (sync,update), before the first fetch. It takes the real path (symlinks and..resolved), requires an existing directory, and requires the git directory and its common directory that git would read there to be under a root too; git then fetches from that git directory. Afile://URL with a host other thanlocalhost, a query or a fragment is refused over HTTP. Not covered: a repository under a root whoseobjects/info/alternatesnames objects outside it (it takes write access to that.git). Also in phase 0, for the CLI and HTTP alike: arevorsubdirpart starting with-, a locked commit that is not a full hex id, and a dependency name or lock key that is not a library name are refused before git runs or anything is removed; git gets--end-of-optionsand--before them. Implemented (phase 1): with kept repositories git can read a commit with no fetch, so these checks (_check_spec,_check_commit,_own_dir, in that order) and, over HTTP, the--dep-rootcheck now run before every read of a kept repository, not only before a fetch. A request cannot read a repository kept from a URL outside the roots, even one alibcommand fetched; a sync that installs from the store reads none and is not checked. - Remote hosts. A
project.jsonfrom an untrusted clone, or an HTTP request, can name any URL. From HTTP a fetch goes only to the host of a configured index ($UNDINE_HOME/config.toml), a host given with--dep-host(repeatable), or a URL the person confirmed in the fetch dialog, which shows it in full. Each fetch has a size limit (UNDINE_FETCH_MAX, default 1 GB) checked while it runs, and the all-refs fallback is not used from HTTP without that confirmation. Implemented (phase 2): every git URL, rev and subdir a manifest names goes through the same checks as one fromproject.json(_check_spec, and over HTTP the--dep-rootrule of_source) before any git call, and a local URL (file://, an absolute path,~) in a manifest is refused unless the requirer's own URL is local, so a remote package cannot read a private repository on the machine into the project. A remote host is not restricted yet: until phase 3's--dep-host, an add, update or lock over HTTP fetches from any host a manifest it reaches names, as it does from any host the request names. Implemented (phase 3):serve --dep-host HOST(repeatable). Over HTTP,deps._sourcelets a remote URL through only when its host (gitsrc.remote_host: the host of anhttps,http,sshorgitURL or ofuser@host:path, lower case, without user or port) is one ofallowed_hosts. Those are the hosts of the indexes$UNDINE_HOME/config.tomlconfigures, with the default index (whose host goes when a mirror replaces it), then--dep-host. A project's ownindexesadds none, since a clone brings it. The check covers the request's URL and every one that project.json, a manifest, an index (a project's own, or one an index line names) or an index line names, before git runs. The URL then goes to git as agitsrc.HttpRemote, andgitsrc.fetchdoes not use the all-refs fallback for it: it says to run the operation once from the command line, which keeps the commit. A URL the person confirmed in a dialog is phase 4, with the confirmation token for dependency fetches. The size limit applies to every fetch, from the CLI too (gitsrc._fetch_git, also for an index's copy): git keeps what it receives as one pack (fetch.unpackLimit=1), whose size (index-pack'stmp_pack_*, then the new packs) is read every 0.1 s while git runs and again when it ends. PastUNDINE_FETCH_MAXgit and its index-pack are stopped (their process group) and the kept repository is removed under its exclusive lock, so no part of the fetch stays (FetchTooBig). - Index lines are checked (schema, name rules, URL schemes, hash format) before use; a tree is installed only after it matches the lock or the index.
- Fetches a person confirms. Implemented (phase 1), for PDKs only: a PDK fetch over HTTP needs the
single-use confirmation token, bound to the project and the exact plan (family, variant, full commit, libraries,
ciel source, root), which only the browser's dialog asks for after the person's click; from HTTP only ciel's default
source is fetched (Confirmation token). The command line asks before it runs ciel.
Dependency fetches over HTTP still need no token until phase 4 (the
--dep-rootrule above applies to them). - Signing (later). For now only git hashes pin content: the index line's commit and the tree hash (SHA-256 over
the blobs, so it does not rest on git's SHA-1), checked on every fetch, on
syncand onsync --from-vendor. A changed file, a moved tag or a force-push is detected; who may publish is whoever may merge the index's PRs, and someone who can push to the index can still publish a line that points to other content. Signed index commits or signed package tags come later. - Locks. No server-wide lock during a fetch: fetches run as jobs that write only to the store (objects are written
to a temporary name and renamed, so two processes fetching the same file is harmless). The project's locks are
flocks in
<project>/.undine/, taken by the CLI and by the server, briefly:project.lockaround every write ofproject.jsonandundine.lock(also by a changeset's adopt and revert), anddeps.lockaround one dependency operation at a time, held while it fetches. Phase 0 adds them. Implemented (phase 0): both locks, under these names (an earlier draft had a single.undine/lock). Fetches still run inside the request (withoutsrv.lock), into a staging directory in.undine/deps/, not as jobs into a store. Implemented (phase 1): dependency fetches go into the store, still inside the request (jobs for them are phase 4); a PDK fetch over HTTP is a job. The order isdeps.lock, then the store's.lockheld shared for the whole operation (fetch, tree record, install, lock write and the project's entry inprojects/), then one flock per kept repository (git/<h>.git.lock; readers take it shared).store gctakes the store lock exclusive, so it never runs between a tree being recorded and the lock that names it being written. Every object, record, stamp cache and project record goes totmp/first, then is renamed in. Hashing a PDK and running ciel hold no project lock and no server lock. Oneciel fetchof a PDK commit into a root runs at a time, across processes (the CLI and every server):pdkops.fetchholds an flock on<root>/ciel/<family>/.undine-fetch-<commit>.lockaround the fetch and the hash that follows; the in-memory job lock only shows a second browser fetch as queued. A lock file git leaves in a kept repository when it is killed (shallow.lock,packed-refs.lock,refs/**/*.lock) is removed before the next fetch into it, under that repository's exclusive flock, which every writer of it holds. A project's.undineand.undine/depsmust be real directories: a symbolic link there is refused before any dependency operation lists, locks, stages or removes anything.
Phases¶
| Phase | Delivers | Tests | Status |
|---|---|---|---|
| 0. Fixes | format checked on read (an unknown format refused); set-pdk always writes the lock; sync removes dependencies gone from project.json, status reports leftovers; the project flock for the CLI and the server; HTTP fetches outside srv.lock; --dep-root; the netlister refuses two cells of one name with other content in one netlist; osdi.sh fixed (it still writes into the PDK tree; phase 1 moves it); docs for the missing endpoints |
lock tests in test_deps.py; a lock with format: 2 is refused by this code; the CLI and a server sync one project at once and the lock and project.json stay whole; a slow fake remote does not block another request; file:// refused over HTTP; two libraries with an inv of other content in one testbench are refused, naming both |
Done, see Phase 0 status |
| 1. Store and lock 2 | the store, per-file dedup, frozen and offline sync; lock format 2 for git dependencies, written only by explicit commands; tree hash from git's blobs; PDK full commit + tree hash with the exclusion list, pdk verify, pdk fetch through ciel with confirmation (root chosen as above), PDK rescan; the PCell worker with PYTHONDONTWRITEBYTECODE=1; OSDI built into the store: profile osdi entries become model names (psp103), found first in store/build/<commit>/<variant>/osdi/, then at today's path in the PDK tree (installs that have them), and sim.py and doctor take the paths from one function; tool version numbers and origins |
two versions share unchanged objects; offline sync from the store; sync refuses a lock that no longer satisfies project.json; one commit hashes the same with core.autocrlf=true and an export-subst file; lock 1 → 2 only on an explicit command; prefix → full commit; a sky130 tree with __pycache__ and an IHP tree with osdi/ verify; a changed PDK file is reported; ciel replaced by a fake executable; pdk fetch refused when only ~/.volare is scanned; an IHP simulation loads OSDI from the store; each tool's banner above gives its number |
Done, see Phase 1 status |
| 2. Manifest and resolution | undine-package.toml; transitive dependencies from manifests (still git URLs); project-wide local names, a package installed under each; cell-name disjointness; PDK compatibility; the trust rule for code blocks and RTL |
diamond installs once; conflicting requirements name both chains; two packages with a comparator are refused naming both; a local alias resolves, also with the Xschem netlister backend; two libraries giving one local name different packages are refused; an incompatible PDK is refused; a dependency cell with .control or an absolute .include is refused until trusted |
Done, see Phase 2 status |
| 3. Index | index format, several indexes, the default index created (kadomoto-lab/undine-index), pkg search / info / add / update / lock / outdated / tree / why, version ranges, yank, remote-host policy and fetch size limit, library browser Packages section; pkg licenses, pkg vendor (also run by the tapeout export) and pkg sync --from-vendor |
a file:// index in a temporary directory; yanked skipped but kept when locked; private index named in project.json; a dependency without index comes from the declaring package's index; one name from two indexes refused; an unlisted host refused over HTTP without confirmation; a fetch over the size limit stopped; a vendored bundle, with the package repository deleted and the store emptied, installs the project again offline through sync --from-vendor with the locked trees; a vendored file changed by hand is refused; the tapeout bundle has vendor/ and report.json lists it |
Done, see Phase 3 status |
| 4. Changesets and agents | dependency operations in changesets, review of a dependency change (code blocks and RTL listed), the confirmation token, the notice after a fetch, agent tools with the DATA rule, the merge rule for project.json and undine.lock in the git merge driver (merge.md updated), evidence with package and PDK information outside key and model files and profile fields inside it |
add in a changeset, adopt, restore; an agent's add records needs fetch, never fetches, and continues after the person's fetch; no tool returns a token; two changesets adding different packages adopt in either order; conflicting requirements are refused at adopt; an update that changes one cell outdates only results that use that cell; a README change outdates nothing | In progress, see Phase 4 status |
| 5. Publishing | pkg init / check / pack / publish / yank, [checks], the check report and its link in the index line and PR, semver report, spec results in the report, [renames], cell-level update report, refreshed layout copies, index CI workflow (no DRC / LVS) |
publish refuses without a passing report for this tree, and the PR body links the report with its SHA-256; the index CI refuses a report whose SHA-256, commit or tree differs from the line; a renamed cell is reported as renamed and references rewritten; a removed pin or a moved layout pin needs a major version; a waived rule passes check and is shown; publish refuses a tree the repository does not serve; tag <name>-v<version> in a repository of two packages; after an update the copied cell is refreshed in the changeset and its LVS is outdated, and an edited copy is listed, not replaced |
In progress, see Phase 5 status |
| 6. PDK profiles | profile.toml per family with schema and undine, formula registry, profile packages in the index with expected PDK tree hashes, compatibility tags, pcell.SUPPORTED / pixi.toml / setup.sh read from them |
data profile equals today's Profile for sky130, GF180, IHP; an unknown field is refused; a changed deck option outdates DRC results and a changed profile field no job reads (label, limits) does not |
In progress, see Phase 6 status |
Phase 0 status¶
Where each item of phase 0 is in the code. The items done in the first part of phase 0 were checked against the code right after that part, the others against
the code when phase 0 was finished. Tests are in server/tests/test_deps.py unless named otherwise.
| Item | Status | Where |
|---|---|---|
format checked on read |
Done | read_lock (store/deps.py:90-115) refuses a larger integer format ("made by a newer Undine ... update Undine"), any other value but 1 ("which no Undine writes"), JSON that does not parse and JSON that is not an object, each with what to do and "nothing was changed"; a lock without format is read as 1. Every reader refuses it: lib add / update / remove / sync / lock / status, set-pdk, the /deps routes and POST /pdk (HTTP 400, the CLI one line and exit 1); add and update read it before fetching (store/deps.py:402-472), remove and set_pdk before project.json changes. Opening the project (GET /api/p/{p}) does not read it. Tests: test_a_lock_of_a_newer_format_is_refused_by_every_operation_and_left_byte_for_byte, test_a_lock_of_a_newer_format_is_refused_over_http_and_by_the_cli, test_a_lock_without_format_is_read_as_format_1_and_a_broken_one_is_refused |
set-pdk always writes the lock |
Done | Project.set_pdk (store/project.py:164-187) writes pdk into undine.lock whenever the lock is missing or pins another PDK, also when project.json already names the new one; a project without a lock gets a new format 1 lock with the tool versions now, as lib lock would. The tools answer before meta_lock is taken. cli.md, api.md and pdks.md say so. Tests: test_set_pdk_writes_the_lock_whether_or_not_the_project_has_one, test_set_pdk_over_http_and_from_the_cli_creates_a_missing_lock |
sync removes dependencies gone from project.json; status reports leftovers |
Done | leftovers (store/deps.py:351-369): a directory in .undine/deps/ or a lock entry that project.json does not name, with its locked and installed commits. sync (store/deps.py:488-550) removes each with its lock entry and reports it as removed; one gone from project.json while sync fetches is left to the next sync, and a sync whose fetch fails removes nothing. status (store/deps.py:597-611) lists each as state leftover with an issue naming lib sync (so ok is false), and a lock key that is not a library name as an issue that sync leaves alone. .undine/deps/ stays one entry of the symbol path (Project.symbol_paths, store/project.py:237, which says why), so a leftover's symbols resolve until sync; the netlist cache key covers every directory in .undine/deps/ (server/api.py:353-357). The UI shows the state and offers sync first. Tests: test_status_lists_a_dependency_gone_from_project_json_and_sync_removes_its_files_lock_entry_and_symbols, test_a_sync_that_fails_removes_no_leftover_either, test_the_cli_and_http_report_a_leftover_and_sync_removes_it, test_status_reports_a_leftovers_installed_commit_from_its_files_not_from_the_lock, test_the_netlist_hash_changes_when_sync_removes_a_leftover_a_testbench_places, test_sync_never_removes_a_directory_a_lock_key_names_outside_the_deps_folder, and from phase 0 test_what_changed_while_fetching_is_not_written_back |
| The project flock for the CLI and the server | Done | meta_lock, an flock on .undine/project.lock (store/deps.py:110-115), taken by _committing (store/deps.py:238-244), relock, Project.editing (store/project.py:184-190, also set_pdk and create_library) and a changeset's adopt or revert (store/changeset.py, _merge); .undine/deps.lock (store/deps.py:118-121) for one dependency operation at a time. Tests: test_project_files_are_written_one_writer_at_a_time, test_meta_lock_holds_across_processes_and_a_rename, test_an_adopt_waits_for_a_writer_of_project_json, test_an_operation_that_waited_works_from_what_the_one_before_wrote |
HTTP fetches outside srv.lock |
Done | add, update, sync fetch under .undine/deps.lock only and take the caller's guard (srv.lock) to install and write (store/deps.py:238-244, server/api.py:1654-1694). Tests: test_slow_fetch_does_not_stop_the_server, test_sync_is_all_or_nothing |
--dep-root |
Done | serve --dep-root DIR (__main__.py:267-270; repeatable, no default) into Server.dep_roots (server/api.py:141-143), passed as local_roots to add, update and sync. _source (store/deps.py:206-263, with _local_path, _under_roots, _git_dirs) checks every URL to fetch, from the request or from project.json, before the first fetch: a local one must be an existing directory whose real path, git directory and common directory are under a root, and git fetches from that git directory; a file:// URL with a host, query or fragment is refused. The CLI passes no roots and takes any path. Also for the CLI: _check_spec and _check_commit (store/deps.py:182-203) refuse a rev or subdir part starting with - and a locked commit that is not a full hex id, _own_dir (store/deps.py:384-391) a name that is not a library name, and git gets --end-of-options and -- (_fetch, store/deps.py:266-310). Not covered: objects/info/alternates in a repository under a root. Tests: test_a_local_repository_is_refused_over_http_without_dep_root_and_the_cli_still_takes_it, test_a_local_repository_under_a_dep_root_is_added_updated_and_synced_over_http, test_a_path_that_escapes_the_dep_root_by_dotdot_or_a_symlink_is_refused_over_http, test_a_path_whose_git_directory_is_outside_the_dep_root_is_refused_over_http, test_sync_over_http_fetches_a_local_path_project_json_names_only_under_a_dep_root, test_serve_dep_root_is_repeatable_and_off_by_default, test_a_rev_or_a_lock_commit_that_git_would_read_as_an_option_is_refused_before_git_runs, test_a_subdir_that_looks_like_a_git_option_is_refused_before_anything_is_fetched |
| The netlister refuses two cells of one name with other content | Done | spice keeps the schematic behind each subcircuit name (homes, schematics/schematic.py:630-633 and 654, the top cell included); same_name_check (schematics/schematic.py:563-584) netlists a second cell of that name and raises SameNameError (schematics/schematic.py:390), naming both, unless it gives the same lines, which are written once. So every mode refuses it, and with it simulation, LVS, specs and sweeps. The vlsirtools backend runs spice first (schematics/schematic.py:789); the Xschem backend runs it with XschemOrder (schematics/schematic.py:796-806, Xschem's own symbol order) before looking for Xschem and stops only on that error (schematics/schematic.py:815-823). Some PDK launcher pages are now refused (sky130 sky130_tests/top, IHP start_page). Tests in test_netlist.py: test_two_cells_of_one_name_with_other_content_are_refused_naming_both, test_the_xschem_backend_refuses_only_what_xschem_itself_would_netlist_as_one_cell, test_every_pdk_schematic_netlists_the_same_from_its_netlist_view; the stand-in clash as before in test_substitute.py |
osdi.sh fixed |
Done, and more | It imports undine.pdks.pdk and compiles a copy of libs.tech/verilog-a into Pdk.osdi_dir (pdks/pdk.py:43-58), so the PDK stays as installed: part of phase 1, without the store. Test in test_pdk.py: test_a_pdk_with_verilog_a_models_needs_them_compiled |
| Docs for the missing endpoints | Done | GET /api/pdks, POST /libs, POST /pdk and the /deps calls in docs/reference/api.md |
Phase 0 also made pick_pdk prefer, for a bare PDK name, a version whose PCells run and then pcell.SUPPORTED
(store/project.py:115-121; test_a_bare_pdk_name_picks_the_supported_version in test_pdk.py), made dependency operations re-read
project.json once they hold the dependency lock, made sync all or nothing, and kept machine paths out of
undine.lock (test_refusals_and_drift).
Two parts of phase 1 landed early, in phase 0: OSDI is compiled outside the PDK tree (into Undine's cache,
~/.cache/undine/osdi/<PDK id>/, not yet a store), and the simulation and doctor read its paths through one function
(Pdk.osdi_files); and the lock records tool versions, not paths (still as banners, with no number or origin).
Phase 1 finished both (Phase 1 status).
Phase 1 status¶
Where each item of phase 1 is in the code, by module and function (server/undine/). Tests are in server/tests/;
ciel is replaced by a fake executable (fake_ciel.py), and every test runs with a temporary UNDINE_HOME and
UNDINE_STORE, so none touches ~/.undine, ~/.ciel or the network.
| Item | Status | Where |
|---|---|---|
| The store | Done | home.py (home(): $UNDINE_HOME or ~/.undine; store_root(): $UNDINE_STORE or home()/store) and store/cas.py: objects named by the SHA-256 of their bytes, mode 0444; tree records and format 1 archive records, each re-hashed against its name when read; install re-hashes every object (one whose only fault is its mode, a chmod through a project's link, is made read-only again and kept), then hard-links it (link_or_copy: a read-only copy on EXDEV, EPERM, EMLINK, ENOTSUP, or with UNDINE_STORE_LINK=copy); store_lock; register_project; verify and gc. Every write goes through tmp/. Tests in test_cas.py: test_tree_hash_golden_and_order_by_utf8_bytes, test_put_twice_keeps_one_read_only_object, test_install_links_read_only_files, test_install_copies_when_asked_or_when_linking_fails, test_install_restores_the_mode_of_a_good_object_changed_through_a_project, test_install_accepts_a_good_object_whose_mode_does_not_stick, test_an_object_edited_in_place_is_caught, test_gc_keeps_live_trees_and_frees_the_rest, test_gc_waits_for_a_shared_lock |
| Tree hash from git's blobs, per-file dedup | Done | store/gitsrc.py: a kept bare repository per URL (git/<sha256 of URL>.git, each commit kept by refs/undine/<commit>, one flock per repository), fetch (depth 1, the all-refs fallback as before, the result checked against the locked commit), blob_tree (ls-tree -r -z -l, then cat-file --batch; regular files only; symlinks, submodules, LFS pointers, paths that leave the directory and paths with a .git part refused; a stale git lock file in the kept repository removed before a fetch, and git killed on a timeout with SIGTERM first; the file count and UNDINE_FETCH_MAX checked before any blob is read). Tests in test_gitsrc.py: test_one_commit_hashes_the_same_with_autocrlf_and_export_subst, test_modes_enter_the_tree, test_two_versions_share_unchanged_objects, test_limits_are_checked_before_any_blob_is_read, test_blob_tree_refuses_what_cannot_be_installed, test_a_kept_commit_needs_no_network_and_survives_gc, test_ext_protocol_is_refused, test_a_git_part_in_a_path_is_refused_as_git_refuses_it, test_a_lock_file_left_by_a_killed_fetch_does_not_stop_the_next_one, test_a_timed_out_git_is_asked_to_stop_before_it_is_killed; in test_store_sync.py: test_update_keeps_the_inode_of_an_unchanged_file |
Format 1 entries keep their sha |
Done | gitsrc.archive_tree: git archive in a throwaway repository that holds only the commit, fetched at depth 1 from the kept one, and none of its refs, so export-subst expands as after phase 0's depth-1 fetch (%h too, however large the kept repository grew); cas.archive_sha is phase 0's tree_sha byte for byte. Tests: test_archive_sha_equals_phase0_in_a_kept_repository_with_refs, test_export_subst_h_is_abbreviated_as_phase_0_did_in_a_large_kept_repository (test_gitsrc.py, against a frozen copy of phase 0's code), test_a_format_1_lock_keeps_format_1_and_phase_0_shas (test_store_sync.py) |
Frozen sync, lib lock re-resolves |
Done | store/deps.py: sync refuses through _stale and _frozen before anything is read, naming each entry and lib lock / lib update; it never writes an entry into the lock. relock (lib lock, POST /deps/lock) fetches and installs only what the lock does not pin as project.json names it, with add's checks, and takes no deps.lock when nothing is stale. Tests in test_store_sync.py: test_sync_refuses_a_lock_that_no_longer_satisfies_project_json_and_lib_lock_resolves_it, test_a_project_with_git_dependencies_and_no_lock_is_refused_and_no_lock_is_made |
Offline sync |
Done | deps.sync(offline=...): from the store's records and objects (no repository read, no git), else from a kept repository that has the commit, else refused naming every entry missing (_not_here); lib sync --offline, UNDINE_OFFLINE=1, POST /deps/sync {"offline": true}. An install that matches its lock entry but is not in the store is imported (_seed); a format 2 install is checked against its tree, without the store's record, with the modes its marker records (modes: the files of mode 100755), else as 100644 with #! files as 100755. Tests in test_store_sync.py: test_offline_sync_installs_from_the_store_with_no_fetch, test_offline_sync_with_an_empty_store_names_what_is_missing_and_changes_nothing, test_an_install_made_before_the_store_is_imported_by_sync_and_then_syncs_offline, test_a_failed_online_fetch_is_named_with_its_commit_and_hash, test_a_tree_with_an_executable_file_is_ok_and_seeded_without_the_stores_record, test_a_script_is_found_by_its_shebang_when_the_marker_has_no_modes, test_a_chmod_through_one_project_does_not_cost_another_its_offline_sync, test_a_linked_deps_directory_is_refused_and_nothing_outside_is_touched |
The store lock and the --dep-root rule on kept repositories |
Done | deps._in_store holds cas.store_lock(shared=True) from the first fetch to register_project; the checks of rule (e) run before every gitsrc call that passes lock data to git, and over HTTP _source runs before any read of a kept repository (a callable that _resolve calls only when git is about to be read). Tests in test_store_sync.py: test_over_http_a_kept_repository_outside_the_dep_roots_is_not_read_but_the_store_installs, test_an_option_looking_locked_commit_is_refused_before_any_git_call, test_gc_waits_for_an_operation_between_its_tree_and_its_lock_write, test_the_store_lists_the_project_and_its_locked_and_installed_trees |
| Lock format 2, only from explicit commands | Done | deps.read_lock reads formats 1 and 2 (READS), lock_pdk, lock_tools, entry_hash; add, update and lib lock write a tree entry into a format 2 lock and a sha entry into a format 1 one. pdks/pdkops.py: upgrade, the one writer of format 2, run by pdk verify --pin and lib lock --format 2 (which leaves a format 2 lock's pin alone). Tests: test_lock_format_2_only_on_an_explicit_command, test_lib_lock_format_2_prints_what_it_resolved_and_leaves_a_format_2_pin_alone (test_cli_phase1.py), test_a_format_2_lock_gets_trees_from_update_and_sync_never_converts_an_entry (test_store_sync.py), test_a_format_2_lock_is_read_and_lib_add_keeps_its_format_and_pdk_object, test_today_s_commands_write_format_1_with_banners_and_a_lock_without_format_is_format_1 (test_lock_format2.py) |
| PDK full commit and tree hash, with the exclusion list | Done | pdks/pin.py: parts (libs.tech, each libs.ref/<lib>, each other top-level libs.*), excluded (__pycache__/, *.pyc, libs.tech/ngspice/osdi/), part_hashes with the stamp cache in store/pdk/ (a file written in the last 2 s is read again next time, as git does for its index), pin_object, verify, expected_trees (the phase 6 seam, returns None, so a match is unverified). A symlink, fifo or device in a PDK is an error naming it. Tests in test_pdk_pin.py: test_a_sky130_tree_with_pycache_still_matches, test_an_ihp_tree_with_compiled_osdi_still_matches, test_a_part_hash_is_the_store_tree_hash_of_its_files_with_the_exec_bit_as_mode, test_a_changed_pdk_file_is_reported_by_part_and_file, test_an_extra_library_verifies_and_a_missing_locked_one_is_reported, test_a_second_hash_reads_no_file_and_a_touched_file_is_read_again, test_hashing_writes_nothing_into_the_pdk |
| Prefix to full commit | Done | pin.resolve_commit: a unique prefix among the installs of the variant, else ciel ls-remote --pdk-family F with the network allowed; several matches refused, listing them. Tests in test_pdk_pin.py: test_a_unique_prefix_is_found_among_installs, test_an_ambiguous_prefix_is_refused_listing_both, test_an_unknown_prefix_offline_is_refused_saying_how_to_allow_the_network, test_with_the_network_the_fake_ciel_ls_remote_gives_the_commit_by_family; test_a_format_1_id_is_pinned_to_the_full_installed_commit (test_pdk_ops.py) |
pdk verify and the PDK in status |
Done | pdkops.verify (a full hash, from the stamp cache where unchanged; --pin through upgrade) and pdkops.pdk_block, which deps.status uses: it reads the stamp cache only, never the PDK, so a stale cache is unchecked; a file that vanishes while it is walked (a ciel rm meanwhile) makes the state error, not a failed request. The ciel rm hint is built from the install found here, never from the lock's family. Unverified, unchecked and not pinned are notes; a requirement that disagrees with the pin, a missing or modified PDK and a missing locked library are issues. Tests: test_verify_without_pin_writes_nothing_and_pin_writes_format_2, test_a_modified_pdk_is_reported_and_pin_re_pins_it_reporting_what_changed, test_pdk_block_issues_and_notes (test_pdk_ops.py); test_a_pinned_pdk_with_a_warm_cache_is_unverified_as_a_note_and_ok, test_project_json_changed_by_hand_is_an_issue_naming_set_pdk, test_no_stamp_cache_is_unchecked_and_reads_no_file_of_the_pdk, test_the_reinstall_hint_names_the_install_found_here_not_the_locks_family, test_a_pdk_file_that_vanishes_while_status_walks_it_is_an_error_state_not_a_crash (test_status_pdk.py) |
pdk fetch through ciel, with confirmation |
Done | pdks/ciel.py (exe: $UNDINE_CIEL, then the ciel beside Undine's Python, then PATH; fetch with the ciel family; ls_remote), pdkops.fetch_plan and fetch, pin.fetch_root; the CLI asks first (--yes), HTTP needs the token and runs a job (pdks/fetchjobs.py); one fetch of a PDK commit at a time across processes (an flock, see Locks). Tests: test_pdk_fetch_is_refused_when_only_a_volare_root_is_scanned, test_pdk_fetch_asks_then_installs_with_ciel_and_reports_verify (test_cli_phase1.py), test_a_pdk_fetch_needs_a_fresh_token_bound_to_what_is_fetched, test_no_agent_tool_or_client_method_reaches_a_confirmation_or_a_pdk_fetch (test_api_phase1.py), test_fetch_with_the_fake_ciel_installs_and_verifies_against_the_lock, test_two_processes_fetching_one_pdk_commit_run_one_after_the_other (test_pdk_ops.py), test_ihp_is_fetched_as_its_family (test_ciel.py) |
| PDK rescan | Done | Server.rescan_pdks, POST /api/pdks/rescan, run after an HTTP fetch: an id is changed when its root or what the scan derived from its files (layer properties, xschem and KLayout directories, source) differs, so a PDK first scanned half extracted is replaced; new and changed ids are set first and removed ones popped after, so a reader never finds the dict empty; readers iterate a copy (pick_pdk); the open layouts and editors of a project whose PDK now resolves differently are dropped. Tests in test_api_phase1.py: test_rescan_finds_a_pdk_added_after_start_and_an_open_project_resolves_it, test_rescan_racing_readers_never_shows_an_empty_dict_or_loses_an_unchanged_id, test_a_pdk_first_scanned_half_extracted_is_replaced_once_its_files_are_there |
set-pdk on a format 2 lock |
Done | Project.set_pdk writes the new PDK's pin (hashed before meta_lock) and keeps the format and the tools; the same PDK keeps its pin, so a modified tree stays reported. Tests: test_set_pdk_on_a_format_2_lock_writes_the_new_pin_and_on_format_1_the_id (test_lock_format2.py), test_set_pdk_on_a_format_2_lock_hashes_without_the_server_lock (test_api_phase1.py) |
The PCell worker with PYTHONDONTWRITEBYTECODE=1 |
Done | pcell._run and pcell_worker._load. Tests in test_pcell.py: test_the_worker_is_run_without_writing_bytecode, test_a_worker_run_leaves_no_pycache_in_the_pcell_package |
| OSDI built into the store | Done | Profile osdi entries are model names (psp103); Pdk.osdi_dir is store/build/<commit>/<variant>/osdi/ (local-<root hash> for a PDK with no version) and Pdk.osdi_files looks there, then in the PDK's libs.tech/ngspice/osdi/, then in the cache phase 0 compiled into; scripts/osdi.sh compiles into the store and moves what the old cache has; doctor notes models still in the old cache. Tests in test_osdi_store.py: test_profile_lists_model_names, test_an_ihp_simulation_loads_osdi_from_the_store, test_osdi_sh_moves_the_old_cache_into_the_store, test_store_gc_and_verify_leave_build_alone; test_osdi_row_hints_at_moving_the_old_cache (test_doctor.py) |
| Tool version numbers and origins | Done | tools.py: VERSION_RULES and parse_version (the table in Lockfile), locate (path and origin), tool_record, tools_record (with undine_version and pixi_lock_sha); deps.status warns of another number or origin and notes another Undine commit or pixi.lock; doctor shows number and origin per tool, Undine's version and the pixi.lock hash. Tests in test_tools.py: test_each_tools_banner_gives_its_number, test_an_unmatched_banner_is_kept_as_text_and_a_silent_tool_as_a_question_mark, test_locate_labels_where_each_tool_came_from, test_undine_version_without_git_and_in_a_checkout; test_status_on_format_2_warns_of_another_tool_and_notes_another_commit (test_lock_format2.py); test_tool_rows_show_number_and_origin (test_doctor.py) |
store verify, store gc |
Done | __main__.store_command: verify re-hashes every object and record and removes what does not match (--dry-run reports only); gc reads the locks and installs of the projects under --root, their changesets' worktrees and the store's list, under the exclusive store lock, and removes nothing when a lock cannot be read. Tests in test_cli_phase1.py: test_store_verify_finds_an_edited_object_and_removes_it, test_store_gc_keeps_what_projects_here_lock_or_install_and_removes_the_rest, test_store_gc_with_an_unreadable_lock_removes_nothing |
| Changeset worktrees hard-link the dependencies | Done | changeset._dep_copy, through cas.link_or_copy. Tests in test_changeset_deps_link.py: test_worktrees_link_the_dependency, test_copies_where_linking_is_refused |
Decided while building phase 1 (with the user):
- Format 2 is written by
pdk verify --pinand bylib lock --format 2, through one function;pkg addandpkg migratecome in phase 3 (Lockfile). - A PDK fetch over HTTP is confirmed by a narrow, PDK-only token now, built so that phase 4 generalises it.
- OSDI models that phase 0 compiled into
~/.cache/undine/osdi/<id>/are still found there untilpixi run osdimoves them into the store;doctorsays so. - An unverified PDK (trust on first use) is a note, not an issue:
statusstays ok and the badge is not red. - Dependencies installed before phase 1 are imported into the store by the first
sync, so they sync offline without a fetch.
Differences from the text above, and additions:
GET /api/p/{p}/pdk/fetch-planis an addition: the fetch dialog shows family, variant, commit, libraries, source and root before the person confirms.POST /api/confirmationsalso returns the plan.lib lock --format 2on a lock that is format 2 already does not pin the PDK again (that would silently accept a modified PDK): it says to usepdk verify --pin.statusnever reads the PDK; after a fetch, a PDK is unchecked untilpdk verifyhashes it more than 2 s after its last write.- The CLI's
pdk fetchcannot rescan a running server; it says to use Rescan PDKs orPOST /api/pdks/rescan. store gcalso clearstmp/entries older than a day, and the store's project list drops projects that are gone.
Left for later phases: the pkg commands (phase 1 has lib sync --offline for the frozen offline sync), manifests,
transitive resolution and the include file set (phase 2: a phase 1 tree covers every file of the subdir), index
clones, pkg migrate, --dep-host and the size monitor during git fetch (phase 3: phase 1 checks the size and file
count from ls-tree before any blob is read), dependency fetches as jobs, the general confirmation token and the merge
rule for format 2 (phase 4), an HTTP route or button for lib lock --format 2 and pdk verify --pin (the command
line only), expected PDK hashes and verified_by (phase 6), pruning git/ from store gc, and keeping PDK fetch jobs
across a restart.
Phase 2 status¶
Where each item of phase 2 is in the code, by module and function (server/undine/, and web/src/ for the UI). Tests
are in server/tests/ (the web's in web/src/); as in phase 1, every test runs with a temporary UNDINE_HOME and
UNDINE_STORE and local git repositories, so none touches ~/.undine, ~/undine-projects or the network.
| Item | Status | Where |
|---|---|---|
undine-package.toml |
Done | store/manifest.py: parse (the first problem refused as <where>: undine-package.toml: <key path>: <what>: <fix>; unknown keys refused naming the known ones, [checks] and [renames] kept as tables for phase 5; tomllib, else tomli on Python 3.10), select and check_files (the [files] include set: *.sch matches top-level files only, rtl/** any depth, the manifest always in; an include set with no cell, or without an exported cell, refused), from_files, from_tree, from_archive, from_dir (store objects re-hashed before they are parsed; a symlinked manifest refused). store/semver.py: Version (strict SemVer 2.0.0, build metadata ignored in precedence) and Req (bare, ^, ~, =, *; comparators and comma lists refused with the phase 3 hint). gitsrc.blob_tree(files="include") reads only the included blobs, and gitsrc.read_file one file at a commit. Tests: test_manifest.py (test_the_design_example_parses, test_git_and_sibling_requirements, test_each_refusal_names_its_key_path_and_what_to_do, test_select_matches_whole_paths_segment_by_segment, test_check_files_wants_a_top_level_cell_and_every_export, test_from_files_refuses_a_modified_store_object, test_parse_falls_back_to_tomli_on_python_3_10, ...), test_semver.py (test_precedence_follows_semver_section_11, test_requirement_meanings, test_ranges_are_refused_with_the_phase_3_hint, ...), test_gitsrc_include.py (test_the_include_tree_is_the_selected_files_plus_the_manifest_and_nothing_else_is_read, test_without_files_or_without_a_manifest_include_gives_the_all_tree, test_a_link_or_submodule_is_refused_only_inside_the_include_set, test_an_oversized_or_irregular_manifest_is_refused_before_any_other_blob_is_read, test_read_file_reads_one_file_without_the_tree, ...) |
| Transitive dependencies from manifests (git URLs) | Done | store/resolve.py: resolve (the walk: one node per package name, each problem collected as a Problem with its code and chains, ResolveError once), same_package (equal trees, or the mixed files modes rule), chain_text, lock_graph, reachable, incomplete. store/locked.py: files_of and manifest_of, a locked entry's files and manifest from the store or a verified install, else NotAvailable ("run lib sync first"). store/deps.py: add, update (only the named requirement's subtree), remove (from the lock graph, no fetch; keeps what another package still places), relock (upgrade= for lib lock --format 2), all through _Graph; NotDirect for a name a package placed, NeedsFormat2 on a format 1 lock. Tests: test_resolve.py (test_conflicting_requirements_name_both_chains, test_a_phase_1_entry_and_an_include_requirement_at_one_commit_agree_and_the_locked_one_stands, test_a_sibling_is_read_at_the_requirer_s_commit_with_a_root_relative_subdir, test_every_problem_of_a_requirement_is_collected_in_one_error, test_a_cycle_is_walked_once, ...), test_locked.py, test_deps_phase2.py (test_update_moves_a_package_and_its_dependencies_and_drops_what_it_no_longer_needs, test_update_of_a_placed_name_is_refused_and_remove_keeps_a_name_another_package_places, test_a_fresh_clone_must_sync_before_add, test_format_1_refuses_dependencies_and_another_name_and_adds_a_plain_package_as_a_sha_entry, test_relock_upgrade_writes_format_2_once_and_a_refusal_leaves_format_1, test_plain_dependencies_keep_phase_1_entries_and_a_phase_1_entry_is_not_converted, test_a_remote_manifest_may_not_name_a_local_repository, test_over_http_dep_root_applies_to_the_urls_a_manifest_names, ...) |
| Project-wide local names, a package installed under each | Done | resolve.resolve gives each node its names (the requirers' keys and the package's own name) and refuses a name claimed for two packages or reserved (casefolded); deps._reserved and deps._shadowed (project and PDK libraries, devices, Xschem's library folders, symbol_paths folders); deps._apply and deps.sync install one tree under every name, hard-linked from the store, each with its marker (name, direct, package, version, via). store/project.py: a transitive name is a read-only dependency Library with package, direct and via, learnt from markers (cached by inode, mtime and size; the lock is never read); create_library refuses an installed name. Tests: test_an_alias_installs_one_package_under_both_names, test_one_local_name_for_two_packages_is_refused_naming_both_requirers, test_two_local_names_differing_only_in_case_are_one_name, test_a_reserved_local_name_is_refused_naming_the_chain (test_resolve.py); test_an_alias_installs_the_package_under_both_names_sharing_inodes, test_one_local_name_for_two_packages_is_refused_naming_both (test_deps_phase2.py); test_a_transitive_install_is_a_read_only_dependency_library_and_the_lock_is_never_read, test_markers_and_manifests_are_read_once_until_they_change, test_create_library_refuses_an_installed_local_name (test_project_phase2.py); test_a_new_local_name_may_not_be_a_folder_of_xschems_library_or_of_symbol_paths, test_a_name_project_json_already_gives_keeps_working_and_status_notes_what_it_shadows (test_reserved_names_phase2.py) |
| Cell-name disjointness | Done | resolve.cell_names (top-level logic views, and layout-only .oas cells) and resolve.check_disjoint, fed by deps._project_cells and deps._pdk_cells; refused only with a package with a manifest on one side; status reports the rest (Project.dependency_cell_names). Tests: test_check_disjoint, test_cell_names_are_top_level_logic_and_layout_only_cells (test_resolve.py); test_cell_names_clash_between_packages_but_not_with_one_package_under_two_names_or_a_layout_copy (test_deps_phase2.py); test_a_project_cell_named_like_a_packages_cell_is_an_issue_a_layout_copy_is_not (test_store_sync_phase2.py) |
| PDK compatibility | Done | pdks/compat.py: project_target (the lock's pin, or a format 1 id as a prefix), check (family through family_of, variants, full commits, built_with; a PDK with no version or no PDK refused with what to do), note (drawn on another commit than used), tag_commits (the phase 6 seam, knows no tag), locked_incompatible and IncompatiblePdk, which Project.set_pdk raises before hashing and again under editing(); pdkops.upgrade and verify --pin report a pin a locked package does not accept as incompatible. Tests: test_pdk_compat.py (test_family_variant_and_commit_accepted_and_each_refusal, test_a_format_1_prefix_matches_a_full_commit, test_compat_only_is_refused_with_the_phase_6_hint, test_a_pdk_without_a_version_and_no_pdk_are_refused, test_ihp_family_accepts_its_variant, ...), test_pdk_check_names_a_transitive_package_and_its_chain (test_resolve.py), test_set_pdk_is_refused_listing_every_locked_package_that_does_not_accept_it, test_plain_git_dependencies_never_block_set_pdk, test_upgrade_reports_locked_packages_the_new_pin_does_not_accept (test_project_phase2.py) |
| The trust rule for code blocks and RTL | Done | store/trust.py: the scanners (spice_findings, rtl_findings, tcl_findings, scan_dir), the record (key_of, trusted_keys, the real project's keys only in a changeset worktree, untrusted_dirs, set_trusted, drop) and the gates (check_netlist, check_tcl, check_rtl, raising UntrustedContent). schematic.spice records each line's Source (cell body and symbol) and owner names it. The gates run in Server.start_sim (rules sim), at Run.launch for KLayout and netgen LVS (Server._paths_gate, rules paths), in the netlist route with backend=xschem (Tcl only), in rtl.lint and rtl.submit_sim, and in librelane.submit (a flow whose top library is a dependency; run_librelane returns the refusal, the run route answers 403 untrusted, and the sign-off suite returns an LVS refusal per check with its code and findings). Tests: test_trust_scan.py (test_control_blocks_and_control_lines_with_their_lines, test_paths_after_substitution, test_an_include_into_a_dependency_install_is_refused, test_rtl_findings, test_tcl_findings_in_a_symbol, test_scan_dir_lists_cells_rtl_and_tcl, ...), test_netlist_sources.py (test_code_blocks_are_attributed_to_their_cell_body_with_the_exact_text, test_a_trusted_symbol_placed_in_a_dependency_cell_is_attributed_to_that_cell, test_owner_names_packages_plain_dependencies_and_directories, ...), test_trust_run.py (below) |
| Frozen sync, leftovers and status over the lock graph | Done | deps.sync installs every entry of the lock graph under each name (_plan, _frozen, _fetch_missing), from the store when it can, rewrites a package's marker when its names or requirers moved, and removes what no project.json dependency still reaches; plain-dependency rows stay {name, action, commit}. deps.status (_Installed) gives one row per local name with package, version, names, direct, via, chain, trusted, flagged, internal_cells and pdk. Tests in test_store_sync_phase2.py: test_a_frozen_sync_installs_every_package_and_name_from_the_store_with_no_fetch, test_offline_with_an_empty_store_names_each_missing_package_once_and_changes_nothing, test_plain_dependency_rows_stay_name_action_commit, test_a_phase_1_entry_whose_manifest_has_files_still_syncs_every_file_from_git, test_removing_a_dependency_from_project_json_removes_what_only_it_reached, test_sync_refuses_an_incomplete_graph_or_a_name_claimed_twice_before_anything_is_read, test_status_rows_carry_the_package_fields_and_one_row_per_local_name, test_status_reports_an_incompatible_package_and_notes_a_built_with_difference, test_a_broken_installed_manifest_is_an_issue_row_not_a_crash, test_a_plain_dependency_with_control_gets_the_trust_note |
[exports] |
Done | Library.internal and the internal mark in Library.cells; Server.symbols and the agents' list_symbols leave internal cells out; they still resolve, netlist and may be placed by name. Tests: test_a_dependency_row_shows_its_package_and_cells_outside_exports_are_internal (test_project_phase2.py), test_symbols_leave_out_internal_cells_and_the_cell_list_marks_them (test_api_phase2.py) |
| CLI | Done | __main__.py: lib add --package, the transitive (+), moved and dropped package lines of lib add / update / remove, via chains and alias rows in lib status, the ~ line for untrusted content, lib trust PROJECT NAME [--untrust] [--yes] (lists the findings, asks; no terminal and no --yes is refused), lib lock --format 2 converting and resolving in one write, set-pdk refused with every package. Tests in test_cli_phase2.py: test_lib_add_prints_the_packages_it_brings_and_what_is_not_trusted, test_lib_status_shows_via_and_alias_rows_and_remove_names_what_stays_placed, test_lib_update_prints_the_packages_it_brings_moves_and_drops, test_lib_trust_lists_asks_and_writes_trusted, test_lib_lock_format_2_converts_and_resolves_in_one_write_or_leaves_format_1, test_set_pdk_to_a_pdk_locked_packages_refuse_lists_each_and_changes_nothing, test_a_pdk_pinned_by_lib_lock_format_2_that_a_package_refuses_is_a_warning, test_lib_sync_shows_a_package_the_locked_pdk_does_not_fit |
| HTTP | Done | server/api.py: the /deps routes return the new fields (resolved, flagged, packages_changed, removed_packages, kept); refusals carry codes, with detail = the CLI's message: 409 resolution_refused (problems), 409 needs_lock_format_2, 400 not_direct (placed_by), 409 pdk_incompatible (packages), 409 not_synced, 403 untrusted (findings); GET /api/p/{p}/trust, the trust confirmation kind and POST /api/p/{p}/trust. Tests in test_api_phase2.py: test_post_deps_reports_the_transitive_packages_and_a_conflict_is_409_with_both_chains, test_update_and_remove_of_a_placed_name_are_400_not_direct_naming_the_placer, test_the_new_refusals_have_their_codes_and_keep_the_message_as_detail, test_trust_needs_a_fresh_trust_token_bound_to_the_key_and_the_installed_tree, test_a_pdk_tokens_messages_are_phase_1s, test_post_pdk_to_a_pdk_a_locked_package_does_not_accept_is_409_with_the_packages; test_http_sim_refusal_is_403_untrusted_with_findings (test_trust_run.py) |
| Agents | Done | agents/agent_tools.py: run_simulation, check_specs, sweep, lint_rtl and run_rtl_sim return a trust refusal as their result, unchanged; no tool, undine call command or client method reaches the trust routes or a confirmation. agents/agent.py's prompt and the skill's verification.md say that trusting is a person's action. Tests in test_agent_phase2.py: test_no_tool_client_method_or_call_reaches_trust_or_a_confirmation, test_the_prompt_says_trusting_a_dependency_is_a_persons_action, test_run_simulation_on_an_untrusted_dependency_returns_the_refusal_unchanged, test_check_specs_and_sweep_pass_the_refusal_on, test_run_rtl_sim_and_lint_rtl_on_an_untrusted_dependency_return_the_refusal |
| Web UI | Done | App.tsx: dependency rows with package, version, via and alias tags, Update / Remove on direct rows only, the lock dialog's package and trust columns, a refused add listing every problem, a refused PDK move listing the packages; TrustDialog.tsx (findings by file, then the token, then trust; untrust); UntrustedNotice.tsx, and Refused in panels/SimPanel.tsx for the simulation, LVS, intent, netlist and RTL views. Tests: App.phase2.test.tsx, TrustDialog.test.tsx, panels/UntrustedPanels.test.tsx |
| Test: diamond installs once | Done | test_a_diamond_reads_the_shared_package_once_and_makes_one_node (test_resolve.py), test_diamond_installs_c_once_hard_linked_and_fetches_it_once (test_deps_phase2.py), test_packages_of_one_repository_and_commit_are_fetched_once (test_store_sync_phase2.py) |
| Test: conflicting requirements name both chains | Done | test_conflicting_requirements_name_both_chains (test_resolve.py), test_conflicting_requirements_name_both_chains_and_nothing_is_written, test_a_transitive_name_becomes_direct_for_the_same_package_and_conflicts_for_another_version (test_deps_phase2.py), test_post_deps_reports_the_transitive_packages_and_a_conflict_is_409_with_both_chains (test_api_phase2.py) |
Test: two packages with a comparator are refused naming both |
Done | test_check_disjoint (test_resolve.py), test_cell_names_clash_between_packages_but_not_with_one_package_under_two_names_or_a_layout_copy (test_deps_phase2.py): comparator: analog_ip 1.2.0 and sar_common 1.3.1, nothing written |
| Test: a local alias resolves, also with the Xschem netlister backend | Done | test_a_local_alias_resolves_with_undines_netlister, test_a_local_alias_resolves_with_the_xschem_netlister (test_alias_netlist.py), test_an_alias_installs_the_package_under_both_names_sharing_inodes (test_deps_phase2.py) |
| Test: two libraries giving one local name different packages are refused | Done | test_one_local_name_for_two_packages_is_refused_naming_both_requirers, test_two_local_names_differing_only_in_case_are_one_name (test_resolve.py), test_one_local_name_for_two_packages_is_refused_naming_both (test_deps_phase2.py) |
| Test: an incompatible PDK is refused | Done | test_an_incompatible_pdk_is_refused_directly_and_through_a_chain (test_deps_phase2.py), test_set_pdk_is_refused_listing_every_locked_package_that_does_not_accept_it (test_project_phase2.py), test_set_pdk_to_a_pdk_locked_packages_refuse_lists_each_and_changes_nothing (test_cli_phase2.py), test_post_pdk_to_a_pdk_a_locked_package_does_not_accept_is_409_with_the_packages (test_api_phase2.py), and the dialog in App.phase2.test.tsx |
Test: a dependency cell with .control or an absolute .include is refused until trusted |
Done | test_trust_run.py: test_a_dependency_cell_with_control_or_an_absolute_include_is_refused_until_trusted, test_a_project_symbols_value_set_in_a_dependency_cell_is_refused, test_a_dependency_symbols_template_default_is_refused_in_its_own_cell_and_in_a_project_cell, test_an_absolute_include_of_a_file_in_the_dependencys_own_install_is_refused, test_star_hash_lines_and_control_in_a_dependency_subcircuit_are_refused, test_includes_under_the_pdk_root_and_the_generated_corner_line_pass, test_a_project_cells_own_control_runs_and_display_netlists_only_warn, test_lvs_is_refused_only_at_launch_and_only_for_paths, test_the_xschem_route_refuses_untrusted_tcl_whatever_the_prepass_makes_of_it, test_rtl_of_an_untrusted_dependency_is_refused_by_simulation_and_lint_until_trusted, test_a_changeset_worktree_sees_trust_given_in_the_project_after_it_opened; test_trust_goes_with_an_update_to_another_commit_and_start_sim_refuses_again (test_deps_phase2.py); the refusal in the panels in panels/UntrustedPanels.test.tsx |
Decided while building phase 2 (proposed in the phase 2 PR, open to review):
- Git requirements before the index. A manifest names a dependency as
{ git, rev, subdir?, package?, version? }(git and rev required, so published data never means a moving HEAD;packagedefaults to the key) or as{ subdir }alone, a sibling in the same repository and commit, its subdir relative to the repository root. The index forms are parsed and refused at resolution, showing the git form. - Semver.
Versionand^,~,=, bare and*; comparators and comma lists wait for phase 3, where an index gives them versions to choose among. - Where packages go in the lock. Direct dependencies stay in
libraries; packages reached only through manifests go topackages, with noindexkey. A package name appears once across both tables. - What a version of a git package is: (package name, tree), the identity taken from the tree's manifest even for
a phase 1 entry, and
git:<local>only for a tree with no manifest. Whenfilesmodes differ (a phase 1 entry over every file and a phase 2 include set), equal (git, commit, subdir) decide, and the locked entry stands as it is. - The file set of a tree is an explicit
files: "include", written only byadd,updateandlib lock, never bysync; a manifest without[files]gives the phase 1 tree id. - Fixed packages are read from the store, else a verified install, else refused with "run lib sync first"
(
store/locked.py; HTTP 409not_synced), byadd,update,lib lockandset-pdkalike: none of them fetches what it does not re-resolve. - An entry locked before phase 2 (format 1, or a phase 1 format 2 entry: no
packagerecorded) is held as that plain entry by a command about another dependency: its manifest gives its identity, but its dependencies, its own name and its[pdk]are neither fetched, checked nor written, and its entry and install stay byte for byte, unless a package the command resolves now needs it (then it getspackage,versionandnames, and nodepsuntil it is resolved). Onlylib update NAMEorlib lock(format 2, or--format 2) resolves it.lib statusnotes one whose package is not installed under its own name, and one whose dependencies are not locked on a format 1 lock; on a format 2 lock unlocked dependencies stay an issue. - One package, one name in
project.json. A second direct name for a package locked with its fields is refused (y: a 1.0.0 is a dependency of this project already, as x), as it would be a second entry for one package. A local name spelled like an existing library in another case is refused too (library IP exists (as ip)), as is one package reached under two spellings: on a case-insensitive file system they are one directory. - The self-name. A package with a manifest is also installed under its own
[package]name, since its own schematics place its cells as<name>/<cell>.sym. - Reserved local names: project and PDK libraries,
devices, every subdirectory of Xschem's library and ofproject.jsonsymbol_paths; compared casefolded; names already inproject.jsonare only noted. - Only
project.jsonmay name a plain git dependency: a manifest's dependency needs a manifest of its own (identity, version and[pdk]come from it). - Format 1. Manifests on a format 1 lock are read and checked;
[dependencies], or a name other than the key, is refused with thelib lock --format 2hint.lib lockkeeps phase 1's order (relock, then upgrade) and, only when relock needs format 2, resolves with an in-memory upgrade and writes once. - No built-in compatibility tags.
compat.tag_commitsis the phase 6 seam; the checks are family, variants, full commits andbuilt_with; a package without[pdk]is noted, not checked. - No PDK, a PDK with no version, a format 1 id: refused with a reason and what to do; a 12-hex id matches as a prefix; nothing is fetched.
- A pin that moves (
pdk verify --pin,lib lock --format 2) to a PDK a locked package does not accept is reported as!warnings and status issues, not refused: the pin records what is installed. - Cell clashes are refused only with a package with a manifest on one side; project libraries contribute logic views only; a package's layout-only cell is excused against its own closure and the PDK; the rest are status notes.
- Trust. The key is
name@version, orlocal@commit12without a manifest, dropped on any commit or tree change. The rules are those under Security; the only absolute root allowed ispdk.root, and a line is attributed to both its cell body and its symbol. - Where each gate is: simulation at
start_sim(before a run is reused), LVS atRun.launch(paths only), the Xschem backend in its route (Tcl only, whatever the pre-pass does), Verilator inrtl.py, LibreLane inlibrelane.submit(Verilator's full rule on the top library). A project that declares no git dependency is not looked into, so a linked.undinewith nothing to install refuses nothing. - Changeset worktrees read trust from the real project only: their own
project.jsonis a copy from when they opened, so a key taken back in the project is taken back in them too. - A person trusts through
lib trust, the HTTP route with a single-usetrusttoken, and the dialog; no agent, client orundine callpath reaches any of them. - A project cell named like a dependency cell is not refused when it is made:
statusreports it (an issue against a package, a note against a plain dependency), and phase 0's netlist-time check stays the backstop. - Local URLs in a fetched manifest are refused unless the requirer is local; every transitive URL goes through
_check_spec, and over HTTP through_source. [exports]: internal cells are hidden from the picker andlist_symbolsand marked internal; they still resolve and netlist, and placing one is allowed (flagging it ispkg check's, phase 5).addandupdatere-resolve only the named requirement's subtree; every other locked package is held fixed, since git revs likemainmove.- Python stays >= 3.10, with the
tomlifallback. - Phases 0 to 2 are released together. Should phase 1 ship first, its
read_lockmust refuse a format 2 lock withpackagesor with alibrariesentry carryingfiles,package,version,namesordeps, or a phase 1 Undine would take self-name and alias directories for leftovers and delete them. - Commands.
libis extended and gains onlylib trust; chains appear inlib status. Thepkggroup starts in phase 3. - Unknown keys in a manifest are refused, naming the key and the known keys, so a typo such as
revision =does not silently mean HEAD.
Differences from the text above, and additions:
[dependencies]takes the git form{ git, rev, subdir?, package?, version? }and the sibling form{ subdir }, which Packages does not show; its index forms are refused until phase 3.- Lock entries have a
fileskey:"include"when the tree covers the manifest's include set, absent when it covers every file of the subdir (Lockfile's note). - Direct dependencies with a manifest stay in
libraries, withpackage,version,namesanddeps;packagesholds only what manifests reach. The example in Lockfile puts every package inpackages. - An entry over every file (phase 1) and a requirement for the include set of the same (git, commit, subdir) are one
package; the locked entry stands. Identity comes from the tree's manifest,
git:<local>only without one. - No compatibility tag is built in: a package that lists only
compattags is refused with a hint to list its commits undercommits, until profile packages define tags (phase 6). - A package is also installed under its own name (the self-name), not only under the names requirers give it.
- Local names may not be
devicesor a subdirectory of Xschem's library or of asymbol_pathsentry, besides project and PDK library names (Resolution rule 5 names only these). - Trust: the key of a dependency without a manifest is
<local>@<commit12>; HTTP has atrustconfirmation kind; a changeset worktree reads trust from the real project only. Security says an agent can propose trust; in phase 2 it cannot (proposals are phase 4). - The Xschem netlister backend is gated on Tcl only (
tcleval(, andexpr(with[or$), since its netlist is for display and Xschem evaluates nothing else. - LVS and netgen are gated when the run launches, with the paths rule only (no
.controlrule: they run none). - The trust rule refuses more than Security lists:
*#lines, XSPICE file parameters, paths that keep$or~, absolute paths into the dependency's own install, and in RTL$system, DPI, file tasks,`includepaths and`definebodies. - Semver ranges with comparators (
>=,<, comma lists) are deferred to phase 3, refused with a hint. tomllibhas a fallback: on Python 3.10 Undine reads manifests withtomli(tomli>=1.1; python_version < '3.11'inserver/pyproject.toml), so the documented non-pixi install keeps working.- Three behaviour changes for projects with plain git dependencies, each with a status note and a troubleshooting
entry in Troubleshooting: the trust gate applies to every dependency;
statusreports a project-library logic cell named like a dependency's; a new local name may not be a subdirectory of Xschem's library or of asymbol_pathsentry. Nothing else that worked in phase 1 is refused, and a format 1 lock or a phase 1 format 2 lock keeps working unchanged.
Left for later phases: the package index, index clones, version ranges with comparators, backtracking, yank, index
in lock entries, the pkg commands (search, info, add, update, lock, outdated, tree, why),
pkg migrate, --dep-host and the size monitor during a fetch (phase 3); dependency operations in changesets and
their review (which will reuse trust.scan_dir), agents proposing trust or dependencies, the general confirmation
token, dependency fetches as jobs and the merge rule for undine.lock (phase 4); pkg init / check / pack /
publish, the meaning of [checks] and [renames], SPDX parsing, refusing git dependencies when publishing,
checking that a package places only the local names it declares, and refreshed layout copies (phase 5); compatibility
tags as profile data, the commits whose model files differ in a set-pdk refusal, and expected PDK tree hashes
(phase 6). Not planned in a phase yet: several major versions of one package and per-library local-name tables
(Decisions 3 and 5), a marker for PDK-neutral packages, rewriting a package's relative .include paths
to its install directory, RTL mixed across libraries, refusing a project cell named like a dependency cell when it is
made, renamed, copied or imported, protecting Xschem opened outside Undine and a project.json trusted list
edited by a process with a shell, binding the trust key to the tree, and an HTTP route or button for
lib lock --format 2 and pdk verify --pin (the command line only, as in phase 1).
Phase 3 status¶
Done (store/index.py, the index parts of store/deps.py and store/resolve.py, undine/pkgcli.py; tests in
test_pkg_index.py):
- The index format as Index gives it: the path rule, one JSON line per version checked field by field
(what reaches git and the store is refused unless it is exactly what the format allows),
config.jsonwithformat: 1(another format is refused), each version once per file. - Local copies. Each index is a bare repository in
store/index/<sha256 of URL>.git, its refrefs/undine/indexthe index as last fetched; files are read withgit cat-file, never checked out. Fetched on first use, then only bypkg index update,pkg update, or once by a resolution that finds no version it needs. Over HTTP a local index is taken only below--dep-root, as a local git dependency is. - Several indexes.
undine(the default index's URL, which$UNDINE_HOME/config.tomlmay replace), more inconfig.toml[indexes]and inproject.jsonindexes(which win: a clone finds them). A requirement'sindexis a nickname or a URL; without it, cargo's rule: the index of the package that declares it, the default index for the project's own requirements and for a package from git. One package name from two indexes is refused. - Version ranges (semver.py): comparators and comma lists besides
^,~,=, bare and*. - The choice (index.solve): depth-first with backtracking over the index lines, newest first, with the locked
version tried first (
prefer) or kept (held, an operation about another dependency), yanked versions skipped unless locked, the PDK checked per version from the line'spdk. A failure names the requirements and their chains. The deps layer pins each chosen version as a git requirement at the line's commit and walks it as any other (Resolution 3 to 6 unchanged); an index requirement met first in a manifest during the walk is added to the choice and the walk made again. The manifest at that commit must give the line's name, version and dependencies, and its files the line's tree; a package from an index may depend only on packages from an index. - The lock: an entry from an index records
index(URL) andversion, norev;project.jsonnames{package?, version, index?}. A stale entry (another index or package, or a version the requirement no longer accepts) is resolved bylib lock;syncinstalls from the lock without reading the index. Format 2 only. - Yank: skipped by new resolutions, kept where locked;
statuswarns from the local copy. - Commands:
pkg search / info / add / update / outdated / tree / why / lock / remove / migrate / index list / index update / index add(index addwrites~/.undine/config.toml, or project.json with--project, the rest of the file kept as written).
The default index repository, kadomoto-lab/undine-index, exists
(created on 2026-11-20, config.json only, no package yet).
The remote-host policy (serve --dep-host) and the fetch size limit checked while git runs are built (see
Security, Remote hosts).
pkg licenses, pkg vendor (also run by the tapeout export) and pkg sync --from-vendor are built (see
Fetch and store, Vendoring, and Licences).
The library browser's Packages section and its HTTP endpoints (GET /api/packages, GET /api/packages/{name}, GET /api/indexes,
POST /api/indexes/{name}/refresh, and POST /deps taking a package from an index) are built (see UI, CLI, HTTP, agent tools).
Phase 3 is done; symbol previews of a package's exported cells are left for later.
Phase 4 status¶
Built: the merge rule for project.json and undine.lock (vcs/depmerge.py) in the git merge driver and the
changeset adopt, and the adopt's check of the merged pair (see Changesets and evidence).
Tests: test_depmerge.py (two changesets adding different dependencies adopt in either order, and are installed;
conflicting requirements refused; a merged pair that leaves the lock stale refused with the reason; a git merge of
two branches through the driver).
Built: dependency operations in changesets (dep_add / dep_update / dep_remove / pdk_set in the working copy,
recorded in its project log, taken back by restore), the install after the adopt and after a revert (from the store,
an entry the store lacks refused at adopt, an interrupted install finished at the next start), the browser's
one-click Add, Update, Remove and Set PDK (POST /deps/changeset), and POST /changesets/{id}/deps and /pdk
(see Changesets and evidence and the HTTP API). Tests:
test_pkg_changesets.py (add in a changeset, adopt, restore; never writing through the working copy's links;
update, remove and set-PDK; the baseline refused; the write lease; the adopt refused for a tree the store lacks; the
install finished after an interrupted adopt; two changesets both installed; a revert uninstalling with the cells that
still place it; a restore while an operation fetches is busy; the one-click route applied, and kept open on a
refused adopt; the project log is no library name).
Built: evidence with package and PDK information, and model-file hashes (store/provenance.py, modellib.files_read,
the evidence manifest in store/changeset.py; see Changesets and evidence). Tests:
test_pkg_evidence.py (an update that changes one cell outdates only the results that use it; a README change
outdates nothing; a model file a package's code block includes outdates the simulation with the same netlist text;
profile fields in the key; about outside it; a job from before stays fresh; the manifest lists packages and PDK).
Still to build in phase 4: the review of a dependency change (the lock diff, licences, size, code blocks and RTL), an
agent's needs fetch and the person's fetch with the confirmation token and the notice, agent tools, and
dep_update's refresh of layout copies.
Phase 5 status¶
Built so far (store/manifest.py, store/spdx.py, store/index.py, store/pkgsource.py, store/checkreport.py,
store/trust.py and undine/pkgcli.py; tests in test_manifest.py, test_pkg_index.py and test_pkg_publish.py, every
repository and index local, none touching GitHub):
[checks]and[renames]typed, andmanifest.render(see Packages); SPDX by syntax only (see Licences).- Index lines:
line_text, the strictercheck,yank_rewrite(see Index). - The package source (store/pkgsource.py).
LIBRARYis a project library's directory, the project found by walking up to itsproject.json, orPROJECT/LIBRARYunder--root; a dependency's or a PDK's library is refused. The tag isv<version>, or<name>-v<version>when HEAD holds more than oneundine-package.toml.at_tagwants no tracked or untracked change under the package's subdirectory (the rest of the repository is not looked at) and HEAD at the tag (an annotated one too), with the manifest committed there and byte for byte the working copy read (an ignored file, or one marked skip-worktree or assume-unchanged, is in nogit status).servedreads the tag's commit from the repository URL (git ls-remote).treereads the include set from git's blobs into the store, as a consumer's fetch does, and sums the files' bytes. pkg init LIBRARY [--name] [--version] [--license] [--repository] [--force]writes the manifest: the library's name, version 0.1.0, the cells with a symbol exported, the include list of the example,[pdk]from the project's PDK (its family and variant;commitsandbuilt_withits full commit, from the installed PDK, else the lock's format 2 pin), the project's index dependencies the library's schematics place as^<locked version>(a nickname written as its URL), andrepositoryfrom the git remoteorigin. A placed git dependency or project library is written as a comment saying why. It prints each value it guessed and warns of what is missing (licence, repository, a full PDK commit); an existing manifest is replaced only with--force.pkg pack LIBRARY [--index I] [--check FILE --check-url URL]prints the index line:gitthe manifest's repository,subdir, the tag's commit, the tree hashed from the local repository,pdk,deps(a requirement'sindexas its URL when it is not the line's own index),exports,license,size,description, andcheckonly with both--checkand--check-url. A git dependency or a manifest withoutrepositoryis refused.- The check report (store/checkreport.py, format 1; its fields in the module's docstring) and
pkg check LIBRARY --no-run [--index I] [--offline] [--out FILE] [--json], which writes it to<project>/.undine/check/<name>-<version>/undine-check.json. Problems: the repository URL does not serve the tag at that commit (with--offlinenot checked, a warning); a file overUNDINE_PKG_FILE_MAX(default 50 MB); an LFS pointer or another file the tree refuses (the tree is then null); no licence, or not an SPDX expression; a git dependency; dependencies that do not resolve from the index (fetched first unless--offline) for the package'sbuilt_with; nobuilt_with, or not the project's PDK commit and variant; a placed library that is neither the package, a declared dependency, a PDK library nordevices(also the library's own name when the package is named otherwise). Warnings: internal cells of a dependency placed; symbols placed with no library part; code blocks and RTL the trust rule refuses.trust.listinglists every code block (each.controlblock with its commands,shellandpre_osdiamong them,*#line,.include/.liband XSPICE file path,tcleval) and every RTL file, refused or not, run on the include tree installed into a temporary directory, so only published files are scanned. Per cell the symbol, netlist, DRC (deck, options, the waivers with their reasons), LVS (when the cell has both a schematic and a layout) and specs entries arenot_run;runs.statenot_runis itself a problem, so a--no-runreport never passes (exit code 1). The report holds no machine path or secret: the repository's, the project's, Undine's home's and the home directory's absolute paths are written as<repository>,<project>,<undine home>and~(only as whole paths, so a name is never touched), a URL loses its user and password, and a local index or repository path becomes<local index>or<local repository>.checkreport.verify, for publish and the index CI, refuses another SHA-256, a format or kind it does not know, another name, version, commit or tree than the line's, and a report that does not pass.
Still to build in phase 5: running the symbol and netlist checks, DRC with waivers and LVS in pkg check (KLayout,
with a server that owns no job root), spec results and the semver report, pkg publish and pkg yank, the index CI
(pkg index check and its workflow file), the cell-level update report with [renames], and refreshed layout copies
in dep_update (after phase 4).
Differences from the text above, and additions:
pkg checkwithout--no-runis refused until the runs are built.pkg checkwrites no report for a package that is not clean at its tag: a report names one commit and tree.- A placed library that is not declared is a problem also when the project's PDK is not installed (then its libraries cannot be told), with a hint to install it.
parse_lineis stricter aboutcheck; that is safe while the default index lists no package.pkg initwritescommits = [built_with], not a compatibility tag: tags come with PDK profiles (phase 6).
Phase 6 status¶
Built: profile.toml and its reader, the registry of formulas and stack generators, the three built-in profile
packages, pdkprofile.py serving them with its Python profiles removed, Undine's table of verified and magic_drc,
and the tested commits read from the profiles by pick_pdk, the PCell code's hint, pixi run pdk, setup.sh and two
commands, pdk tested FAMILY [--all] and pdk enable FAMILY [COMMIT] (see PDK profiles as
packages). Tests: test_profile_data.py (the loaded profiles equal the golden data from
the Python ones, per process and per variant, types included, and so does everything pdkprofile serves: deck -rd
values, Magic and Netgen files, the family rule, the MIM formula; each refusal names its key path; Undine's table for a
tested and an untested profile version; the tested commits, the two commands, pixi.toml and setup.sh naming no
commit, wafer.space's GF180 commit among the tested ones).
Still to build in phase 6: profile packages in the index and the lock's profile key (choosing a version, sync,
vendor), expected tree hashes in [[pdk]] and verified_by (the PDK state verified), compatibility tags read from
the profile, the profile fields a job uses in its inputs (with phase 4's evidence), and PDK licences from the profile.
Decisions¶
Decided with the user on 2026-11-05. Each item gives the decision and why.
- The manifest is TOML. People write it by hand, and TOML takes comments; Python reads it with
tomllib. - The default index is a repository in the kadomoto-lab GitHub organisation (
kadomoto-lab/undine-index, created on 2026-11-20); for now kadomoto reviews its PRs. One place next to Undine is enough while publishers are few. - Several major versions of one package in a project are refused. Subcircuits and layout cells are named by cell name, so two versions of one cell would collide in one netlist or layout.
- A cell-name clash between packages is a resolution error that names the cell and both packages. Cell names stay as their authors wrote them, and the clash is found before anything is placed.
- Local names are project-wide, a package installed under each name it is placed by. The Xschem netlister backend and Xschem outside Undine search one path list and cannot express per-library tables.
- A person's own add also goes through a changeset, adopted with one click. One path for people and agents, so review, merge at adopt and evidence work the same way.
- Install from the store by hard link (copies across file systems or with
UNDINE_STORE_LINK=copy). A second local name or a changeset's working copy then costs no disk. - One PDK tree hash per library (and one for
libs.tech). A PDK installed with more libraries still verifies, and a change is reported for the library it is in. - Publishing rests on the publisher's
pkg check, with a link to its report in the index PR. DRC and LVS in CI need the PDK there (1.1 GB per sky130 variant); the report is bound to the commit and tree, so the reviewer sees what was checked. DRC / LVS in the index CI comes later. - The PDK profile package defines compatibility tags, at one level (layout and models). The profile already lists each PDK commit with its tree hashes, and a layout-only level would let spec results change under a published cell.
- For now only git hashes pin content: the commit and the tree hash. They detect any change of content; signing comes later.
- Tools are recorded as version number and origin per tool, plus Undine's version and its
pixi.lockhash; a mismatch only warns. pixi already pins the tools, and a refusal would block someone whose tool differs by a patch release. - Availability: option (b).
pkg vendorwrites the pinned contents into the tapeout bundle together withpkg licenses(Fetch and store). The tapeout evidence then does not depend on publishers keeping their repositories, with no mirror service to run.
Decided with the user on 2026-11-22, for the dependency operations in changesets (phase 4):
- The browser's one-click Add in a project that is not a git repository makes it one (opening the changeset runs git-setup, as any changeset open does), so every Add goes through a changeset.
- A one-click operation whose adopt is refused (a conflict) keeps its small changeset open for review: the 409 names the changeset; it is not discarded, so the work is not lost and the conflict can be seen.
- The browser's Add, Update, Remove and Set PDK all become one-click changesets (a small changeset, the operation
in it, the adopt). The direct
/depsHTTP routes and the CLI keep working unchanged. - Over HTTP, a local repository under
--dep-rootneeds no confirmation token (for the confirmation of fetches, slice 2):--dep-rootis already the server owner's consent for those directories. - A git rev is confirmed by binding (url, rev) (slice 2): the token names what the person saw.
- A revert that drops a dependency the changeset added proceeds, and returns a warning listing the cells outside the changeset that still place that dependency's cells, instead of refusing: the revert stays possible, and the person sees what to fix.
Decided with the user on 2026-11-22, for phase 6:
- One profile package per ciel family (
profile_sky130,profile_gf180mcu, andprofile_ihp_sg13holding SG13G2 and SG13CMOS5L). Compatibility tags (ihp-sg13/...), licences and ciel's fetches are per family, so one package defines them for both IHP processes. - Profile packages live inside undine-eda, as subdirectories that index lines name with
subdir(server/undine/data/profiles/<package>/). The built-in fallback is then the published package byte for byte. - A profile version newer than this Undine has tested is not verified and inherits
magic_drcfrom the newest tested version. A profile update from an index then does not turn Magic DRC off. - Metal stacks may come from a few named, versioned stack generators (
gf180_stack@1,ihp_stack@1) besides literal data. GF180's and IHP's stacks written out are long lists whose rule names follow patterns; a changed rule is a new generator version, so a profile keeps drawing as it did. - The acceptance test "a changed layer colour outdates nothing" becomes "a changed profile field no job reads
(
label,limits) outdates nothing". Profiles hold no colours: they come from the PDK's.lyp.
Decided with the user on 2026-11-22, for phase 5:
- Each external write of
pkg publishandpkg yankis asked first. The writes (gh release upload,git pushof a branch from a clone of the index,gh pr create, and a fork when the person cannot push to the index) are each shown with the exact command and asked y/N on a terminal; without one they are refused unless--yesis given;--dry-runruns none. No environment variable and no agent tool can confirm. The index's owner pushes the branch to the index directly, without a fork. A write to someone's repository is the person's decision. - Undine ships the index CI workflow file; the user adds it to
kadomoto-lab/undine-index. A commit to that repository stays the user's. - An exported cell's spec that is not met, or not measured, fails
pkg check. For analog IP the specs are the contract. - No test touches the real GitHub; the first real publish is the user's own run. Tests use local repositories
and indexes and a fake
gh. - SPDX expressions are checked by syntax only, with no licence list. Undine ships no third-party reference data, and a new identifier needs no release.
- An added symbol pin needs a minor version. It adds to a cell's interface as a new cell adds to the package's, and a consumer's existing instance leaves it unconnected.
- A waiver applies to the waived cell's own DRC run only. A cell that places it is checked in full, so an abutment cell's waiver does not hide an error in the array.
pkg checkruns KLayout DRC and LVS only. GF180 and IHP have no Magic or Netgen decks in Undine.undine-check.jsonis written under<project>/.undine/check/. Outside the package's directory, it cannot dirty the tree it describes.- The clean-worktree check covers the package's subdirectory only. The rest of the repository (the project's
other libraries,
project.json) does not enter the package's tree. - No HTTP route or UI for publishing in phase 5. Publishing is the command line's, run by the person.
Later¶
- Signing: signed index commits or signed package tags.
- DRC / LVS in the index CI (and regenerating PCell variants there), with a PDK in CI.
- Several major versions of one package in a project, by prefixing subcircuit and layout cell names by library.