Comparison with EBI fork - #242
Draft
jamesamcl wants to merge 42 commits into
Draft
Conversation
… on the hot paths Carries a batch of parsing/serialization correctness and performance work on top of devel. Performance - Replace SipHash with rustc-hash FxHash on the hot hashing paths: the IRI / anonymous-individual interning caches, the component / triple stores, the RDF reader's blank-node maps, and the declaration / logically-equal indexes. Each set is sorted before serialization, so output is unchanged. Adds the rustc-hash dependency. - O(N) RDF-list stitching: assemble each rdf:List in one pass over its cells (head -> values) instead of growing every list by one element per full re-scan of all blank nodes (previously O(list-length x bnode-count), pathological on large ontologies). Non-list / incomplete-list blank nodes are left in place. - Pre-size the SetOntology HashSet and build it by moving (not cloning) components out of the index. RDF reader fidelity - Non-destructive class-expression retrieval: a class-expression blank node may be referenced by more than one parent, so retrieve it by clone rather than removing it on first use. (Side effect: such shared nodes remain in the post-parse "incomplete" pool, which horned-validate currently reports as leftover; see below.) - Keep all reified annotation sets per base triple rather than only the last, and type literal assertions on undeclared properties as annotation assertions (matching ROBOT). - Canonically order the operands of intersection / union so a structurally equal collection blank node matches regardless of which blank node carries the list. Serialization fidelity (RDF/XML + OFN writers) - Reified-annotation and n-ary disjoint round-tripping; write SWRL rule annotations directly on the rule node; OFN writer UTF-8 slicing fix and related class-expression rendering fixes. Known issue - horned-bin's integration_validate_ontology_rdf now fails: horned-validate treats any entry left in the parser's incomplete pool as a failed parse, and the non-destructive class-expression retrieval above intentionally leaves shared blank nodes there. The parse is correct; only the leftover-as-incomplete heuristic is tripped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The RDF/XML reader and the SetOntology fast-build call Instant::now() purely for optional perf timing (printed only under a timing env var). On wasm32-unknown-unknown the std clock traps, so merely reading it aborts the module -- which makes RDF/XML parsing (and anything round-tripping through it) impossible in the browser. Route those calls through a small crate::time shim: web-time's drop-in Instant on wasm, std::time::Instant everywhere else. web-time is added as a wasm-only dependency. No behavioural change off wasm. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two changes let horned-owl compile and run on wasm64-unknown-unknown (the owlmake wasm64 reactor target), with no effect on native or wasm32: - `crate::time`: add a wasm64 arm. wasm32 still uses web-time (browser JS / wasip1 std), native still uses std::time. On wasm64-unknown-unknown the std clock traps and there is no JS/WASI, so `Instant` reads a monotonic clock imported from the host (`host.now_nanos`); the wasmtime host (EBISPOT/semantic-mcp) supplies it. - indexmap: request `features = ["std"]`. indexmap 1.x has no default features and only exposes `IndexMap::new()` / the RandomState default when its build script sees CARGO_FEATURE_STD; on tier-3 wasm64 its autocfg sysroot probe for std fails, so without this it compiles no-std and the build breaks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JyCKuMb2FeEdc95HqnTQjH
Build on wasm64-unknown-unknown (host clock + indexmap std)
Rewrite `io::ofn::writer::write` so that OWL Functional Syntax output matches the layout the OWLAPI (and hence ROBOT / dosdp-tools) produces, making it byte-comparable with committed ROBOT `.ofn` files: - Prefix block emitted in canonical order (default `:`, then owl, rdf, xml, xsd, rdfs, then any remaining prefixes), one per line. Every prefix present in the mapping is still emitted, so the reader round-trips the prefix set. - Ontology header with the version IRI and each ontology annotation on their own lines, a blank line before the body, and no per-axiom indentation. - A leading block of every `Declaration(...)`, grouped by entity type (classes, then object/data/annotation properties, datatypes, individuals) and sorted by IRI within each group. - One `# Classes` / `# Object Properties` / … banner per entity type that owns non-declaration axioms, each entity introduced by a `# Class: <IRI> (label)` comment (label taken from its rdfs:label) followed by its annotation assertions (sorted by property IRI) and then its logical axioms. Axioms with no named subject are written verbatim before the close so nothing is dropped. Matching OWLAPI's trailing blank lines and no final newline. Axiom bodies and IRI/literal abbreviation are still produced by the existing `AsFunctional` renderers, so only the document structure changes. The `#` comments are already skipped by the OFN reader (grammar `COMMENT` rule), so output continues to round-trip; all `io::ofn` tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two functional-syntax parse fixes surfaced parsing ROBOT-produced OWL:
- LanguageTag: use the concrete-syntax LANGTAG production shared by
Turtle/SPARQL/OWL 2 FS (`@ [a-zA-Z]+ ('-' [a-zA-Z0-9]+)*`) instead of the
full BCP47 grammar. As a PEG, BCP47's optional 3-alpha `extlang` subtag
greedily consumed the first three letters of a 4-alpha `script` subtag, so a
well-formed tag like `zh-hans` mis-parsed as `zh-han` + a stray `s`.
- NodeID: also accept the bare node-id form OWLAPI/ROBOT emit for anonymous
individuals (e.g. `anon000001`) alongside the spec's `_:`-prefixed blank node
label. A trailing `!("(" | ":")` guard keeps the bare form from swallowing an
abbreviated IRI (`prefix:local`) or a functional keyword in individual
position (`Variable(...)`), since a real node id is only ever followed by
whitespace or `)`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Owner
|
I notice that you are also using a vendor branch for horned in hermit-rs (which is a bit behind release now). Do these overlap? |
Author
|
Good catch hermit-rs's vendored horned is an older subset of this PR - have checked with claude. So hermit-rs can switch back to upstream horned once this is reconciled. |
The RDF/XML writer emits documents the RDF/XML reader rejects. Reading such a
file back fails with
Validity Error: Unknown entity in equivalent class statement:
[Iri(IRI(".../GO_0051932")), OWL(EquivalentClass), BNode(...)]
Root cause. An OWL entity's type survives into RDF only as an `rdf:type`
triple, and the only component that renders one is `Declaration`. But OWL does
not require a declaration: OWLAPI infers an entity's kind from the axioms it
occurs in, so an ontology may legitimately contain
EquivalentClasses(:GO_0051932 ObjectIntersectionOf(:GO_0007268
ObjectSomeValuesFrom(:RO_0000057 :CHEBI_59888) ...))
with no `Declaration(Class(:GO_0051932))` anywhere. We rendered that subject as
a bare `<rdf:Description rdf:about=".../GO_0051932">`, and on re-read
`distinguish_term_kind` — which can only consult declarations — returned None
and the parse was aborted.
Writer. `Render for &ComponentMappedOntology` now walks the ontology's entity
signature (`visitor::immutable::Walk`, over the *annotated* components so
annotation properties count) and emits the `rdf:type` triple for every entity
no `Declaration` already covers, keyed on (IRI, kind) so punning still gets
both triples. Built-in vocabulary (owl:/rdf:/rdfs:/xsd:/swrl:) is skipped,
since its type comes from the spec, not the document. This reproduces OWLAPI's
`RDFRendererBase`, which renders a declaration triple for every signature
entity: an ontology whose only content is
`EquivalentClasses(:U (:A and :p some :V))` gets `<owl:Class rdf:about=":U">`,
`<owl:Class rdf:about=":V">` and `<owl:ObjectProperty rdf:about=":p">`. The
triples are emitted after the component loop so subjects that already appear
keep their document position and merely gain the type property.
Reader. Files written before this fix (and by any other tool that omits the
triple) still have to parse, so `owl:equivalentClass` no longer insists on a
declaration: the new `distinguish_equivalence_kind` falls back to the kind the
axiom position implies, using the object as tie-breaker — a data-range object
means `DatatypeDefinition`, anything else means `EquivalentClasses`.
Validated: a previously-unreadable file now converts (reader path), a fresh
write of an ontology with an undeclared `EquivalentClasses` subject emits
`<owl:Class rdf:about=".../GO_0051932">` and round-trips (writer path), and the
`io::rdf::writer` test set is unchanged (91 pass / 36 pre-existing failures
before and after, identical failure set).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…r-byte Make the functional-syntax writer produce output byte-identical to OWLAPI/ROBOT for import-bearing OBO edit files (verified against the Cell Ontology's cl-edit.owl: `robot convert` and this writer now agree exactly). The previous writer matched only prefix-poor files. Ordering (all per OWLAPI's `compareTo` / renderer, not lexical): - IRI abbreviation: longest declared prefix that yields a valid CURIE, so `obo:` and a more specific `uberon:` can both be declared and each IRI takes its most specific valid CURIE (falling back to a full <IRI>). Prefixes are emitted in the mapping's (document) order. - Entity sections in OWLAPI order (Annotation Properties, Object Properties, Data Properties, Datatypes, Classes, Named Individuals), each entity's axioms written after its annotation assertions. - Axioms ordered by AxiomType index then structurally (EquivalentClasses before SubClassOf, a named superclass before an anonymous restriction, …). - Annotations (banner-less values and axiom annotations) ordered by OWLAPI's annotation-value type index; `AnnotationValue`'s variant order is aligned to it (IRI < anonymous individual < literal) so the derived `Ord` matches. - Blank-line structure: banner with one trailing blank, a trailing blank per entity, and a `writeSortedEntities` blank per non-empty-signature type (including signature-only types such as Datatypes). - n-ary DisjointClasses (>2) and DifferentIndividuals, plus GCIs, written as trailing general axioms; closing `)` with no trailing blank. - CURIE banners `# Class: obo:CL_0000000 (label)`, with an optional external label map (`write_with_labels`) so a caller can resolve banner labels from the import closure without merging it, and an optional `import_order` so the source document's import order survives the unordered in-memory set. Also fixes a latent bug: a section whose entities carry axioms but no declaration is no longer skipped (which dropped those axioms).
Two fixes so the RDF/XML reader accepts real-world ontologies that OWLAPI/ROBOT load but oxrdf's strict mode rejects (both surfaced building EFO from GSSO): - Enable oxrdf's lenient parser in lax mode (the reader's default), so an invalid BCP47 language tag such as GSSO's `xml:lang="e"` keeps its raw tag instead of panicking the whole document (OWLAPI preserves such literals verbatim). - `owl:equivalentProperty` / `owl:propertyDisjointWith` with a literal object (a cannot-be-a-property triple owlready2 emits, and GSSO even declares the predicate an annotation property) is read as an annotation assertion in lax mode rather than erroring "Cannot distinguish the types". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* OFN writer reproduces OWLAPI's FunctionalSyntaxObjectRenderer byte-for-byte: OWLAPI's axiom grouping and ordering, NCName-only CURIE locals, and angle-bracketed full IRIs in entity banners. * RDF/XML writer types the whole entity signature, not just declared entities. * OBO 1.4 reader and writer, with a ROBOT oracle harness. * Lenient RDF parsing for inputs OWLAPI accepts and a strict parser rejects. * wasm32/wasm64 support; FxHash on the hot paths. Rebased onto upstream devel (v2.x, edition 2024); pretty_rdf is vendored in-tree as horned-pretty-rdf, and build.rs is no longer needed.
A blank node used as an object twice was inlined at neither site: the inline branch was gated on `object_count(bn) == 1`, so both references fell through to a bare `rdf:nodeID` and the collection itself was written nowhere. Re-reading the document then lost the axiom. RO shows it clearly. Its eight ANNOTATED property chains have their list node referenced twice — once as `owl:propertyChainAxiom`, once as the `owl:annotatedTarget` of the `owl:Axiom` reification — and a round trip turned 160 chains into 152. ROBOT/OWLAPI renders such a list inline at every reference, which is what this now does. Restricted to collections whose members are all named nodes, since those can be rendered repeatedly without consuming anything else from the chunk; the count is decremented per reference so the last one still takes the destructive path.
The general-axiom section, the ontology annotations and the declaration list all sorted by horned's structural `Ord`, which orders by its own enum layout and by rendered strings. OWLAPI orders by `OWLObject.compareTo` — type index first, then `compareObjectOfSameType` — and the two disagree in four ways that are all visible in a real module: * Class expressions have `CLASS_EXPRESSION_TYPE_INDEX_BASE`-relative indices, a quantified restriction compares its PROPERTY then its FILLER, and an n-ary boolean compares its operand SET (`compareSets` sorts both sides first). * `OWLSubPropertyChainOfAxiom` compares the chain element-wise, then its length, then the super property. * `IRI.compareTo` is NAMESPACE then remainder, not the whole string, so `…/obo/MF#manifestationOf` sorts after every plain `…/obo/RO_…`. * An annotation value is compared by type index first, and `IRI` is 0 while a literal is `DATA_TYPE_INDEX_BASE`+, so IRI-valued annotations precede literal-valued ones; literals then compare on their DATATYPE IRI before their lexical form. Entity banners also take the LAST `rdfs:label`, matching OWLAPI's short-form provider, where this took the first. Indices and comparators were read off owlapi4's OWLObjectTypeIndexProvider and the *Impl classes rather than inferred.
Owner
|
I have a bunch of ofn/rdf commits going in at the moment, so we may be working on the same thing. |
A bare quoted literal already denotes xsd:string in OWL 2, and OWLAPI's functional renderer omits the datatype. Writing it out also preserved across the file a distinction OWLAPI loses there — an OBO parse yields OWLLiteralImplString and a functional parse OWLLiteralImplPlain, and the two order differently.
OWLAPI leaves the datatype implicit and ROBOT's output follows suit, so that stays the default. Tools bundling an OWLAPI that renders it — dosdp-tools among them — cannot be reproduced byte for byte without the explicit form, so expose a switch for it.
`cargo test` did not reach a single test. `horned-bin` and `horned-macro` still asked for `horned-owl` 2.x by version while this fork's lib stayed on 1.4.0 — deliberately, so it satisfies hermit-rs and whelk-rs's `^1.4` — so resolution failed outright. With that fixed the tests compile, and three of them were stale: * two `InverseObjectProperties` constructions passed an `ObjectProperty` where the axiom now takes an `ObjectPropertyExpression` (the OMN reader test and the `model` doc example); * `test_ofn_literal_datatype` still expected an explicit `^^xsd:string`, which the writer now leaves implicit unless `set_write_xsd_string` turns it on. It asserts the default instead, and a second case covers a datatype that IS written out — the flag is process-global, so toggling it under a parallel test binary is not something a test should do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`retrieve_to_ce` reads the blank-node class-expression map without removing the entry, so one bnode restriction can be referenced by several axioms — which is what ROBOT's RDF/XML writer produces and what dropping the entry on first use silently lost. The consequence was that the map could no longer say what was left over. Every blank-node class expression in a document was reported as unparsed, so `horned-validate` failed on files it had read perfectly: `src/ont/owl-rdf/and.owl`, a single `SubClassOf` over an `ObjectIntersectionOf`, was one of them. Record which ids were actually handed out and subtract them when the parse is decomposed. A genuinely unconsumed expression is still reported, which is what the check is for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OWLAPI writes the leading `Declaration(...)` block from `sortOptionally(ontology.getSignature())`, and `OWLObject.compareTo` compares the TYPE INDEX before the structure. Those indices are not the section ranks this writer uses to group axioms: reading them off owlapi4's `OWLObjectTypeIndexProvider`, Class is 1001, ObjectProperty 1002, DataProperty 1004, NamedIndividual 1005, AnnotationProperty 1006 and Datatype 4001 — so individuals precede annotation properties and datatypes come last, where the rank order puts annotation properties third and individuals last. The section EMIT order already had its own table for the same reason; the declaration sort did not. OBA's `imports/merged_import.owl` is where it showed, being the ODK artefact in functional syntax whose signature holds both individuals and annotation properties. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OWLAPI writes
DLSafeRule(Annotation(…) Body(ClassAtom(…) ObjectPropertyAtom(…))Head(…))
— a space after the annotations and between consecutive atoms, and
`Body(…)Head(…)` adjacent. Everything ran together here, so all 23 SWRL
rules in OBA's `imports/merged_import.owl` were a byte off ROBOT's.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OWLAPI's functional-syntax renderer works from the ontology's SIGNATURE, not from its declaration axioms. Two things followed from that and neither was reproduced here. An entity used in an axiom whose `Declaration(...)` lives in an imported ontology is still in `get<Kind>InSignature()`, so it gets its own `# Object Property: <IRI> (label)` banner and carries its annotation assertions. HPO's `hp-edit.owl` declares no object property at all — BFO and RO come from `merged_import.owl` — yet ROBOT's conversion of it opens a full `# Object Properties` section. Ranking entities by declaration alone dropped those assertions into the trailing general-axiom block instead. `writeDeclarations` then synthesises a `Declaration(...)` for any signature entity that has none of its own, skipping entities that are built in (`OWLEntity.isBuiltIn()`, which differs per kind), illegally punned (`determineIllegalPunnings`: object/annotation, data/annotation, data/object, or datatype/class), or declared somewhere in the imports closure. That last check is why serialising an import-bearing ontology adds nothing while serialising it with the imports stripped adds one declaration per entity that lost its declaring import: `remove --select imports` on `hp-edit.owl` is followed by exactly 2192 of them. `write_full` takes the closure's declared entities so a caller that has resolved it gets that answer exactly. Without one, an ontology that still declares imports gets no added declarations — the `isDeclared(…, INCLUDED)` question cannot be answered, and this matches ROBOT for every import-bearing file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Owner
|
Would suggest that you take this out of draft and we can go for a merge/rebase. I'm also working on the code base and the conflicts are just going to get worse over time! |
`writeSortedEntities` sorts each section with `sortOptionally`, i.e. `OWLObject.compareTo` → `IRI.compareTo`, which compares the NAMESPACE and then the remainder rather than the whole string. A `BTreeSet<&str>` sorted on the full IRI, so `…/obo/chebi/1_STAR` came out before `…/obo/valid_for_gocam` where ROBOT emits the reverse: namespace `…/obo/` sorts before `…/obo/chebi/` however the local parts fall. Also count an ontology annotation's literal towards the Datatypes signature. OWLAPI reports `xsd:string` for it, so `writeSortedEntities` emits that section's trailing blank line; an otherwise empty `definitions.owl` carrying only `Annotation(owl:versionInfo …)` was a line short of ROBOT's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…I does Four things ECTO's merged import needs, each read out of owlapi 4.5.29 rather than inferred. An abbreviated IRI is not the spec's `PNAME_LN`. `CustomTokenizer.readTextualToken` takes every character up to one of `= " ( ) < > @ ^` or whitespace and calls the run a `PNAME_LN` whenever it holds a colon that is not its last character; `getIRI` then splits at the FIRST colon. So a local part may carry colons — and ROBOT writes one: FoodOn's `schema:image` provenance is `wikipedia:User:Lupin`, which the spec production cannot read back at all. The writer has the same rule in reverse. `getPrefixIRIIgnoreQName` falls back to testing the tail after a declared namespace with `XMLUtils.isQName`, and a QName is an NCName *or two NCNames joined by one colon* — so that IRI abbreviates rather than going out in full. An entity's annotation assertions sort by `OWLAnnotationAssertionAxiomImpl. compareObjectOfSameType`: subject, property, value, and nothing else. The derived `Ord` stood in for it and compared `Literal` by variant, so every entity holding both an `@en` label and a plain one came out in the other order. And an untyped literal is `rdf:PlainLiteral`, not `xsd:string`: OWLAPI's RDF parser builds `OWLLiteralImplPlain` with or without a language tag, and the third comparison key is the language. That interleaves an entity's synonyms the way ROBOT writes them — `"beef mince"`, `"beef mince"@en`, `"ground beef"@en`, `"hamburger meat"` — where a datatype split put them in two runs. The banner label follows from the same order. OWLAPI takes the first assertion out of a hash-ordered set whose seed is per-JVM — `robot convert` run twice over one unchanged file labels `oboInOwl:hasDbXref` differently in 6 runs of 20 — so no rule matches it every time; the first label in `compareTo` order agrees with the reference on 6 of the 7 entities that carry more than one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XTcpemgTmBzRfjLA1i6YXf
`AnnotationValueShortFormProvider` keeps the FIRST `rdfs:label` it finds walking `getAnnotationAssertionAxioms(iri)`: with an empty preferred-language map `AnnotationLanguageFilter.visit(OWLLiteral)` sets `lastLangMatchIndex = 0`, and the axiom visit is guarded by `lastLangMatchIndex > 0`, so every later assertion is skipped. That set is a `java.util.HashSet` of the subject's annotation assertions, so "first" is bucket order over the axiom hashCode -- reproducible, not per-JVM as the old comment claimed: `robot convert` run twice over one file gives byte-identical banners. So compute the bucket instead of guessing. It decides ten of the eleven multi-label entities in MP's `uberon_import.owl` (`has_broad_synonym`, `molecular_function`, `biological_process`, `cellular anatomical structure`, `regulates (processual)` ...); the eleventh, `IAO_0000027`, has both labels in one bucket, where the order is OWLAPI's parse-insertion order. Fall back to the old `compareTo` rule there and wherever a label assertion is annotated or typed, since the hash is only reproduced exactly for plain and language literals. Also give SWRL rules their place in the output. `AxiomType.SWRL_RULE` sits between `HAS_KEY` and `ANNOTATION_ASSERTION`, so a rule belongs at the END of the leftover block, after the property chains -- the default index of 0 put UBERON's three at the front of it. And rules sort among themselves by `SWRLRuleImpl.compareObjectOfSameType`: `compareSets` over the body atoms, then over the head atoms, with atoms ranked by `RULE_OBJECT_TYPE_INDEX_BASE` + the visitor ordinal (a class atom before an object-property atom) and then by predicate and arguments. horned's derived order put RO's twenty-four rules in a quite different sequence. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuS6PNkerhM1rLBk8GHfxh
`FunctionalSyntaxObjectRenderer.write(Collection)` has a special arm for a collection of EXACTLY TWO: it takes the first element and, unless that one IS the focused object (the entity whose block is being written), writes the SECOND first. An SWRL atom is never the focused object, so a two-atom body or head always comes out in the opposite order to the one stored. UBERON's three rules show it plainly: the RDF list in its mirror runs `BFO_0000050(x,y)`, `BSPO_0000120(y,z)` and ROBOT writes `Body(BSPO_0000120(y,z) BFO_0000050(x,y))`. Reading that back and writing it again swaps it once more -- which is exactly what `robot convert` does to its own output, so the quirk lives in the renderer, not in the model. It reached MP through `ro_import.owl`, whose seventeen two-atom rules put 222 lines into each of `mp.owl`, `mp-full.owl` and `mp-international.owl`. The round-trip test compares rules with their atoms sorted now. A rule's body and head are SETS in OWL -- horned stores them as `Vec` to keep a document's order -- so permuting a two-atom one loses nothing, and the reader must NOT swap back: ROBOT's own parser does not, which is why the RDF list in `mp-full.owl` follows the functional file rather than the model it came from. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuS6PNkerhM1rLBk8GHfxh
Two `rdfs:label` assertions on one subject can land in the same bucket of that subject's annotation-assertion set, and then which one `AnnotationValueShortFormProvider` reaches first is decided by which was inserted first. Insertion order is the iteration order of the axiom set the ontology was created from -- a `HashSet` too -- so it is bucket order again, in a table sized for the whole ontology rather than for one subject. That settles both ties MP has: `data item` over `data entity` for `IAO_0000027` in `uberon_import.owl`, and `has cross-reference` over `database_cross_reference` for `oboInOwl:hasDbXref` in `ro_import.owl`. The capacity is taken from the ontology being written, standing in for the set it was created from; the two differ by whatever a later `remove` took, and land in the same power-of-two band here. A tie at that capacity as well still falls back to `compareTo`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuS6PNkerhM1rLBk8GHfxh
`oboInOwl:hasDbXref` carries the same two labels and the same five annotation assertions in `uberon_import.owl`, `cl_import.owl` and `ro_import.owl`, so all three tie in the subject's own bucket -- and ROBOT resolves the tie one way for uberon and cl (`database_cross_reference`) and the other for ro (`has cross-reference`). A table sized for the whole ontology puts `has cross-reference` first at every capacity above 16, so it cannot produce both; it fixed ro and broke the other two. Whatever decides a within-bucket tie is the order the module's axioms were added, and `SyntacticLocalityModuleExtractor` returning `new HashSet<>(module)` does not explain it. Back to `compareTo`, which is right for uberon and cl and wrong for ro, until the real order is known. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuS6PNkerhM1rLBk8GHfxh
`owlapi_literal_cmp` called every untyped literal `rdf:PlainLiteral`, which is right for a parse and wrong after `robot query --update`: OWLAPI hands the updated model back with each of them as `xsd:string`, and 2001#string sorts AFTER 2001#anyURI where 1999#PlainLiteral sorts before it. MONDO's `imports/merged_import.owl` ends `query --update` x3 then `convert -f ofn`, so all seven of its `xsd:anyURI` ontology annotations came out after the `ISBN:`/`MESH:`/`PMID:` ones instead of before -- the last difference between owlmake's refreshed import module and the ODK's. Only the caller knows which kind it has, so the caller says: default Plain (every ordinary parse, including the RDF/XML trip that made the untagged synonym sort before its `@en` twin), and `set_plain_literals_typed(true)` for a document that came back from Jena. A language-tagged literal is `OWLLiteralImplPlain` either way and is untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D1J3937rhqCcUdJgpGERUG
`IRI.compareTo` orders by namespace then remainder, and `IRI.create` splits those at `XMLUtils.getNCNameSuffixIndex` — the last position from which the rest of the string is a valid NCName. That is not "after the last / # or :": a local part starting with a digit is not an NCName, so the boundary moves right past the digits. `…/10.1161/circ.105.9.e5` therefore splits after `1161/` and sorts BEFORE `…/10.1161/01.CIR.0000132478.60674.D`, which splits after `1161/01.`. RO's `skos:narrowMatch` targets under `identifiers.org/metacyc.reaction/` are the same shape. An axiom's annotation set was also being ordered with plain string and derived comparisons instead of OWLAPI's own keys, so a datatyped literal sorted by its lexical form rather than by its datatype: CL's `"…"^^xsd:anyURI` cross-references came out after the plain ones where ROBOT puts them first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D1J3937rhqCcUdJgpGERUG
`writeEntities` drops any axiom already in `writtenAxioms`, so an IRI declared as more than one kind has its annotation assertions rendered under whichever of its sections comes first, and the later ones show only what is left. `IAO_0000125` is both an annotation property and a named individual: its three assertions belong to the Annotation Properties section, and the Individuals one gets only its `ClassAssertion`. The rank map kept whichever declaration it saw last instead, so the assertions moved to the Individuals section and the Annotation Properties section lost the entity altogether. Also: the within-bucket tie in the banner-label rule has no reproducible answer, and the comment now says which mechanism makes it so rather than claiming ROBOT is stable there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D1J3937rhqCcUdJgpGERUG
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D1J3937rhqCcUdJgpGERUG
`owlapi_axiom_cmp` fell through to horned-owl's derived `Ord` for two axioms of the same type, and that is not OWLAPI's order for class expressions: `OWLObjectTypeIndexProvider` puts `ObjectExactCardinality` (3009) before `ObjectMaxCardinality` (3010), where the enum here has them the other way round. PRO's `PR_000050469` carries one of each over the same property, so its max-cardinality restriction came out three axioms early. `SubClassOf`, `EquivalentClasses` and `DisjointClasses` now use the class-expression comparator that was already here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D1J3937rhqCcUdJgpGERUG
`data_ranges` fills `data_range` over repeated passes, so an anonymous member of a `DataIntersectionOf`/`DataUnionOf` sequence may not be a data range yet when the enclosing bnode is visited — the bnode map is a HashMap and its iteration order decides which comes first. `retrieve_to_dr_seq` retrieved regardless, and `retrieve_to_seq` removes the sequence from `bnode_seq`, so the later pass that could have completed it found nothing left to read and the axiom was dropped. Guard as `retrieve_to_ce_seq` already does: leave the sequence in place unless every anonymous member is a resolved data range.
A digit is an XML name character but not a name start character, so a local part beginning with one is not an NCName and the IRI is written in full. ChEBI's subset values reach uPheno's merged mirror as `<http://purl.obolibrary.org/obo/3_STAR>` for this reason, in ninety `oboInOwl:inSubset` assertions, and `obo:3_STAR` would not read back as the same IRI under a functional-syntax parser that follows the same rule. The test is now OWLAPI's own — a start character, then name characters, colon excluded at both — in place of a character blacklist that could only reject what it listed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0179rxRLc2jQ3XHaiuAd72GA
…datatype Three corrections to the functional-syntax writer, all visible in one file: FoodOn's ODK mirror, written as OFN, differed from ROBOT's on 3,835 lines. **Abbreviation.** `shrink_valid` picked the declared namespace that left the shortest local part. The rule is the IRI's OWN namespace — everything before its longest NCName suffix — looked up EXACTLY, and only when that misses, the longest declared namespace the IRI starts with that leaves a QName behind. The NCName start-character rule that arrived in the commit before this one is folded into that QName test, so `…/obo/3_STAR` still goes out in full; what it did not catch is `…/obo/FOODON:03415183`, whose local part is a name-then-colon-then-digits and so is not a QName either, and `…/wiki/Citrus_×_sinensis`, whose U+00D7 is excluded from the name characters between 0xC0–0xD6 and 0xD8–0xF6. `wikipedia:User:Lupin` still abbreviates: a QName may carry one interior colon. **Built-in datatypes.** `is_builtin_entity` treated every IRI in the four schema namespaces as built in, so no datatype ever got a synthesised declaration. The set is the OWL 2 datatype map, and `xsd:date` is not in it — nor `xsd:time`, nor the `gYear` family — so a document that uses one declares it. **Signature.** A typed literal puts its datatype in the signature; the scan marked only `Datatype` entities, so that declaration had nothing to come from. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FjUSQe47ZwZx9f9sHgwhVu
An `owl:Axiom` block names the axiom it annotates by subject, predicate and object. A document that carries the block without stating that triple still means the axiom, so the triple is put back and the ordinary translation builds it with the block's annotations. An SSSOM mapping set in RDF is written exactly this way: every mapping is an `owl:Axiom` block and no base triple is stated. uPheno reads one to build `components/upheno-mappings.owl` — a SPARQL update whose WHERE clause is the base triple — and its 51,582 mappings arrived as nothing but anonymous individuals, leaving the component empty. Only a triple between named things is restored. A blank-node subject or object belongs to a construct held elsewhere — a class expression, an RDF list — which the translation reaches by its own route. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0179rxRLc2jQ3XHaiuAd72GA
…o it The rest of the parse reads the simple triples in the order the document states them, which is ascending position but not guaranteed to be — a triple built rather than read carries no position. Sorting the whole sequence would move those; merging the restored triples in at their block's position leaves every existing entry where it was. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0179rxRLc2jQ3XHaiuAd72GA
An ANONYMOUS ontology writes `Ontology(` and carries straight on to its imports, annotations and axioms; only the blank line that separates the header from the body ends that line. Writing the break unconditionally gave such a document a blank line the format does not have — a species subset's tags file is one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Mcp2Q3W4r2d6vxbnZKNrq
A blank node that states a single annotation is read as an anonymous individual; one that states several — a type and its annotations, as an SSSOM mapping set does — fell through to the leftover pile and was dropped whole, and no assertion pattern accepted a blank-node subject at all, so its `rdf:type` never became a ClassAssertion either. A document written from such an ontology could not be read back: uPheno's two mapping-set nodes and the 111,998 mappings hanging off them vanished on the round trip that a `--output $@.tmp.owl && mv` recipe performs. A group whose every triple states something about an INDIVIDUAL — a type naming an ordinary class, or an annotation — is that individual, and the triples take one node id between them rather than one each. Everything else a blank node can be (a restriction, a list cell, a reification) names a built-in `owl:`/`rdf:` object and has been consumed by the time this runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0179rxRLc2jQ3XHaiuAd72GA
`OwlapiNodeID` reads the colon-free form OWLAPI writes for an anonymous individual. Its alphanumeric run stops at the first character that is not alphanumeric, and the trailing guard only rejected a following `(` or `:` — so in `mp-edit:Europhenome_Terms` it matched `mp`, leaving `-edit:…` unconsumed and the axiom unparseable. A prefix holding a hyphen is ordinary: ROBOT's own functional-syntax output for MP carries that annotation value, and reading it back failed. A node id is a whole token, so what follows it must END the token. Guard on the token-character set rather than on two of its members.
Reading every leftover blank-node group as an anonymous individual took in the structure a parse leaves behind as well, and a 439 MB merged mirror grew past 11 GB before it was killed. What separates the two is the type triple: an individual is typed by an ordinary class, while a restriction, a list cell and a reification all name a vocabulary term there — a different `Term` variant. Without such a triple the group is structure this parse did not understand, and it stays where it was. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0179rxRLc2jQ3XHaiuAd72GA
A parse minted a fresh `anonNNNNNN` for each anonymous individual it recognised, counting from one per `Build`. Two documents parsed for one merge therefore named different nodes the same thing, and the labels bore no relation to the ids the document's other blank nodes would be written with. `Build::set_bnode_base` turns on document numbering: every blank node a parse meets is given `genid<n>` in the order the document first mentions it, counting up from where the caller set the base, and `bnode_base` reads the count back so the next document carries on from there. An anonymous individual is then known by its node's own id, so every triple about one node meets one individual. Off by default — a parse keeps its own labels, and an anonymous individual gets the predictable name it did before. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0179rxRLc2jQ3XHaiuAd72GA
…tions them A document takes one id for each blank node it declares, and the individuals among them take the ids that follow — the individuals' ids are consecutive, and they run in the order the document first mentions each node, not in the order the parse happened to collect the groups. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0179rxRLc2jQ3XHaiuAd72GA
`skip_bnode_labels` held the counter borrowed across the write that advanced it, so the first document numbered this way panicked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0179rxRLc2jQ3XHaiuAd72GA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch is the EBI (EBISPOT) fork's
devel, opened as a comparison against upstreamphillord:devel. It carries six commits of correctness, performance, interoperability, and WebAssembly work, driven by using horned-owl to read and write the OWL that ROBOT / the OWLAPI produce over large ontologies.The changes fall into four themes. Serialization output stays stable throughout: each set is sorted before writing, so the hash-function switch below doesn't perturb ordering, and all
io::ofntests pass.1. RDF/OFN reader & writer fidelity
disjointround-tripping; SWRL rule annotations written on the rule node; OFN writer UTF-8 slicing and class-expression rendering fixes.2. Performance on the hot paths
rdf:Listis assembled in a single pass over its cells, instead of re-scanning every blank node once per element (previously O(list-length × bnode-count), pathological on large ontologies).SetOntologyHashSetand build it by moving components out of the index rather than cloning.3. ROBOT / OWLAPI functional-syntax compatibility
io::ofn::writer::writenow emits the layout the OWLAPI (and hence ROBOT / dosdp-tools) produces: a canonical prefix block, an ontology header, a leading block ofDeclaration(...)grouped by entity type and sorted by IRI, then# Class: <IRI> (label)sections carrying each entity's annotation assertions followed by its logical axioms. This makes output byte-comparable with committed ROBOT.ofnfiles. The#comments are skipped by the OFN reader, so output still round-trips.LANGTAGproduction shared by Turtle / SPARQL / OWL 2 FS instead of the full BCP47 grammar, which as a PEG greedily mis-parsed a well-formed tag likezh-hansaszh-han+ a strays.anon000001-style ids OWLAPI/ROBOT emit for anonymous individuals, alongside the spec's_:-prefixed blank-node label, guarded so the bare form can't swallow an abbreviated IRI or a functional keyword.4. WebAssembly support (wasm32 + wasm64)
crate::timeshim replaces the directInstant::now()calls in the RDF reader andSetOntologybuild (used only for optional perf timing behind a timing env var). Onwasm32the std clock traps, so merely reading it aborts the module — which otherwise makes browser RDF/XML parsing impossible. The shim routes throughweb-timeon wasm32, a host-imported monotonic clock (host.now_nanos) on the wasm64 reactor target, andstd::timenatively. No behavioural change off wasm.indexmapgainsfeatures = ["std"]so it builds on tier-3wasm64-unknown-unknown(where its autocfg sysroot probe would otherwise compile it no-std and break the build). Harmless elsewhere.Known issue
horned-bin'sintegration_validate_ontology_rdfnow fails:horned-validatetreats any entry left in the parser's incomplete pool as a failed parse, and the non-destructive class-expression retrieval in theme 1 intentionally leaves shared blank nodes there. The parse itself is correct — only the leftover-as-incomplete heuristic is tripped.Scope
15 files changed, +950 / −207. New dependencies:
rustc-hash(all targets) andweb-time(wasm32 only).Commits
🤖 Generated with Claude Code