Skip to content

Concept Registry — epistemic-graph

Prefix: CONCEPT:EG-* / CONCEPT:EPG-* Bridge: CONCEPT:AU-ECO.messaging.native-backend-abstraction (Unified Toolkit Ingestion)

Project-Specific Concepts

These concepts are actively realized by the compiled Rust/Python Epistemic Graph backend in this repository.

Concept ID Name Description
CONCEPT:EG-KG.compute.graph-compute-engine High-Performance Graph Compute Engine Optimized native-compiled memory model and search traversal (DFS/BFS) for the Knowledge Graph.
CONCEPT:EG-KG.mining.frequent-itemset-mining Association-rule / frequent-itemset mining The unified data-mining surface (Phase 1): pure-Rust, dependency-light frequent-itemset + association-rule mining (support / confidence / lift) over EITHER explicit transactions OR a graph-derived transaction source, compute-near-data. One Method::MineAssociate (feature mining) → handlers/mining.rsclient.mining.associate → the graph_mine MCP verb + /api/mining/associate REST twin. crates/eg-compute/src/mining/association.rs.
CONCEPT:EG-KG.mining.apriori-levelwise Apriori (level-wise) frequent-itemset engine Breadth-first candidate-generation + downward-closure prune. One of three interchangeable engines; exact and agrees with FP-Growth/Eclat for a given min_support. crates/eg-compute/src/mining/association.rs::apriori.
CONCEPT:EG-KG.mining.fpgrowth-prefix-tree FP-Growth (prefix-tree) frequent-itemset engine Frequency-ordered FP-tree + conditional-pattern-base recursion — NO candidate generation (default engine, fastest on dense baskets). crates/eg-compute/src/mining/association.rs::fpgrowth.
CONCEPT:EG-KG.mining.eclat-vertical-tidset Eclat (vertical tid-set) frequent-itemset engine Depth-first search over the vertical data layout — a k-itemset's support is the size of the intersection of its items' transaction-id sets. crates/eg-compute/src/mining/association.rs::eclat.
CONCEPT:EG-KG.mining.graph-derived-transactions Graph-derived transaction source Turns node neighborhoods into transactions (TransactionSource {node_label, direction, item_field, relation, limit}) so association mining runs directly over resident graph data — the cross-modal hook (e.g. "for each :Capability, the set of concepts it touches" ⇒ concept↔capability co-occurrence rules). src/server/handlers/mining.rs::derive_from_graph.
CONCEPT:EG-KG.mining.rule-writeback :AssociationRule KG write-back (the discovery flywheel) writeback=true materializes each mined rule as a typed :AssociationRule node (antecedent/consequent/support/confidence/lift) with a deterministic id, linked (RULE_ITEM) to any item that is a resident node, so OWL reasoning + the next mining pass consume it. WAL-replayed by re-mining deterministically. src/server/handlers/mining.rs::materialize_rules.
CONCEPT:EG-KG.mining.dbscan-density DBSCAN density clustering (Phase 2) Density-based clustering: a core point has ≥ min_pts points within radius eps; clusters grow by density-reachability; un-dense points are noise (label -1). Pure-Rust, deterministic. Method::MineCluster (algorithm=dbscan) → client.mining.clustergraph_mine action=cluster + /api/mining/cluster. crates/eg-compute/src/mining/cluster.rs::dbscan.
CONCEPT:EG-KG.mining.hierarchical-linkage Hierarchical agglomerative clustering Agglomerative merge of the closest clusters (single/complete/average linkage via Lance-Williams) cut to a flat k-cluster partition. Deterministic. crates/eg-compute/src/mining/cluster.rs::hierarchical.
CONCEPT:EG-KG.mining.gmm-em Gaussian Mixture Model (EM) Diagonal-covariance Gaussian mixture fit by Expectation-Maximization; seeded k-means++ init, soft responsibilities (n × k) + an argmax hard label. crates/eg-compute/src/mining/cluster.rs::gmm.
CONCEPT:EG-KG.mining.kmedoids-pam k-Medoids (Partitioning Around Medoids) PAM greedy BUILD + best-improving SWAP; cluster centers are actual data points (robust to outliers). Deterministic, seed-free. crates/eg-compute/src/mining/cluster.rs::kmedoids.
CONCEPT:EG-KG.mining.zscore-mad z-score / MAD anomaly detection Robust modified z-score (0.6745·(x−median)/MAD, Iglewicz–Hoaglin) per feature, aggregated to a row score. Method::MineAnomaly (algorithm=zscore) → client.mining.anomalygraph_mine action=anomaly + /api/mining/anomaly. crates/eg-compute/src/mining/anomaly.rs::zscore_mad.
CONCEPT:EG-KG.mining.isolation-forest Isolation Forest anomaly detection Random isolation-tree ensemble; anomalies isolate in short paths → high 2^(−E[h]/c(ψ)) score. Deterministic per seed. crates/eg-compute/src/mining/anomaly.rs::isolation_forest.
CONCEPT:EG-KG.mining.lof-local-density Local Outlier Factor (LOF) k-neighbor local reachability density ratio; a point in a sparser neighborhood than its neighbors scores > 1. crates/eg-compute/src/mining/anomaly.rs::lof.
CONCEPT:EG-KG.mining.oneclass-svm One-Class ν-SVM anomaly detection Schölkopf ν-SVM boundary fit by a lightweight SMO (linear / RBF kernel); a point outside the boundary (f(x)<0) is anomalous. crates/eg-compute/src/mining/anomaly.rs::one_class_svm.
CONCEPT:EG-KG.mining.node-embedding-source Cross-modal vector source VectorSource {node_label, limit} gathers the stored embeddings of a node label as the mining feature rows (compute-near-data), so "cluster / anomaly-detect the vectors of these nodes" runs in one round-trip with the mined node linked back to its source. src/server/handlers/mining.rs::gather_embeddings.
CONCEPT:EG-KG.mining.cluster-writeback :Cluster KG write-back writeback=true materializes each non-noise cluster as a typed :Cluster node (algo/cluster_id/size/members/centroid/score, deterministic id) linked (CLUSTER_MEMBER) to its resident member nodes. WAL-replayed deterministically. src/server/handlers/mining.rs::materialize_clusters.
CONCEPT:EG-KG.mining.anomaly-writeback :Anomaly KG write-back writeback=true materializes each flagged row as a typed :Anomaly node (algo/score/source, deterministic id) linked (ANOMALY_OF) to its resident source node — RCA feeds OWL reasoning + the next mining pass. WAL-replayed deterministically. src/server/handlers/mining.rs::materialize_anomalies.
CONCEPT:EG-KG.mining.naive-bayes Naive Bayes classification (Phase 3) Gaussian + Multinomial (Laplace-smoothed) Naive Bayes — PREDICTIVE fit→model-blob→predict, mirroring the datascience estimators. Method::MineClassifyFit/MineClassifyPredict (feature mining) → client.mining.classify_fit/.classify_predictgraph_mine action=classify_fit\|classify_predict + /api/mining/{classify_fit,classify_predict}. crates/eg-compute/src/mining/classify.rs::{fit_gaussian_nb,fit_multinomial_nb}.
CONCEPT:EG-KG.mining.knn-classify k-NN classification Brute k-nearest-neighbor majority vote over the training rows (proba = per-class vote fraction); deterministic distance/index tie-break. crates/eg-compute/src/mining/classify.rs::knn_predict.
CONCEPT:EG-KG.mining.logistic-regression Logistic regression (one-vs-rest) One-vs-rest linear classifier fit by batch gradient descent on the logistic loss (+L2); recovers a linear boundary, handles binary + multiclass. Deterministic (zero init, index-ordered batch). crates/eg-compute/src/mining/classify.rs::fit_linear_ovr.
CONCEPT:EG-KG.mining.linear-svc Linear SVM (SVC, one-vs-rest) One-vs-rest linear SVM fit by Pegasos-style sub-gradient hinge-loss descent (regularization C). Deterministic. crates/eg-compute/src/mining/classify.rs::fit_linear_ovr.
CONCEPT:EG-KG.mining.truncated-svd Truncated SVD reduction (Phase 3) DESCRIPTIVE row transform: top-k right singular vectors via the Gram-matrix eigendecomposition; coords = X·V = U·Σ, reconstructs a low-rank matrix to round-off. Method::MineReduce (algorithm=svd) → client.mining.reducegraph_mine action=reduce + /api/mining/reduce. crates/eg-compute/src/mining/reduce.rs::truncated_svd.
CONCEPT:EG-KG.mining.lda-discriminant Fisher LDA (supervised reduction) Linear Discriminant Analysis (needs labels): whiten by the within-class scatter, take the leading eigenvectors of the between-class scatter; projects onto ≤ (n_classes−1) discriminants. crates/eg-compute/src/mining/reduce.rs::lda.
CONCEPT:EG-KG.mining.umap-layout UMAP layout (approximate) Fuzzy k-NN neighbor graph laid out by attractive/repulsive SGD (pure-Rust CPU). Approximate + small-N (viz-scale); preserves cluster structure, deterministic per seed. crates/eg-compute/src/mining/reduce.rs::umap.
CONCEPT:EG-KG.mining.tsne-embedding t-SNE embedding (approximate) Perplexity-calibrated Gaussian affinities matched to a Student-t low-D layout by gradient descent (adaptive-gains optimizer). Approximate + small-N; preserves cluster structure, deterministic per seed. crates/eg-compute/src/mining/reduce.rs::tsne.
CONCEPT:EG-KG.mining.classify-writeback :Classification KG write-back classify_predict with writeback=true materializes each prediction as a typed :Classification node (label/proba/source, deterministic id) linked (CLASSIFIED_AS) to its resident source node. WAL-replayed deterministically. src/server/handlers/mining.rs::materialize_classifications.
CONCEPT:EG-KG.mining.reduce-writeback :Embedding2D KG write-back reduce with writeback=true materializes each row's reduced vector as a typed :Embedding2D node (coords/source, deterministic id) linked (REDUCED_FROM) to its resident source node — feeds the web-UI graphviz + downstream clustering. WAL-replayed deterministically. src/server/handlers/mining.rs::materialize_embeddings.
CONCEPT:EG-KG.graphlearn.link-predictor KAN link-predictor (Track G / EpiGNN) A pure-Rust Kolmogorov-Arnold Network link-predictor over the resident graph: per-candidate-pair structural features (common-neighbors / Jaccard / Adamic-Adar / preferential-attachment / PageRank-product / neighbor-cosine / 1-hop-aggregated node-feature dot) fed through a 1–2 layer KAN (each layer output = a SUM of learnable KanEdgeFns per input) to a sigmoid link score, trained BATCH with the existing Adam kernels on observed edges (positives) vs sampled non-edges (negatives). One Method::GraphLearnFit/GraphLearnPredict (feature graphlearn) → handlers/graphlearn.rsclient.graphlearn.fit/predict → the graph_learn MCP verb + /api/graphlearn/* REST twin. crates/eg-compute/src/graphlearn/link_predict.rs.
CONCEPT:EG-KG.graphlearn.kan-edge-function Learnable univariate edge function The KAN atom: a learnable f(x)=Σ_k c_k·B_k(tanh x) over an orthogonal-polynomial basis, with forward + analytic gradients (w.r.t. coefficients for training, w.r.t. input for message passing). Its coefficients ARE the interpretable artifact. Serializable. crates/eg-compute/src/graphlearn/edge_fn.rs::KanEdgeFn.
CONCEPT:EG-KG.graphlearn.chebyshev-basis Polynomial (Chebyshev/Jacobi) basis The edge-function basis is a closed-form three-term polynomial recurrence — Chebyshev (first kind) by default, Jacobi optional — deliberately NOT B-spline/RBF: zero special functions, matching the no-BLAS/no-libm Raspberry-Pi contract. crates/eg-compute/src/graphlearn/edge_fn.rs::{chebyshev_values,jacobi_values}.
CONCEPT:EG-KG.graphlearn.neighbor-aggregation 1-hop message-passing (lite) Refine each node's feature vector by mixing in its 1-hop neighbours' mean (x'_v = α·x_v + (1−α)·mean(x_u)) — optionally edge-weighted by a learned KanEdgeFn (the "the edge IS the computation" primitive) — before pair scoring. O(deg), deterministic. crates/eg-compute/src/graphlearn/neighbor_aggregate.rs.
CONCEPT:EG-KG.graphlearn.predicted-edge-writeback :PredictedEdge KG write-back GraphLearnPredict with writeback=true materializes each scored candidate as a typed :PredictedEdge node (src/dst/score/model, deterministic id) linked (PREDICTED_LINK) to its resident endpoints — inferred links feeding the evolution/reasoning flywheel. WAL-replayed deterministically. src/server/handlers/graphlearn.rs::materialize_predicted_edges.
CONCEPT:EG-KG.graphlearn.edge-function-writeback :EdgeFunction KG write-back (the differentiator) GraphLearnFit with writeback=true materializes each learned per-feature edge function as a typed :EdgeFunction node (feature/basis/degree/coefficients, deterministic id) — the LEARNED scoring curve is itself queryable/explainable from Cypher/SQL, the one place KAN's interpretability is load-bearing. WAL-replayed deterministically. src/server/handlers/graphlearn.rs::materialize_edge_functions.
CONCEPT:EG-KG.mining.prefixspan PrefixSpan sequential-pattern mining (Phase 4) Projection-based, no-candidate-generation sequential-pattern miner: recursively counts frequent next-items over the current projected database (the suffix after the first occurrence of the pattern-so-far's last item) and grows the pattern one item at a time. Method::MineSequence (algorithm=prefixspan, feature mining) → client.mining.sequencegraph_mine action=sequence + /api/mining/sequence. crates/eg-compute/src/mining/sequence.rs::prefixspan.
CONCEPT:EG-KG.mining.gsp GSP (Generalized Sequential Pattern) mining The sequence analog of Apriori: level-wise candidate generation (join two frequent (k-1)-patterns whose overlap matches) + a downward-closure prune, support counted by a direct ordered-subsequence-containment scan. Exact; agrees with PrefixSpan for the same min_support (parity-tested). crates/eg-compute/src/mining/sequence.rs::gsp.
CONCEPT:EG-KG.mining.sequence-writeback :SequentialPattern KG write-back sequence with writeback=true materializes each mined pattern as a typed :SequentialPattern node (items/support/count, order-sensitive deterministic id) linked (PATTERN_ITEM) to any item that is a resident node — the "what reliably follows what" discovery feed for the loop engine's next-step prediction. WAL-replayed deterministically. src/server/handlers/mining.rs::materialize_patterns.
CONCEPT:EG-KG.mining.arima ARIMA forecasting (Phase 4) ARIMA(p,d,q): d-order differencing to stationarity, then AR(p)/MA(q) coefficients fit by the Hannan-Rissanen two-stage method (a long auxiliary AR gives a residual proxy, then AR+MA terms are jointly estimated by OLS via hand-rolled Gaussian elimination — no forecasting crate). Deterministic. Method::MineForecast (algorithm=arima, feature mining) → client.mining.forecastgraph_mine action=forecast + /api/mining/forecast. crates/eg-compute/src/mining/forecast.rs::arima.
CONCEPT:EG-KG.mining.holt-winters Holt-Winters / ETS forecasting Additive level/trend/seasonal exponential smoothing (alpha/beta/gamma, seasonal period); degrades to Holt's linear-trend method (ETS(A,A,N)) when period is 0 or the series is shorter than two seasonal cycles. Level/trend seeded by a lag-free whole-series OLS regression (not a first-vs-second-season mean difference, which would bias the seed toward the window center). crates/eg-compute/src/mining/forecast.rs::holt_winters.
CONCEPT:EG-KG.mining.stl-decomposition STL-style decomposition + forecast A classical (moving-average) trend/seasonal/residual decomposition — a lightweight, hand-rolled stand-in for full iterative Loess-STL (documented scope cut, like Phase 3's approximate UMAP/t-SNE) — extended into a forecast by linearly extrapolating the trend's last valid slope and repeating the last seasonal cycle. crates/eg-compute/src/mining/forecast.rs::{stl_decompose,stl_forecast}.
CONCEPT:EG-KG.mining.forecast-writeback :Forecast KG write-back forecast with writeback=true materializes the point forecast + confidence band as a typed :Forecast node (algo/horizon/values/lower/upper/series_id, deterministic id — from series_id when given, else a hash of the input values) linked (FORECAST_OF) to a resident node named series_id when one exists — feeds the evolution flywheel's "anticipate where to invest" trajectory forecasting. WAL-replayed deterministically. src/server/handlers/mining.rs::materialize_forecast.
CONCEPT:EG-KG.mining.tfidf TF-IDF text mining (Phase 4) Term-frequency × smoothed-inverse-document-frequency (ln(N/(1+df))+1, never diverges for a term in every doc); descriptive, read-only per-document term weights, sorted by descending weight. Method::MineText (algorithm=tfidf, feature mining) → client.mining.textgraph_mine action=text + /api/mining/text. crates/eg-compute/src/mining/text.rs::tfidf.
CONCEPT:EG-KG.mining.lda-topic-model LDA topic modeling Latent Dirichlet Allocation fit by collapsed Gibbs sampling (symmetric Dirichlet priors alpha/beta); deterministic per seed. Returns each topic's full term distribution (sorted descending) and each document's topic-membership distribution. crates/eg-compute/src/mining/text.rs::lda.
CONCEPT:EG-KG.mining.nmf-topic-model NMF topic modeling Non-negative Matrix Factorization of the TF-IDF matrix by Lee & Seung's multiplicative-update rule (W·H minimizing \|\|V-WH\|\|²); seed only determines the initial W/H factors, the update rule itself is deterministic given them. crates/eg-compute/src/mining/text.rs::nmf.
CONCEPT:EG-KG.mining.topic-writeback :Topic KG write-back text with writeback=true (lda/nmf only — tfidf has no topics, the flag is ignored) materializes each topic as a typed :Topic node (algo/terms, deterministic id from its top terms) linked (HAS_TOPIC) from every resident source document whose DOMINANT topic (argmax of its doc_topics distribution) it is — a downstream feed for association-mining topics↔entities. WAL-replayed deterministically. src/server/handlers/mining.rs::materialize_topics.
CONCEPT:EG-KG.mining.gspan-frequent-subgraph gSpan-style frequent subgraph mining (Phase 4, graph-native differentiator) UNLIKE every other mining family member, mines the RESIDENT GRAPH's own topology directly (no rows/vectors handed in): level-wise growth from every frequent single labeled edge up to max_edges, canonicalizing each candidate (brute-force permutation over its tiny node set) and exactly re-counting embeddings by backtracking subgraph isomorphism. Support = raw embedding count (a documented simplification vs. minimum-node-image support). Method::MineSubgraph (algorithm=gspan, feature mining) → client.mining.subgraphgraph_mine action=subgraph + /api/mining/subgraph. crates/eg-compute/src/mining/subgraph.rs::mine_gspan.
CONCEPT:EG-KG.mining.motif-counting Topological motif census Classical, label-agnostic census (Milo-style) over the host graph's underlying undirected simple graph: open wedges (2-paths), triangles (closed triads, via common-neighbor counting), and directed 3-cycles (checked against the original directed edge set). crates/eg-compute/src/mining/subgraph.rs::count_motifs.
CONCEPT:EG-KG.mining.subgraph-writeback :FrequentSubgraph KG write-back subgraph with writeback=true (gspan only — motif is a pure census, the flag is ignored) materializes each frequent pattern as a typed :FrequentSubgraph node (nodes/edges/support/count, deterministic id from its canonical shape) linked (SUBGRAPH_MEMBER) to every resident host node appearing in any of its embeddings — the KG discovering recurring structure about ITSELF. WAL-replayed deterministically. src/server/handlers/mining.rs::materialize_subgraphs.
CONCEPT:AU-ORCH.execution.typed-failure-classification Compiled Orchestration Kernel A fast, deterministic core designed to resolve multi-agent dependency loops and order pipeline executions.
CONCEPT:EG-KG.compute.compiled-semantic-reasoner Compiled Semantic Reasoner Ultra-fast native-compiled Datalog OWL forward chaining reasoning engine.
CONCEPT:EG-KG.compute.reasoning-closure-gpu Semi-naive reasoning + GPU transitive-closure seam The EG-KG.compute.compiled-semantic-reasoner fixpoint re-architected for a GPU offload. The five OWL/RDFS rules are evaluated SEMI-NAIVELY over integer-interned relations (each round derives only from the prior round's delta, not a full re-scan of the accumulated facts) in crates/eg-compute/src/reasoning_closure.rs; the one sparse-matrix-shaped rule — Rule 5, transitive closure — is the boolean-semiring join {(x,z) | (x,y)∈A, (y,z)∈B} factored behind a ClosureBackend trait. The always-compiled CpuBackend (hash-join on the middle key) is the ground truth; cuda::CudaBackend (feature gpu-cuda) NVRTC-compiles a two-pass CSR-join kernel (binary-search count → host exclusive-scan → scatter) and runs it on a device, degrading to CPU on any device/compile/launch failure. Correctness: infer_semi_naive derives the SAME fact set as the prior naive fixpoint (differential oracle test over randomized ontologies), and the CUDA join matches the CPU join pair-for-pair (a #[cfg(feature = "gpu-cuda")] parity test that SKIPs on a GPU-less host and auto-validates on the GB10). Mirrors the EG-KG.compute.gpu-distance-seam/EG-KG.backend.real-cuda-tensor-backend pattern; gpu/gpu-cuda are OUT of pi/default/full (a pi build links no cudarc).
CONCEPT:AU-KG.retrieval.evidence-weighted-memory High-Performance Quant Epistemic-Graph Engine Native-compiled quantitative metrics computation, portfolio optimization, regime detection, and order matching simulation engine (replacing Python numpy/scipy).
CONCEPT:EG-KG.query.wire-protocol Tokio Service Layer High-performance Tokio async service exposing RPC endpoints over UDS/TCP for inter-agent communication.
CONCEPT:EG-KG.query.plan-optimize-seam Logical-plan optimization seam (Lane 0) The single hook eg_plan::plan_optimize(plan, ctx) -> Plan where a cost-based rewrite transforms a logical Plan into a cheaper-but-equivalent one before execution. In the foundation tier it is an IDENTITY passthrough (so execute is byte-for-byte the prior fold); Lane A's cross-modal cost optimizer fills the body WITHOUT re-touching the per-op arms or execute. crates/eg-plan/src/exec.rs.
CONCEPT:EG-KG.query.exec-driver-seam Physical execution-driver seam (Lane 0) The eg_plan::Driver trait (run(ops, ctx) -> RowSet) that a plan's op list runs THROUGH, so the scheduling/materialization strategy is swappable without touching the per-op arms (apply) or execute. SerialDriver reproduces today's sequential fold exactly; Lane B swaps in a parallel (rayon-morsel, memory-accounted, spilling) driver behind the SAME trait. crates/eg-plan/src/exec.rs.
CONCEPT:EG-KG.query.cost-cardinality-traits Shared cost/cardinality traits (Lane 0) The eg_plan::Cardinality per-op estimator trait (rows_out(op, in_card, ctx) -> f64) and the CostEstimate { rows, cpu, io } triple that Lane A's optimizer and Lane B's runtime both read to size/budget an op. Kept in eg-plan so cost metadata NEVER leaks into the wire Op DTO (Rule R1); this tier only defines the shape (no estimator wired), leaving Stats/CostModel/reorder_filter_rank intact. crates/eg-plan/src/cost.rs.
CONCEPT:EG-KG.query.exec-arm-dispatch Per-op exec dispatch table (Lane 0) The behavior-identical extraction of the remaining inline apply() match arms (Traverse/Rank/RankEmbed) into per-op *_op fns, so apply becomes a thin dispatch table over one function per op (matching the pre-existing filter_op/reason_op/foreign_scan/tsdb_scan_op shape) — de-conflicting the hot exec.rs file so Lanes A/B build in parallel. crates/eg-plan/src/exec.rs.
CONCEPT:EG-KG.query.parallel-runtime Physical parallel runtime (Lane B) The physical execution runtime hung off the Lane-0 Driver seam: eg_plan::runtime::execute_ops (the ONE driver-dispatch execute routes through) + the opt-in (par-runtime feature) rayon-morsel ParallelDriver. The driver OWNS scheduling / morsel batching, parallelism (a rayon pool sized from EPISTEMIC_GRAPH_EXEC_THREADS), memory accounting + spill-to-disk of oversized intermediates (EPISTEMIC_GRAPH_EXEC_SPILL_MB), and result materialization; the per-op physical fns stay modality-specific and unchanged. A linear plan's ops are data-dependent, so parallelism is intra-op — morsel-split the input of a row-parallel op (conservatively only AsOf), run the morsels on the pool, concat in morsel order — hence byte-for-byte the serial fold (proved by serial_and_parallel_drivers_agree + the differential oracle). Default (and any non-par-runtime build) dispatches to the Lane-0 SerialDriver. crates/eg-plan/src/runtime.rs.
CONCEPT:EG-KG.query.driver-modality-fn Modality availability in the physical fn (Lane B) Lane B's rule that a modality's feature-availability lives in the PHYSICAL fn, not as a #[cfg(not(feature=…))] fallback arm in the apply() driver loop. The Op::Window/Op::WindowAgg arms (whose variants exist unconditionally in the wire enum) now carry ONE unconditional arm each, delegating to window_op/window_agg_op which are defined for BOTH feature states — the native eg-tsdb windowed aggregate under timeseries, a byte-for-byte pass-through without it. Keeps the driver loop free of per-modality #[cfg] branches (mirrors the pre-existing foreign_named shape). crates/eg-plan/src/exec.rs.
CONCEPT:EG-KG.storage.invalidate-indexes-carve Index-invalidation carve on mark_dirty (Lane 0) Behavior-identical extraction of GraphCore::mark_dirty's lazy-secondary-index invalidation block (label / property / JSONPath maps) into fn invalidate_indexes(&self), preserving the exact prior ordering (index drop before the CDC changes.emit). Pre-splits the one C (matview result-cache) ↔ D (incremental per-WriteOp index maintenance) reconcile point: D owns the method body, C owns the adjacent cache line. crates/eg-core/src/graph.rs.
CONCEPT:EG-KG.query.xmodal-cost-optimizer Cross-modal cost-based optimizer (Lane A) The rule engine over the logical Vec<Op> that exec::plan_optimize drives (eg_plan::optimizer::optimize), reordering operators ACROSS modalities (relational / vector / graph / OWL / time) into a cheaper-but-equivalent plan — the decision a caller stitching three siloed surfaces cannot make. Active under query (beside the executor); runtime kill-switch EPISTEMIC_GRAPH_COST_OPT=0 → identity passthrough; the cost-opt cargo feature is the facade tier selector (implies query, default-on in full). Answer-preserving within the EG-405 boundary (adjacent narrower-vs-Rank only, input from a source, both intermediates ≥ 1 row). crates/eg-plan/src/optimizer.rs.
CONCEPT:EG-KG.query.cardinality-estimators Per-modality plan-time cardinality/cost estimators (Lane A) ModalityCardinality (impl Cardinality) over an O(1) PlanStats catalog (node / edge / embedding counts + average out-degree, derived from .len() — NEVER a per-node blob scan, so optimizing costs O(1) not O(N)): graph Traverse degree-histogram × path-length, ANN Rank recall@k / over-fetch vs the embedding-store size, OWL Reason inferred-closure membership × confidence retention, bi-temporal AsOf range selectivity, relational Filter per-predicate-product; cost_of → the CostEstimate { rows, cpu, io } triple. Kept in eg-plan so no cost metadata leaks into the wire Op (Rule R1). crates/eg-plan/src/cost.rs.
CONCEPT:EG-KG.query.filter-pushdown-rule Push selective Filter/AsOf before Rank (Lane A) The optimizer rule FilterAsOfBeforeRank: push a SELECTIVE relational Filter or bi-temporal AsOf ahead of an adjacent expensive vector Rank when the cost model says filter-first wins; keep Rank first for a BROAD narrower (vector-first). FOLDS the legacy CostModel::reorder_filter_rank into the engine as one rule, sharing its CostModel::order decision and the extracted place_narrower swap primitive (No-Legacy — one reorder implementation). crates/eg-plan/src/optimizer.rs + cost.rs.
CONCEPT:EG-KG.query.reason-rank-reorder-rule Order Reason vs Rank by candidate-set change (Lane A) The optimizer rule ReorderReasonRank (owl-gated): order an adjacent mid-pipeline OWL Reason (a confidence-preserving FILTER over the candidate set) vs a vector Rank by which shrinks the set more, so the second leg runs over fewer rows. Only a mid-pipeline Reason (input from a preceding source, index ≥ 1) is a narrower — a LEADING Reason is a SOURCE and is never reordered. Same adjacency + EG-405 non-empty guard as the filter-pushdown rule. crates/eg-plan/src/optimizer.rs.
CONCEPT:EG-KG.query.fuse-rrf-branch-reorder-rule Reorder FuseRrf branches by branch cost (Lane A) The optimizer rule ReorderFuseBranches (text-gated): reorder the sub-plan branches of a FuseRrf op by ascending branch cost (the sum of each op's CostEstimate::weight). Reciprocal-rank fusion accumulates into a map and sorts by (score desc, id), so the fused result is INDEPENDENT of branch order — the reorder is byte-for-byte answer-preserving and only runs the cheaper branch first. Recurses into nested FuseRrf. crates/eg-plan/src/optimizer.rs.
CONCEPT:EG-KG.query.global-plan-cost Global N-ary chain reorder (Track H, Lane A) ModalityCardinality::permutation_cost + the optimizer rule GlobalChainCost: generalizes the pairwise (narrower, Rank) swap to the WHOLE chain — a maximal run of 3+ consecutive Filter/AsOf/Reason/Rank(Embed) ops (drawing candidates from the SAME id-set) is exhaustively costed over every permutation (bounded MAX_CHAIN_SEGMENT = 5, so ≤ 120 orderings) and rewritten to the cheapest EG-405-safe one. Closes the gap the pairwise rules cannot: they advance by non-overlapping adjacent pairs and can permanently strand a later op behind an already-decided pair (e.g. [Rank, broad_filter, selective_filter] never re-examines selective_filter). A Rank first in a candidate permutation is costed via the SAME CostModel::vector_first_cost the pairwise decision uses, generalized to over-fetch against the COMBINED selectivity of every narrower still ahead. Runs LAST in the rule pipeline so it always has final say; a length-2 run is left to the pairwise rules (already optimal there). crates/eg-plan/src/cost.rs + optimizer.rs.
CONCEPT:EG-KG.query.adaptive-reoptimization Adaptive re-optimization hook (Track H, Lane A) optimizer::reoptimize_remaining(remaining_ops, estimated, actual, card, ctx): a runtime feedback loop beyond the static, plan-time-only cost decision — when an ACTUAL observed cardinality diverges from what optimize() assumed by more than ADAPTIVE_REOPT_THRESHOLD (50% relative), re-costs the not-yet-executed tail's leading reorderable run from the corrected seed via the SAME best_permutation exhaustive search GlobalChainCost uses, and returns a possibly re-ordered tail; below the threshold it is a no-op clone (bounded, churn-free in the common case). CUT: this is caller-driven (a test / future Driver impl invokes it explicitly with an observed RowSet::len()) — wiring it automatically into crate::exec::execute's per-op loop (every apply() reporting its real output length and triggering re-costing) is the documented follow-up, not yet done. crates/eg-plan/src/optimizer.rs.
CONCEPT:EG-KG.query.rls-scoped-result-cache RLS-aware result-cache key (Track C) The version-keyed result-cache Key extended from (query_hash, version) to (query_hash, version, actor_scope_hash) so a cached plan RESULT is NEVER served across RLS actors. actor_scope_hash = 0 is the security-off / single-tenant case (the plain get/put path — byte-identical to before); get_scoped/put_scoped + hash_actor_scope(agent_id) add the actor dimension. A system-scoped (0) entry is unreachable to a nonzero-scope lookup, so an unfiltered result never leaks to a filtered actor. crates/eg-core/src/result_cache.rs.
CONCEPT:EG-KG.storage.plan-backed-matview Plan-backed materialized views (Track C) GENERALIZES the algo-only distributed-compute matview into a NAMED, DURABLE wire::Plan over one graph (PlanMatViewDefine/Get/Refresh/Drop Methods, feature matviewcompute-dist/query/result-cache/streaming). Define executes the plan once via the eg-plan execute runtime seam and caches the RESULT in the version-keyed, RLS-aware result cache; the definition persists to a DISJOINT plan_matviews redb table (disjoint from the compute-dist matviews table AND Lane D's index rows). src/server/matview/ + handlers/dist_compute.rs + redb_store.rs/redb_backend.rs + protocol.rs.
CONCEPT:EG-KG.storage.matview-cdc-invalidation Matview CDC invalidation (Track C) The plan-backed matview manager subscribes to CdcHub::emit (fires per committed change): note_change(graph) marks every view over that graph stale so the next Get recomputes. Reuses the (query_hash, version) result-cache discipline — a committed write already bumped the graph OCC version (retiring the cached bytes) — as the belt-and-braces manager-side signal. src/server/cdc.rs + src/server/matview/manager.rs.
CONCEPT:EG-KG.query.symmetric-foreign-scan Symmetric ForeignScan (Track C) federation::foreign_source_rows(spec, registry) is the SINGLE leaf-source seam (the foreign analogue of scan_label for Op::Scan) through which EVERY spec kind (Named-via-registry / remote-engine / HTTP-JSON / SQL) resolves to the SAME RowSet currency, so the exec.rs foreign_scan arm is a thin resolve+fuse mirror of the Op::Scan arm and a ForeignScan leaf composes through the Driver seam byte-identically to an internal Scan leaf (a compose-oracle proves ForeignScan == Scan for matching rows). crates/eg-plan/src/federation.rs + the foreign_scan fn in crates/eg-plan/src/exec.rs.
CONCEPT:EG-KG.txn.per-graph-write-isolation Lock-Free Compute + Engine Observability Heavy read-only algorithms compute on structural snapshots via the blocking pool (writers never starved by analytics), plus Prometheus metrics (per-op rate/latency, admission, per-graph size, checkpoint, auth/ACL counters) on a --metrics-addr HTTP listener.
CONCEPT:EG-ORCH.routing.lexical-capability-escalation Ontology Lexical Classification Gate Embedding-free aho-corasick match of a query against capability-node names+synonyms (Tool/Skill/MCPServer/…), cached per node-count. The "free" (~µs) tier of chat-turn classification: a turn naming a real fleet capability escalates to the full graph without a vector search. Method::MatchOntologyTerms (read-only).
CONCEPT:EG-KG.backend.adaptive-linger-coalesce Adaptive group-commit micro-linger When the eg-redb-writer is about to commit a SHALLOW barrier batch it spends ONE bounded recv_timeout(linger) (default 1 ms, env EPISTEMIC_GRAPH_REDB_GROUP_LINGER_US; 0 disables = legacy commit-on-drain) letting concurrent in-flight authoritative writers fold into the SAME group-commit fsync — the profiled write ceiling was ~1 op/fsync from serial awaits. Adaptive: it lingers only while pending.ops.len() is below EPISTEMIC_GRAPH_REDB_GROUP_SHALLOW (default 32, clamped 1..4096); a deeper batch already coalesces and commits immediately, so it adds NO latency on a deep queue. Durability is unchanged — writes still commit Durability::Immediate before their ack (CONCEPT:EG-KG.backend.authoritative-dispatch), only the batch widens. src/server/persistence/redb_backend.rs (commit 6f601b0).
CONCEPT:EG-KG.storage.embedded-store O(1) audit-chain tail cache The EG-KG.sharding.row-level-security hash-chained tamper-evident AUDIT log now appends in O(1): the per-graph chain tail (seq + previous hash) is cached in-process and advanced on each append instead of re-reading the tail row from redb per mutation, so a high-rate ingestion stream no longer pays a durable read on every audited write. Folds into the EG-024 group-commit so the audit append rides the same fsync as its graph mutation. src/server/persistence/redb_backend.rs (commit 8873fbf).
CONCEPT:EG-KG.storage.write-changeset Write-path change-set plumbing (Track D1) The per-graph write coalescer (src/write_coalescer.rs::apply_batch) collects each committed batch's successful per-WriteOp deltas (node/edge add·update·remove) into an eg-core index::ChangeSet inside the single topology lock, then hands it to GraphCore::maintain_indexes, which routes it through the IndexManager so the heavy secondary indexes move together with the topology mutation — a concurrent hybrid read never observes a torn index. Gated by the process-cached EPISTEMIC_GRAPH_INCREMENTAL_INDEX (default ON); the kill-switch 0/false/off/no skips the seam entirely, so the dispatch shell's per-op mark_dirty performs the legacy invalidate-and-rebuild byte-identically. Zero hot-path cost when off (no id clones, no change-set).
CONCEPT:EG-KG.storage.index-manager-seam Unified secondary-index write seam (Track D1) The one SecondaryIndex/IndexManager registry (crates/eg-core/src/index.rs) gains a WRITE side to match its read side: SecondaryIndex::apply_delta(core, change) + full_rebuild(core) (both defaulted — apply_delta returns DeltaUnsupported, so the lazy label/property/ontology descriptors keep their exact rebuild-on-read semantics), and IndexManager::commit_batch (try each index's apply_delta, fall back to full_rebuild) + rebuild_all, returning a BatchMaintenance tally. GraphCore::invalidate_indexes() is carved out of mark_dirty (behavior-identical; the one C↔D reconcile point — RECONCILE-WITH-LANE-0, whose carve owns the same body).
CONCEPT:EG-KG.storage.incremental-ann Incremental ANN upsert/delete (Track D2) SemanticStore::remove_embedding on BOTH backends removes a node's vector with NO full rebuild: the semantic_hnsw default drops it from the map and tombstones its internal id (deferred compaction past the ratio); the semantic_store_ann (eg-ann IVF-PQ, active under ann/full) swap-removes the row from the dense EmbeddingArena in O(dim) (keeping data.len() == ids.len()·dim) and tombstones the IVF-PQ row via AnnIndex::removeIvfPq::delete. VectorIndexDescriptor::apply_delta tombstones the embeddings of removed nodes so kNN never returns a dead node (adds/updates still ride the already-incremental TxnAddEmbedding path). Proven incremental ≡ full-rebuild and concurrency-coherent (never-torn) by the seam + coalescer tests.
CONCEPT:EG-KG.storage.incremental-text Incremental text-index maintenance (Track D3) The eg-text Tantivy TextIndex lives in the server layer (ABOVE eg-core in the DAG), so GraphCore::maintain_indexes can't reach it. GraphTextIndex (src/server/secondary_indexes.rs) wraps it as a SecondaryIndex and is registered per-graph into the IndexManager by ServerIndexFactory (via GraphCore::register_index + GraphRegistry::set_secondary_index_factory, mirroring the read-through factory). Its needs_content() flips the coalescer's wants_change_content flag so apply_batch captures each added/updated node's property blob into the ChangeSet (zero-clone when no content index is registered). apply_delta derives the document body from the captured blob (a conventional text field) and upsert/delete+commits Tantivy incrementally — reading ONLY the blob, never core (which would re-enter the held topology lock and deadlock); full_rebuild (out-of-lock only) re-indexes every live node. incremental ≡ rebuild proven on the ranked id list (BM25 scores legitimately differ pre-merge between a tombstoned-incremental and a fresh index; the result SET + ranking are identical) + concurrency-coherent (never-torn) under the coalescer.
CONCEPT:EG-KG.storage.incremental-temporal Incremental temporal-index maintenance (Track D3) The eg-tsdb SeriesStore lives in server state, so GraphTemporalIndex (src/server/secondary_indexes.rs) wraps it as a SecondaryIndex registered per-graph by ServerIndexFactory over the SAME shared store the Ts* handlers use. apply_delta extracts a node's measurements from the captured blob and idempotently REPLACES the node's per-node series (SeriesStore::delete_series then append_batch — the new delete_series primitive drops every chunk + the meta row), and drops the series on a node removal. incremental ≡ rebuild proven (a delete-then-append sequence == a rebuild appending only survivors) + full_rebuild (out-of-lock) rebuilds from live nodes.
CONCEPT:EG-KG.storage.incremental-derived-owl Derived-OWL index wired as a no-op deferring to the reasoner (Track D3) The eg-rdf OWL reasoner (Reasoner::add_axioms, CONCEPT:EG-KG.ontology.incremental-materialization) is ALREADY a differential materializer invoked ON-DEMAND at query time. DerivedOwlIndex (src/server/secondary_indexes.rs) registers it into the seam so derived-OWL is a first-class member of the one registry (discoverable, counted in the batch tally), but its apply_delta/full_rebuild are documented NO-OPS — re-running materialization on the write batch would duplicate the reasoner's own incremental pass. The incremental ≡ rebuild equivalence of that materializer is proven directly against Reasoner in eg-rdf's own suite (owl::tests::incremental_add_axiom_only_adds).
CONCEPT:EG-KG.backend.sharded-k-way-durable Sharded K-way durable writer redb is single-writer-PER-FILE, so the durable writer shards by graph into K independent graph-<n>.redb files, each with its OWN writer thread / bounded channel / Pending — so the EG-024 micro-linger, the EG-KG.storage.embedded-store audit-tail cache, commit-before-ack, group-commit and backpressure all hold PER shard and K cores commit in parallel. A graph ALWAYS routes to the same shard via FNV-1a(sanitized_name) % K (a txn stays within one graph = the group boundary, so per-shard atomicity holds; no durable commit spans shards). K = clamp(cpu/2, 1, 8) (env EPISTEMIC_GRAPH_REDB_SHARDS, clamped 1..64), FIXED per persist-dir once created (the on-disk layout is detected + honored at open via reconcile_shard_layout). K=1 is byte-for-byte the old single-graph.redb path; forced to 1 under an active Raft node (multi-Raft sharding is M2). src/server/persistence/redb_backend.rs (commit b0c3755). See engine.md.
CONCEPT:EG-KG.storage.snapshot-read-off-writer Snapshot/MVCC reads off the writer redb 4.1 is MVCC, so the point-read / read-through path (read_node, hit only on a RAM miss — the EG-KG.storage.read-through-seam-exercised eviction read-through) serves the evicted node DIRECTLY off a Database::begin_read() snapshot on the target shard's shared Database (same EG-KG.backend.sharded-k-way-durable shard_for), running CONCURRENTLY with the single writer — a read NEVER routes through the writer thread's channel and NEVER forces a group-commit (previously a read-through was sent as Cmd::ReadNode over the writer's blocking channel and forced a Durability::Immediate commit first). Consistency model: reads see the latest COMMITTED state per shard; commit-before-ack (KG-2.187) guarantees any ACKED write is already committed and therefore visible to a later begin_read(). The writer owns the SOLE strong Arc<Database> per shard; the read path holds a Weak and upgrade()s it per read (redb's exclusive per-process file lock means the handle is SHARED, not re-opened, and the lock lifetime is unchanged). src/server/persistence/redb_backend.rs (commit 59f7096). See engine.md.
CONCEPT:AU-KG.backend.b-auto-size Dynamic capacity auto-sizing + Pi-OOM correctness The SAME server binary sizes its concurrency / buffer / per-graph-node-cap DEFAULTS from (cpu_count, total_RAM) at startup (src/autosize.rs detect_capacity()Capacity { cpus, total_ram_bytes, tier ∈ {Pi,Node,BigBox} }, via available_parallelism() + /proc/meminfo — pure-Rust, no deps), mirroring CoalescerConfig::auto. EPISTEMIC_GRAPH_MAX_INFLIGHT default = cpus·64 clamped [256, 8192] (Pi 256, 64-core 4096 vs the old fixed 1024); EPISTEMIC_GRAPH_WAL_QUEUE default = cpus·256 clamped [1024, 65 536]. Pi-OOM fix: EPISTEMIC_GRAPH_MAX_NODES_PER_GRAPH default was 0 (unbounded → a 1 GiB Pi OOMs); now default_node_cap(ram) = (ram/2)/2048 clamped [50 000, 100 000 000] — a 1 GiB Pi caps a runaway graph to ~262 k resident nodes while a 247 GiB box stays effectively unbounded (100 M ceiling). Safe because eviction is read-through (CONCEPT:EG-KG.storage.read-through-seam-exercised): nodes past the cap still serve from the durable redb tier — bounds RESIDENT RAM with ZERO data loss, making enforce_memory_budgets/evict_oversized_all actually bound memory by default. Every value stays env-overridable; explicit 0 node-cap is the "truly unbounded" opt-out. Also trims the per-mutation ledger hex encode (crates/eg-core/src/graph.rs): a HexLedger Display adapter streams hex straight into the format! buffer (was a double-allocated 2·N hex String under the topo.write() guard), byte-identical on-disk format.
CONCEPT:EG-KG.sharding.atomic-shard-swap Offline K-shard migration tool OFFLINE tool that rewrites an existing durable redb store (graph.redb for K=1, or a graph-<n>.redb set) into a NEW shard count K, routing every graph with the SAME EG-KG.backend.sharded-k-way-durable FNV-1a(name) % K so the running engine finds each graph in the shard it looks for. Copies every durable row VERBATIM — per-graph (NODES/EDGES/LEDGER/SEMANTIC/GRAPH_META) and the tamper-evident hash-chained AUDIT log (EG-KG.sharding.row-level-security) — so encryption-at-rest blobs survive without the key and the audit chain stays verifiable; global shard-0 records (Raft log/meta, cross-shard 2PC, matviews) re-home to the new shard 0. src/server/persistence/shard_migrate.rs + migrate-shards CLI (src/bin/migrate_shards.rs); in-place mode leaves a recoverable timestamped backup. Building block for online per-tenant resharding (EG-031). See m3_resharding.md.
CONCEPT:EG-KG.sharding.empty-catalog-routing Tenant catalog routing-override seam Durable, rebalanceable graph/tenant → {shard, node} map (src/server/persistence/tenant_catalog.rs, optional catalog.redb) that OVERRIDES EG-KG.backend.sharded-k-way-durable hash routing per graph. TenantCatalog::resolve_shard returns an explicit assignment if one exists, else falls back to the exact EG-KG.backend.sharded-k-way-durable shard_index — so an EMPTY catalog is byte-for-byte identical to no catalog (pure FNV-1a) and never destabilizes the running engine. Attached to RedbBackend via a builder seam (with_catalog); None is the default. Core lookup/assign/reassign/remove + write-through durability only — the actual data movement of a reassignment is remaining ONLINE M3 work. See m3_resharding.md.
CONCEPT:EG-KG.backend.catalog-shard-resolve Online single-node per-tenant resharding (R1) Move ONE graph between shards while the engine RUNS, then flip the EG-031 catalog route (src/server/persistence/online_reshard.rs + RedbBackend::reshard_graph). The graph's rows are copied VERBATIM (raw blobs — encryption-at-rest + the EG-KG.sharding.row-level-security hash-chained AUDIT log preserved, exactly as EG-030's offline copy) source→dest via new Cmd::ExportGraphRaw/ImportGraphRaw on the two shard writer threads, then import(dst) durably committed → catalog flip durable → purge(src) (crash-consistent ordering). A backend routing_epoch RwLock quiesces ONLY the catalog-attached durable writes for the duration of the flip (shared READ on every record_durable/commit_crossmodal, exclusive WRITE held across the move) so no write is lost or misrouted across the flip; other graphs + the steady no-catalog path are untouched. Proven by online_reshard_moves_graph_live_no_loss (20 nodes incl. concurrent writes across the flip, edge, audit chain, other graph intact). See m3_resharding.md.
CONCEPT:EG-KG.sharding.r5-feature Catalog auto-attach gate (R5) RedbBackend::open attaches the EG-031 tenant catalog to the LIVE routing seam when EPISTEMIC_GRAPH_TENANT_CATALOG=1 OR a durable catalog.redb already exists (maybe_attach_catalog_from_env); otherwise NO catalog is attached and routing is byte-for-byte EG-KG.backend.sharded-k-way-durable FNV-1a (verified by catalog_auto_attach_gate_and_empty_is_fnv1a). An attached-but-empty catalog also routes identically, so turning the flag on is a no-op until an online reshard (EG-032) assigns a placement. RedbBackend::catalog() exposes the catalog so the admin/API can populate/persist placements (assign/reassign/remove, durable). The explicit-K test constructor open_with_shards never auto-attaches. See m3_resharding.md.
CONCEPT:EG-KG.sharding.eg-r6 Cold-tenant whole-graph offload (R6) Time-windowed proactive offload of IDLE tenants to bound RAM across many tenants (src/server/persistence/cold_offload.rs). A ColdTenantTracker records per-graph last-access (touch) and cold_graphs(window) selects graphs idle longer than a window; offload_cold_tenants hibernates each (drops its in-RAM topology/props/vectors via GraphCore::hibernate, EG-KG.storage.100m-tenant) while the durable redb rows are retained — so reads serve via the EG-KG.storage.read-through-seam-exercised node read-through and rehydrate on access. Durability-gated (under authoritative mode every acked write is already committed, so dropping the core is loss-free); __commons__ is never offloaded. Complements the EG-KG.compute.lane-v budget-pressure enforcer (this is idle-driven, that is budget-driven). Proven by cold_offload_evicts_then_serves_on_access. See m3_resharding.md.
CONCEPT:EG-KG.sharding.even-load-rebalance Rebalancing planner (M3 R3) PURE, deterministic policy layer (src/server/persistence/rebalance.rs) that reads observable per-shard/per-graph load + the EG-031 tenant catalog and EMITS a RebalancePlan — an ordered Vec<ReshardMove { graph, from_shard, to_shard }> — to even out load across the K durable shards. Greedy hottest→coldest: each step moves the ONE graph that most shrinks the hottest-coldest gap without overshooting (new gap \|gap − 2·load\|), stopping at tolerance·mean, on non-improvement (an indivisible hot graph ⇒ no thrash), or max_moves. shard_loads_from_catalog/shard_loads_from_graph_loads assemble the input by EG-031 routing. It does NOT execute the plan — applying a move (snapshot-copy rows + catalog.reassign) is R1 online-reshard (separate). Same inputs ⇒ same plan; unit-tested against synthetic skew. See m3_resharding.md.
CONCEPT:EG-KG.sharding.m3-r4 In-process BLOB streaming facade (M3 R4) Bounded-memory in-process twin of the KG-2.206 wire upload/fetch cursor (src/server/blob/stream.rs): stream_blob_put<R: Read> / stream_blob_get<W: Write> stream a multi-GB blob between an arbitrary byte source/sink (a local media file, a decompressor, an embedding writer) and the content-addressed CAS WITHOUT buffering the whole blob — one chunk resident at a time. Composes the existing ChunkStore verbatim, so it is the same dedup'd / refcount-GC'd store, just driven by a Read/Write instead of the protocol cursor. Independent modality (never touches the graph write path); used by file import/export and the cold-tier object-store offload (R6). Round-trip + bounded-RSS tested over a 256 MB blob (EG_BLOB_RSS_MB runs 1 GB+). See m3_resharding.md.
CONCEPT:EG-KG.backend.multiplexed-connections Pooled/multiplexed engine connections (roadmap E) Client-side connection multiplexing with NO server change: the Python ConnectionPool / ShardRouter (epistemic_graph/pool.py) auto-size to the box (2*cpu clamped 8..64 — no knob) and fan INDEPENDENT ops across N pooled connections, because the engine tokio::spawns one task per connection (N connections = N parallel server tasks). pool.connection() / router.connection(graph) are RAII acquire/release (the hot path can't leak a connection and starve the cap); pool.map_concurrent(ops) / router.map_concurrent(graph, ops) collapse the serial sum to ~one op, results in input order. Ordering preserved — a single logical write (node before its edges) runs as ONE fn on ONE connection; only independent ops spread across connections. The in-CONNECTION concurrency story it worked around is later removed by EG-043 single-connection request pipelining, which it now COMPOSES with (the pool multiplexes connections AND each connection multiplexes requests). Commit aff2426; tests/test_mux_pool.py.
CONCEPT:AU-KG.backend.roadmap-f-parallel-cross Parallel cross-shard read fan-out (M3 roadmap F) The durable cross-shard read path (RedbBackend::load_all/load_into) now fans each shard's dump CONCURRENTLY off its OWN begin_read() MVCC snapshot (CONCEPT:EG-KG.storage.snapshot-read-off-writer) on the blocking pool — via join_blocking_in_order, which spawns EVERY per-shard task before awaiting any — instead of routing each shard's Cmd::Load SERIALLY through its writer thread. The K reads run in parallel on K cores and NEVER touch a writer thread (the EG-KG.storage.snapshot-read-off-writer invariant: a read never forces a group-commit nor serializes behind a write). Consistency is unchanged: each MVCC snapshot sees its shard's latest COMMITTED state, and commit-before-ack (KG-2.187) guarantees any acked write is already committed/visible. Proven concurrent (not serial) by a Barrier(K) fan-out test + a multi-shard parallel-load recovery test. See m3_resharding.md.
CONCEPT:EG-KG.backend.m3-admin-dispatch M3 catalog-driven resharding admin RPC The wire surface that DRIVES the M3 ops over the protocol (Method::Reshard/CatalogAssign/CatalogReassign/CatalogRemove/CatalogList/RebalancePlan/RebalanceExecute, src/server/handlers/admin.rs, epistemic_graph/client.py ReshardingClient). Reshard calls the existing RedbBackend::reshard_graph (EG-032); the Catalog* ops drive RedbBackend::catalog() (EG-031 assign/reassign/remove/entries); RebalancePlan runs rebalance::plan_rebalance (EG-035) over LIVE per-graph load (resident node counts routed through the catalog) and RETURNS the plan. These CALL the existing persistence APIs — they do not reimplement EG-031/032/035. Redb-only; a non-redb build returns a clean "not available" error. See m3_resharding.md.
CONCEPT:EG-KG.backend.r3-plan-execution Rebalance plan execution (M3 R3) RedbBackend::rebalance_execute(plan) applies an EG-035 RebalancePlan move-by-move via online resharding (EG-032 reshard_graph) — online, ONE graph at a time, every other graph unaffected. Each move resolves its source from the catalog's CURRENT state (the plan's from_shard is informational), so applying moves in order is robust to placement drift. The RebalanceExecute admin RPC (EG-038) assembles the live plan and runs it. Proven by rebalance_execute_balances_and_preserves_data (a fully-skewed K=4 set → plan → execute → graphs spread across >1 shard, every node survives). See m3_resharding.md.
CONCEPT:EG-KG.backend.r6-feature Cold-tenant touch() wiring + idle offload sweep (M3 R6) Wires the EG-KG.sharding.eg-r6 ColdTenantTracker into the LIVE path: a cold_tracker on ServerState is touched on every graph access in dispatch_graph_op (reads + writes) and forgetten on DeleteGraph, so access recency is tracked off the hot path; a main.rs interval task (reusing the engine's existing background-task cadence — NO new daemon, armed by EPISTEMIC_GRAPH_COLD_OFFLOAD_SECS) periodically calls offload_cold_tenants(window) to hibernate graphs idle longer than the window. Recently-touched graphs are never selected; idle ones offload (durably-gated, read-through-safe — no loss). Redb-only. Proven by touch_keeps_accessed_graph_resident_idle_is_cold. See m3_resharding.md.
CONCEPT:EG-KG.backend.flush-pending-first Online-reshard snapshot+delta copy (M3 R1 refinement) Shrinks the moved graph's write-pause in reshard_graph (EG-032) from O(graph) to O(delta). PHASE 1 (bulk_copy, NO exclusive quiesce) copies the bulk verbatim off a src begin_read() snapshot while writes keep flowing to src; PHASE 2 (delta_flip_purge, under the exclusive routing_epoch WRITE guard) re-exports the latest src state, compute_deltas it against the bulk snapshot (upserts for new/changed rows + removals for rows deleted since the snapshot, so a deleted row can't resurrect on dst), imports ONLY that delta via Cmd::ImportGraphDelta, flips the catalog route, and GCs the source. Crash-consistent ordering (import bulk → import delta → flip → purge). The report's delta_nodes/delta_edges measure the under-quiesce work. Proven by online_reshard_delta_is_small_for_idle_graph (idle graph ⇒ delta 0, no loss) + the existing live-write no-loss reshard test. See m3_resharding.md.
CONCEPT:EG-KG.backend.framed-response True single-connection request pipelining Removes the per-connection wire SERIALIZATION that the EG-037 connection pool worked around: the engine's per-connection loop (src/server/transport.rs::handle_connection) tokio::io::splits the stream and, for each decoded frame, tokio::spawns a dispatch task whose id-tagged Response is written back through a SINGLE writer task fed by an mpsc channel — so many requests on ONE connection process CONCURRENTLY and responses return OUT OF ORDER (the read loop never blocks on dispatch). The write half is a lone writer task draining an mpsc::channel<Vec<u8>> of pre-framed bytes (chosen over Arc<Mutex<WriteHalf>>: a mutex held across a back-pressured socket write would serialize every completing task and add hot-path contention; the channel decouples concurrent response encoding from the single ordered socket write). A per-connection semaphore (per_connection_inflight_limit, auto-sized cpus*8 clamped 64..1024) bounds in-flight tasks — acquiring it AWAITS, so the read loop back-pressures instead of spawning unbounded work; the global max_in_flight semaphore stays the box-wide BUSY-shedding cap. The Python client (epistemic_graph/client.py) replaces the lock-across-write-drain-read _send with a background reader task per connection that resolves the matching pending future by Response.id; _send registers a future under the request id, writes the frame under a tiny write-only lock, and awaits only its own future — so per-caller ordering is automatic while INDEPENDENT concurrent calls pipeline. Composes with the EG-037 pool: the pool now multiplexes connections AND each connection multiplexes requests. Single-in-flight behaves byte-for-byte as before; a per-call timeout/EOF/transport error is connection-fatal (fails all in-flight, reconnects). Proven by tests/test_pipelining.py (deterministic mock: 8-in-flight wall-clock ≪ serial + out-of-order demux + per-caller order + one-error-doesn't-corrupt-siblings; real ephemeral engine: a cheap op overtakes an in-flight heavy op on ONE connection + N-concurrent correctness).
CONCEPT:EG-KG.coordination.reserved-read-lane Reserved read-admission lane (read responsiveness under write saturation) Guarantees an interactive MCP read/query is NEVER shed to BUSY behind an ingestion WRITE firehose. The transport admission (src/server/transport.rs::handle_connection) now classifies each request via requires_write(&req.method) and resolves it through a pure, unit-testable admit_request: both reads and writes try the NORMAL path first (a global max_in_flight permit AND this graph's per-graph fairness permit); a WRITE that loses it is shed BUSY (strictly back-pressured — the durable write path stays the bottleneck, never dropped); a READ that loses it FALLS BACK to a dedicated ServerState::read_admission semaphore (auto-sized Capacity::read_reserved = max_inflight/8 clamped 8..1024, env EPISTEMIC_GRAPH_READ_RESERVED) that writes can NEVER acquire, BYPASSING the per-graph cap — so even when the __commons__ firehose saturates both the global pool and that graph's cap, an MCP read keeps an open lane. Only a genuine read FLOOD that also fills the small reserved lane is shed BUSY, so memory stays bounded. Reads serve from MVCC snapshots (in-memory GraphCore snapshot for Cypher/SQL/GraphQL; begin_read() for the redb read-through, CONCEPT:EG-KG.storage.snapshot-read-off-writer) so a read never contends for any write lock — the engine's redb tier never returns "database is locked". New Prometheus counter epistemic_graph_read_reserved_admitted_total. Proven by read_is_admitted_when_write_pool_is_saturated, read_lane_is_bounded_under_a_read_flood, and reads_survive_under_max_write_load_on_hot_graph.
CONCEPT:EG-KG.query.compound-predicate-decode Serializable compound WHERE on DML UPDATE/DELETE on nodes + user tables accept compound predicates (AND/OR/NOT/IN/BETWEEN/ranges/IS NULL) via a RowPredicate decoded in eg-query/sql/classify.rs. The nodes path resolves the matched id-set through the DataFusion read path and applies each mutation through serializable compare_and_set_fields_if/remove_node_if gates (re-check the predicate against the node's current blob under the write guard); user tables evaluate the predicate inside the open redb WriteTransaction.
CONCEPT:EG-KG.query.insert-into-nodes-select INSERT INTO nodes … SELECT Populate the graph node store from a SELECT (which may JOIN user tables + the graph). Classified to InsertNodesSelect; the projected rows are inserted via the shared apply_node_insert_row helper with RETURNING support.
CONCEPT:EG-KG.query.update-delete-from Multi-table DML (UPDATE…FROM / DELETE…USING) Correlated multi-table DML on nodes: the matched ids (and per-row SET values) are resolved through DataFusion, then applied via the serializable CAS/remove gates.
CONCEPT:EG-KG.query.delete-returning-sees-row ON CONFLICT upsert + user-table RETURNING INSERT … ON CONFLICT (cols) DO NOTHING|DO UPDATE for nodes and user tables (reusing the unique/PK validation), and RETURNING on user-table INSERT/UPDATE/DELETE.
CONCEPT:EG-KG.compute.kg-transaction-is-pinned Mixed-store wire transactions + TransactionStatus pgwire BEGIN/COMMIT/ROLLBACK buffer graph-node ops (GraphTxnBuffer) AND user-table ops; reads inside the txn see buffered writes (read-your-own-writes overlay); COMMIT applies the node batch (one GraphCore::txn() + one durable group) then the user-table txn (best-effort ordered, documented non-2PC window); ReadyForQuery reports T/E/I and an aborted txn rejects until ROLLBACK (25P02).
CONCEPT:EG-KG.query.create-drop-view SQL CREATE VIEW / DROP VIEW Durable view catalog (mirror of the user-table catalog); a referenced view expands to its stored SELECT during build_ctx.
CONCEPT:EG-KG.ontology.content-negotiation-serializers SPARQL content negotiation Accept-aware output on /sparql: SPARQL-results JSON/XML/CSV/TSV for SELECT/ASK and Turtle/N-Triples for CONSTRUCT/DESCRIBE (hand-written serializers + oxttl TurtleSerializer), with a per-form default and an output=/format= override.
CONCEPT:EG-KG.ontology.sub-select SPARQL sub-SELECT Nested { SELECT … } evaluated by restricting each inner solution to the projected variables in the GraphPattern::Project arm so it joins correctly to the outer pattern.
CONCEPT:EG-KG.query.sparql-service-federation-client SPARQL SERVICE federation A RemoteSparql seam in eg-rdf + a ureq-backed client in the facade (feature sparql-service) evaluates a SERVICE <ep> { … } clause against a remote endpoint (SILENT semantics, SSRF allowlist EPISTEMIC_GRAPH_SPARQL_SERVICE_ALLOW, default fail-closed).
CONCEPT:EG-KG.ontology.rich-filter Rich SPARQL FILTER Extends the expression evaluator with arithmetic, REGEX, IN, STR/LANG/LANGMATCHES/DATATYPE/BOUND, isIRI/isLiteral/isBlank/isNumeric, string functions (CONTAINS/STRSTARTS/STRENDS/STRLEN/SUBSTR/UCASE/LCASE/CONCAT), COALESCE, IF, with datatype-aware comparison.
CONCEPT:EG-KG.ontology.from-from-named SPARQL FROM / FROM NAMED Honor the in-query dataset spec (parsed by spargebra) to scope the active Dataset, instead of always using the server registry.
CONCEPT:EG-KG.ontology.minus SPARQL MINUS GraphPattern::Minus set-difference arm (remove solutions compatible with the right pattern).
CONCEPT:EG-KG.ontology.negated-property-set SPARQL negated property set !p Evaluate PropertyPathExpression::NegatedPropertySet by scanning edges whose predicate is not in the negated set.
CONCEPT:EG-KG.ontology.capability-catalog SPARQL SPO/POS triple index + selectivity join-ordering Replace full-scan-per-pattern with hashed SPO/POS/PSO indexes over the snapshot + cardinality-based BGP reordering.
CONCEPT:EG-KG.compute.cdc-event-emit GraphQL subscriptions (CDC) A tokio::sync::broadcast change-stream fed by GraphCore mark_dirty + a WS/SSE carrier, replacing the poll-only stub.
CONCEPT:EG-KG.query.fragments-variables-directives GraphQL fragments / variables / directives Lexer $/@ tokens; fragment-spread + inline fragments; variable definitions/refs; @skip/@include.
CONCEPT:EG-KG.query.graphql-cursors GraphQL relay pagination first/after/before/cursor args + an edges/node/cursor/pageInfo envelope derived from the deterministic sort.
CONCEPT:EG-KG.ontology.owl-reasoning rdfs:range in EL completion Lift each ont.ranges entry into the EL closure (an ∃r⁻.⊤ ⊑ D analog) so range constrains concept-level classification, not only RL/instance typing.
CONCEPT:EG-KG.ontology.concept-2 OWL-DL tableau reasoner A non-monotone tableau reasoner (behind an owl-dl feature) alongside the EL⁺∪RL core in eg-rdf, adding constructs outside EL/RL: cardinality (min/max/exact + qualified), complementOf/negation, oneOf nominals, and unionOf reasoning-by-cases. Consistency check → classification → instance checking; the EL/RL fast path stays default and DL-requiring ontologies route to the tableau.
CONCEPT:EG-KG.ontology.concept-3 SWRL / RuleML rules + built-ins Extend the rules.rs Horn-rule DSL to parse SWRL/RuleML atoms and the core SWRL built-in library (comparison/math/string), keeping head-var range-safety.
CONCEPT:EG-KG.query.cypher-execution Cypher REMOVE WriteOp::Remove + parser branch (fixes the at_write_clause mis-route vs access.rs classifier) → apply_remove (property delete + label removal).
CONCEPT:EG-KG.query.eg-extend-read-side Cypher read clauses ORDER BY/SKIP/WITH/OPTIONAL MATCH/OR+IN/STARTS WITH/CONTAINS/IS NULL in WHERE, aggregation (count/collect/sum/avg/min/max), RETURN DISTINCT/* in the parser + executor.
CONCEPT:EG-KG.query.concept-2 Cypher var-length generalization A variable-length hop combined with surrounding fixed hops + path-variable binding (relaxes the single-hop guard).
CONCEPT:EG-KG.query.streaming-execution Op::Window execution Wire eg-tsdb window/aggregation functions behind the planner Op::Window (adds the eg-plan→eg-tsdb dependency edge + cost-based reorder), replacing the pass-through seam.
CONCEPT:EG-KG.temporal.bucket-cutoff-trim Per-point retention trim Trim straddling buckets in eg-tsdb evict_before instead of whole-bucket-only eviction.
CONCEPT:EG-KG.query.concept-4 Observability log search + query API SQL + full-text search over ingested obs streams (AU-KG.ingest.self-ingest): a DataFusion scan over the Parquet segments (EG-KG.retrieval.observability-search, pruned by the segment manifest's time/stream range) UNIONed with the hot eg-tsdb series + the eg-text BM25 index, plus an O2/ES-shaped /api/.../_search query endpoint on the obs listener. Gated obs. Depends on AU-KG.ingest.self-ingest, EG-161.
CONCEPT:AU-KG.ingest.self-ingest OpenObserve-style log ingestion A hand-rolled HTTP ingestion listener (OTLP/HTTP + Elastic-_bulk-compatible + syslog) in the facade (pgwire/sparql_http/metrics idiom, no axum/hyper), landing log records as time-series + text-indexed documents (eg-tsdb + eg-text) with schema-on-read streams. Gated obs, out of pi, folded node/full. First slice of Phase T (surpass OpenObserve).
CONCEPT:EG-KG.retrieval.observability-search Parquet-on-object-store log segments Persist ingested log/observability streams as Parquet columnar segments in the blob CAS / S3 backend (reuse blob-s3 + eg-tsdb Arrow segments EG-089), with partitioning + manifest, for cheap object-store storage + DataFusion query — the O2 "140× cheaper storage" architecture. Depends on AU-KG.ingest.self-ingest, EG-089.
CONCEPT:EG-KG.query.graph-store-http-protocol SPARQL Graph Store Protocol + COPY/MOVE/ADD The remaining SPARQL 1.1 Update graph-management ops (COPY/MOVE/ADD between graphs) in eg-rdf/update.rs, plus the W3C SPARQL 1.1 Graph Store HTTP Protocol (GET/PUT/POST/DELETE on /rdf-graphs/…?graph=) in src/server/sparql_http.rs — direct graph read/write for RDF tooling.
CONCEPT:EG-KG.compute.concept-2 ShEx (Shape Expressions) validation Shape-expression schema validation over the eg-rdf graph (shape refs, triple constraints, cardinality, node constraints, AND/OR/NOT), complementary to SHACL (EG-KG.ontology.concept-6). A pure-Rust module (eg-rdf or a small eg-shex), exposed via a Method::ShexValidate.
CONCEPT:EG-KG.query.concept-3 SQLite-compatible surface A SQLite drop-in via the EG-KG.compute.subsystems-reference WireProtocol: SQLite has no wire protocol, so serve SQL over the trait + provide a .db-file/sqlite3-style embedded served path (execute→classify→exec reused). Own module src/server/sqlite_wire/. Gated sqlite-wire, out of pi, node/full.
CONCEPT:EG-KG.query.kg-2 MySQL / MariaDB wire protocol A hand-rolled MySQL client/server protocol listener (handshake v10, auth, COM_QUERY, result-set + OK/ERR packets) behind the EG-KG.compute.subsystems-reference WireProtocol trait, so a MySQL driver/ORM connects and runs SQL over the ONE exec path. Own module src/server/mysql_wire/. Gated mysql-wire, out of pi.
CONCEPT:EG-KG.query.hand-rolled-tds-server MSSQL TDS wire protocol A hand-rolled TDS (Tabular Data Stream) listener (PRELOGIN/LOGIN7, SQL batch, row/token result stream) behind the EG-KG.compute.subsystems-reference WireProtocol trait, so a SQL Server driver connects. Own module src/server/mssql_wire/. Gated mssql-wire, out of pi.
CONCEPT:EG-KG.compute.subsystems-reference WireProtocol trait (multi-wire keystone) Extract the wire-agnostic core from src/server/pgwire/ into a WireProtocol trait (parse→classify→eg_query exec→encode) + a dialect-translation seam, so every wire reuses ONE exec path. Postgres becomes the first impl (refactor pgwire behind the trait, no behavior change). Unlocks the SQLite/MySQL/MSSQL wires (Phase J) AND the AMQP/broker wire (Phase Y).
CONCEPT:EG-KG.domains.coordinate-reference-system GIS coordinate reference systems + reprojection A crs module in eg-geo (EPSG codes, WGS84↔Web-Mercator + common transforms; pure-Rust, NO PROJ C dep) + a CRS tag on geometries; reproject between CRS. The biggest real-GIS gap beyond EG-KG.ontology.singles-concept's planar geometry. Adds a crs-carrying variant to the spatial wire types.
CONCEPT:EG-KG.ontology.de-9im-relations GIS DE-9IM topological relations Full DE-9IM relation matrix in eg-geo/predicates.rs (contains/covers/touches/crosses/overlaps/equals/disjoint) beyond EG-KG.ontology.singles-concept's within/intersects; surfaced as additional Pred::Spatial* variants in eg-types::wire + eg-plan executor.
CONCEPT:EG-KG.ontology.concept-9 GIS constructive geometry algebra Buffer, convex hull, union/intersection/difference, simplify, centroid in eg-geo (algebra module) — the constructive ops for urban-planning/logistics — with Op::SpatialOp { kind } in eg-types::wire executed in eg-plan. Pure-Rust.
CONCEPT:EG-KG.compute.uncertainty-values Probabilistic / uncertainty values First-class distribution-valued properties (Gaussian/Beta/Categorical/empirical-sample) as a tagged value type in eg-types (types.rs, NOT the wire Op algebra), persisted verbatim in redb; eg-compute (behind reasoning/datascience, reusing rand_chacha) adds Bayesian update + sampling + marginal/credible-interval. Generalizes the scalar eg:confidence/noisy-OR + Ebbinghaus decay into full distributions.
CONCEPT:EG-KG.compute.atomically-claim-oldest-pending Native engine task queue (ClaimNext) The ingest/work queue lives IN the engine (not a per-host SQLite file, which wedged and caused an overload). Tasks + staged graphs are nodes in the isolated __control__ graph; Method::ClaimNext is a single-round-trip atomic claim — under the write guard, pick the smallest-seq pending node of a label and CAS pendingclaimed. Raft/WAL-safe: deterministic pick + no server clock in the marker, so wal::apply reproduces the identical claim on replay/followers. EngineQueueBackend + client-side lease reaper. The starting point for the Phase-Y message-broker modality.
CONCEPT:EG-KG.txn.harness-crash Non-blocking / deterministic cross-shard commit A non-blocking commit protocol alongside the working 2PC (EG-KG.txn.cross-shard): either Calvin-style deterministic ordering (a sequencer assigns a global order so participants commit without a prepare/vote round) OR 3PC/Paxos-Commit (a Raft-replicated commit decision removes the coordinator single-point blocking window). Behind a feature, with a correctness/liveness harness (reuse the nemesis gauntlet). src/raft/.
CONCEPT:EG-KG.txn.cross-shard 2PC commit optimizations Parallel-commit (prepare all participant groups concurrently instead of in GroupId order) + a read-only-participant fast path (skip the prepare/log for groups whose write-set slice is empty) in src/raft/cross_shard_txn.rs, reducing the blocking window of the working N-participant 2PC (EG-KG.storage.lane-n-increment). Under the existing raft feature; no new feature.
CONCEPT:EG-KG.compute.feature RBAC-at-scale Durable roles + role hierarchy + resource/action grants on the security tier (builds on per-agent RLS + ACL + hash-chained audit EG-KG.sharding.row-level-security; AgentRole in eg-types::acl). A pure policy evaluator (sha2/hmac only, no new dep) consulted in the read/plan-path GraphView filter + the wire-auth actor mapping. eg-types + eg-core, out of pi, folded into security tiers.
CONCEPT:EG-OS.observability.slow-query-descriptor OpenTelemetry traces + slow-query log An OTLP span exporter over the existing tracing spans (behind a new otel feature: opentelemetry/tracing-opentelemetry, out of pi) + a slow-query log recording any plan/SQL/SPARQL over a threshold with its Op pipeline + cost estimate. Extends the Prometheus metrics surface; foundation for self-ingest dogfooding (Phase T2 EG-248).
CONCEPT:EG-KG.query.create-drop-extension-over CREATE EXTENSION mechanism + extension catalog pgwire accepts CREATE EXTENSION vector/pg_age/timescaledb/pg_search and records enabled extensions in a durable catalog; the drop-in keystone that lets an unmodified Postgres client/ORM connect. Gates the family surfaces (pgvector/AGE/ParadeDB/Timescale). eg-query classify + pgwire.
CONCEPT:EG-KG.query.pgvector-binary-wire pgvector vector type + distance operators A vector(n) SQL column type + <-> (L2), <=> (cosine), <#> (neg-inner-product) operators lowered in eg-query SQL, with pgwire type OID; ORDER BY emb <-> $1 LIMIT k parses. The eg-ann index-pushdown is EG-KG.query.real-ann-top-k (separate). eg-query + pgwire.
CONCEPT:EG-KG.ontology.concept-6 SHACL validation engine New pure-Rust leaf crate eg-shacl: node/property shapes, sh:targetClass/targetNode, cardinality/datatype/pattern/sh:in/logical (and/or/not) constraints, and SPARQL-based constraints, producing a sh:ValidationReport. Consumes the eg-rdf graph; exposed via a Method::ShaclValidate (not a wire Op). The single largest RDF-standards gap. Depends on eg-rdf.
CONCEPT:EG-KG.query.param-list-drives-unwind Cypher UNWIND UNWIND expr AS var clause in the eg-query Cypher parser + executor (expand a list into rows), composing with WITH/MATCH pipeline stages (EG-KG.query.eg-extend-read-side).
CONCEPT:EG-KG.query.cypher-planning Cypher CALL + procedure framework CALL { subquery } + CALL proc(args) YIELD … with a procedure-invocation registry dispatching to native/WASM procedures (reuses eg-wasm), the Neo4j-parity keystone. eg-query/cypher.
CONCEPT:EG-KG.query.eg-2 APOC-equivalent procedure library + GDS surface A registry of built-in procedures (util/load/refactor/path + graph-data-science: centrality/community/pathfinding — reusing eg-compute's PageRank/betweenness/Louvain/components) callable via EG-KG.query.cypher-planning CALL. eg-compute + eg-query/cypher.
CONCEPT:EG-KG.ontology.concept-8 Geodesic operations (GIS) Great-circle/Haversine + Vincenty distance and geodesic area in eg-geo (geodesic module), alongside EG-KG.ontology.singles-concept's planar predicates; a geometry's CRS tag (EG-KG.domains.coordinate-reference-system) selects planar-vs-geodesic. For accurate real-world GIS distance/area (logistics, urban planning).
CONCEPT:EG-KG.domains.geometry-collections Full geometry model (GIS) Extend eg-geo beyond single-ring Point/LineString/Polygon to MultiPoint/MultiLineString/MultiPolygon + GeometryCollection + polygon interior rings (holes), with EWKT parse/serialize. Widens the geometry enum + WKT codec; updates in-crate predicate/R-tree callers.
CONCEPT:EG-KG.compute.discounted-return Action/policy/trajectory memory Episodic trajectory memory for agents/robotics: an ordered :Trajectory of :Step{ state_ref, action, reward, next_state_ref, t } linked as a temporal chain, with append/query (by trajectory, by reward, windowed returns), discounted-return computation, and best/worst-trajectory retrieval — the substrate for policy learning + replay. Composes with EG-087 scene states + EG-KG.compute.hierarchical-summary-tier-eg/221 memory. eg-core (+ optional eg-plan query helper).
CONCEPT:EG-KG.query.multi-rate-sensor-stream Multimodal sensor fusion (robotics) Time-aligned fusion of heterogeneous sensor streams (camera/LiDAR/audio/tactile/proprioceptive): each stream is an EG-085 tensor series in eg-tsdb; reuse the eg-tsdb ASOF backward-join to align frames to a common clock and emit fused multi-channel rows. Adds Op::SensorFuse { streams, tolerance } (eg-types::wire, geo/tensor-style gating) executed in eg-plan. Composes EG-085 + EG-KG.query.pipelined-execution + eg-tsdb ASOF. Depends on EG-085, EG-KG.query.streaming-execution, EG-088.
CONCEPT:EG-KG.ontology.federation-client Super-cluster federated search Fan a read query (SQL/SPARQL/search) out to a registry of peer engine instances (regions/clusters), merge + dedup + re-rank the partial results, and return one unified answer — the OpenObserve super-cluster / multi-region federated-search capability. A peer registry + a /federated HTTP entry; per-peer timeout + partial-result tolerance (a slow/dead peer degrades, not fails). Reuses the ureq/rustls stack behind a federation/cluster feature; OUT of pi.
CONCEPT:EG-KG.compute.message-broker-exchanges Message-broker exchanges/routing + AMQP wire (Phase Y) Extends the EG-KG.compute.atomically-claim-oldest-pending claim/ack work-queue into a RabbitMQ-class broker: durable exchanges (direct/topic/fanout) + bindings/routing-keys + queues as __control__ graph nodes, publish/consume/ack via additive Method::* ops (EG-KG.compute.atomically-claim-oldest-pending pattern), and a hand-rolled AMQP 0.9.1 listener behind the EG-KG.compute.subsystems-reference WireProtocol seam (out of pi, node/full). First slice of Phase Y.
CONCEPT:EG-KG.maintenance.combined-maintenance-primitive Memory maintenance: decay + reinforcement The agent-native-memory MAINTENANCE module (arxiv 2606.24775, module 4; localized not global): each memory node carries importance + access-count + last-access; reinforce(id, now) bumps importance/recency on retrieval, decay(now, half_life) applies time-based importance decay, and evict_below(threshold) / forget prune low-value memories locally — the substrate the agent-utilities loop schedules. Deterministic (caller-supplied now). eg-core. Extends EG-KG.compute.hierarchical-summary-tier-eg/221.
CONCEPT:EG-KG.memory.byte-bounded-tiers Tiered KV-cache store (hot/warm/cold) A new eg-kvcache crate: a tiered key→block cache with a hot in-RAM tier (LRU + importance/recency scoring), a warm compressed-RAM tier, and a cold redb/blob tier, with automatic promotion/demotion (paging) on access + capacity pressure, and get/put/pin/evict. The substrate for an LLM KV-block cache that survives OOM by offloading to RAM/disk (LMCache/vLLM-style). Pure-Rust, out of pi.
CONCEPT:EG-KG.enrichment.content-address-separation Shared KV-cache backend (multi-instance) A SharedKvBackend trait + a content-addressed shared index so parallel-deployed engine/vLLM instances share KV blocks (hash-keyed, dedup, ref-count), plus a lookup/publish API an external vLLM/LMCache connector calls to fetch/store a block by its token-hash. eg-kvcache. Extends EG-185.
CONCEPT:EG-KG.memory.zero-copy-snapshot-fork Zero-copy KV snapshot-fork (LMCacheMPConnector) The "snapshot → branch" primitive over the KV-cache HTTP surface: POST /kv/snapshot pins a candidate set of pages into an immutable snapshot, POST /kv/snapshot/<id>/fork forks an O(1) zero-copy branch that shares the ONE physical page copy, and a branch PUT copy-on-writes into an isolated per-branch overlay (siblings unaffected) while a branch GET resolves overlay-then-shared. /kv/fork/stats proves resident bytes stay flat at the shared-page total regardless of branch count. The engine-side substrate for LMCache's multi-process (MP) fork/branch semantics. src/server/kvcache_http/mod.rs.
CONCEPT:EG-KG.retrieval.bounded-drill LeanRAG hierarchical retrieval Meet/exceed LeanRAG: structured hierarchical retrieval over the EG-KG.compute.hierarchical-summary-tier-eg summary-node tier — vector-retrieve at the summary/abstraction level (eg-ann), then drill DOWN through SUMMARIZES/CONSOLIDATES provenance edges to the specific supporting memories, assembling a concise multi-level context (bottom-up semantic aggregation + top-down guided traversal) that beats flat top-k RAG on redundancy/coverage. A new eg-plan retrieval module reusing eg-ann + the graph; NOT wire Ops.
CONCEPT:EG-KG.compute.hierarchical-summary-tier-eg Hierarchical summary-node memory tier Native multi-level memory abstraction ladder: :SummaryNode graph nodes that roll up a set of source memories with a level + provenance links to their children, and a summarize/rollup primitive (an eg-core op the agent-utilities memory loop invokes) that materializes a higher level from a cluster of lower-level memories. The engine-side substrate for the paper's representation-module summary/hierarchy gap; summarization CONTENT (LLM distill) stays in agent-utilities. eg-core (+ optional wire Op).
CONCEPT:EG-KG.compute.consolidate-cluster Episodic→semantic consolidation primitive A localized consolidation op (the paper's 'localized maintenance > global reorganization' finding): promote a cluster of episodic memory nodes into a consolidated semantic node — merging properties, redirecting edges, preserving provenance + bitemporal tx_from/tx_to, importance-weighted. A deterministic engine primitive the agent-utilities maintenance loop schedules; no global reindex. eg-core.
CONCEPT:EG-KG.compute.parse-resolve-span Write-lock-gap histogram Labelled-histogram metrics helper tracking the per-graph write-lock acquisition gap (contention observability on the topology write guard). src/write_coalescer.rs.
CONCEPT:EG-KG.storage.semantic-index-directory Off-path ANN warmup Warm the semantic ANN (eg-ann) index in a background task at startup, OFF the request path, so the first vector query doesn't pay index-load latency. src/main.rs.
CONCEPT:EG-KG.query.named-graph-support Deterministic WAL replay The write-ahead-log replay is deterministic: replaying the recorded Method stream reconstructs identical state (the durability + recovery foundation). src/wal.rs.
CONCEPT:EG-KG.query.register-user-tables-alongside Embedded query-surface writes The embedded (in-process) API routes Sql/Cypher/GraphQL write statements through the same governed write path as the wire surface (paired with EG-KG.query.mirrors-pgwire write classification). src/embedded.rs.
CONCEPT:EG-KG.query.mutation GraphQL mutation write routing A GraphQL mutation { … } document is classified as a write and routed through the write path (with EG-KG.query.mirrors-pgwire). src/server/handlers/query.rs.
CONCEPT:EG-KG.query.register-each-user-table Resolved write target routing The query handler resolves a statement's target graph/table and routes the write to it. src/server/handlers/query.rs.
CONCEPT:EG-KG.ontology.eg-runtime-swrl-datalog Off-lock reasoned RDF reads Read-only SPARQL/RDF queries reason over an off-lock analysis snapshot (no topology write-lock held during read reasoning; with EG-KG.query.mirrors-pgwire). src/server/handlers/rdf.rs.
CONCEPT:EG-KG.storage.namespaced-kv-surface Namespace-scoped KV store A generic key-value surface (feature kv), namespace-scoped (keys isolated per namespace), served over the engine. src/server/kv.rs / embedded.rs.
CONCEPT:EG-KG.query.mirrors-pgwire Query-surface write classification Classifies wire Method::Sql/CypherQuery/GraphQl statements as read vs write (a mutating INSERT/CREATE/mutation vs a read-only SELECT/MATCH … RETURN) to gate access + routing. src/server/access.rs.
CONCEPT:EG-KG.storage.redb-store redb store durability invariants redb-backed persistence invariants (the durable key/value + graph store backing the authoritative tier). src/redb_store.rs.
CONCEPT:EG-KG.storage.kg-kg-2 Multi-Raft ring group setup Multi-Raft group-ring construction; n_groups <= 1 leaves the ring empty (single-group degenerate case). src/raft/multi.rs.
CONCEPT:EG-KG.sharding.multi-raft Replicated voter set The multi-Raft voter set is identical on every node (the membership is itself replicated state). src/raft/multi.rs.
CONCEPT:EG-KG.storage.concept-2 Tagged sub-RPC demux The multi-Raft transport demuxes each group-tagged sub-RPC to its owning Raft group and replies in kind. src/raft/multi.rs.
CONCEPT:AU-KG.backend.authority-has-already-acked Raft network trait macro A macro that fills the absent associated types for the Raft network trait impl across groups. src/raft/network.rs.
CONCEPT:EG-KG.query.closure-backed-source Op::ForeignScan resolution + source registry Resolves the UQL/wire Op::ForeignScan leaf against a server-side ForeignSource registry (eg-plan trait already defined): register named foreign RowSet sources (another graph, a SQL table, a remote endpoint) and, at plan-exec, materialize the ForeignScan by invoking the registered source — the federation/OBDA execution seam. eg-plan + a registry the facade populates. Completes Phase K.
CONCEPT:EG-KG.ontology.completing-eg-order-by SPARQL algebra completeness (ORDER BY/VALUES/MINUS/EXISTS) Implements the SPARQL 1.1 algebra arms that currently hit the eval_pattern catch-all: ORDER BY (multi-key ASC/DESC with SPARQL term ordering) incl. ORDER BY … LIMIT top-k, VALUES (inline data-block join), MINUS (compatible-solution subtraction, distinct from FILTER NOT EXISTS), and EXISTS/NOT EXISTS filter functions. Fixes the correctness gap where results were UNORDERED. eg-rdf/sparql.rs.
CONCEPT:EG-KG.domains.geo-registry CRS registry + reprojection A coordinate-reference-system registry (EPSG codes incl. 4326/3857 + proj params) and pure-Rust reprojection (geographic↔Web-Mercator + affine/Helmert) so geometries carry a CRS and ST_Transform/GeoSPARQL CRS-URIs reproject correctly. eg-geo. Extends EG-KG.ontology.singles-concept/256.
CONCEPT:EG-KG.domains.spatial-strtree-index Durable R-tree spatial index A bulk-loaded (STR) R-tree over geometry bounding boxes for fast spatial-predicate/range/kNN pruning, persisted alongside the store, consulted by Op::SpatialScan for selectivity. eg-geo (+ eg-core index hook). Extends EG-083.
CONCEPT:EG-KG.domains.geojson-gpx-formats Geospatial format I/O (GeoJSON/WKB/GPX) Reader/writer for GeoJSON (Feature/FeatureCollection), WKB (binary geometry), and GPX tracks — round-tripping eg-geo geometries; the ingest/serialize path for map data. eg-geo. (Shapefile/KML/GeoParquet = follow-ups.)
CONCEPT:EG-KG.compute.scene-graph-primitives Scene-graph / 3D world model A native scene-graph modality in eg-core: :SceneObject nodes carrying a 3D pose (translation+quaternion rotation+scale = a transform), a parent/child transform hierarchy (world-transform composition down the tree), spatial relationships (on/in/near/supports), and bounding volumes — the substrate for robotics/AR/urban-3D world models. Composes with EG-KG.ontology.singles-concept (geo) + EG-085 (tensor) types (read-only). eg-core.
CONCEPT:EG-KG.ontology.comprehensive-interface-operations-documentation Comprehensive interface + operations docs Complete the docs/ surface: a per-wire-protocol connection guide (Postgres/psql, MySQL, MSSQL, SQLite, Bolt/neo4j-driver, Redis-cli, S3, AMQP, HTTP SPARQL/GraphQL/PromQL/traces), a capability matrix refresh vs Postgres/Stardog/Neo4j/OpenObserve/RabbitMQ, and an operations runbook (features/tiers, env vars, backup/PITR, RBAC). docs/ only.
CONCEPT:EG-KG.compute.massive-scale-benchmark Massive-scale benchmark harness A criterion/bench harness (benches/) + scripts for the core paths: node/edge write throughput, ANN kNN latency/recall, SPARQL/Cypher/SQL query latency, ingest rate, and broker publish/claim throughput — with a README documenting how to run + interpret. benches/ + scripts/ (additive; no crate-source edits).
CONCEPT:EG-KG.temporal.columnar-schema-inference Columnar storage + SQL window frames A columnar (struct-of-arrays) segment layout for analytical scans over the eg-tsdb/table store + SQL window functions with frame specs: ROW_NUMBER/RANK/DENSE_RANK/LAG/LEAD/FIRST_VALUE/LAST_VALUE/SUM/AVG OVER (PARTITION BY … ORDER BY … ROWS/RANGE BETWEEN …). eg-tsdb columnar segments + eg-query window-function planning/exec. Extends EG-KG.query.streaming-execution/089.
CONCEPT:EG-KG.ontology.foreign-source-seam OBDA virtual graphs (R2RML) Ontology-based data access: R2RML-style mappings (:TriplesMap — logicalSource → subject/predicate/object templates) that expose a foreign/relational source (via the EG-073 ForeignSource registry) as a VIRTUAL RDF graph, so SPARQL over the virtual graph rewrites to a ForeignScan against the backing source (no materialization). eg-rdf (mapping model + SPARQL rewrite) + eg-plan (ForeignScan). Completes Phase Q.
CONCEPT:EG-KG.ontology.eg-concrete-syntax-matrix JSON-LD serialization + parse JSON-LD 1.1 expansion/compaction I/O over the eg-rdf graph (context handling, @id/@type/@graph), so RDF round-trips as JSON-LD on ingest + SPARQL CONSTRUCT/DESCRIBE output negotiation. eg-rdf.
CONCEPT:EG-KG.ontology.completes-rdf-concrete-syntax TriG + N-Quads + RDF/XML serialization Named-graph-aware TriG + N-Quads writers/parsers and RDF/XML output (via oxrdfxml), completing the RDF 1.1 concrete-syntax matrix alongside Turtle/N-Triples (EG-KG.ontology.content-negotiation-serializers). eg-rdf.
CONCEPT:EG-KG.domains.map-tiles Map tiling: XYZ/TMS + Mapbox Vector Tiles Slippy-map tile addressing (XYZ/TMS z/x/y ↔ lon/lat/bbox via Web-Mercator) + Mapbox Vector Tile (MVT protobuf-lite) encoding of eg-geo features clipped to a tile, so a web map (Leaflet/MapLibre) renders the graph's spatial data. eg-geo. Extends EG-KG.domains.geo-registry/264.
CONCEPT:EG-KG.domains.geo-routing Weighted routing + isochrones + TSP Graph routing over a spatial network: weighted shortest path (Dijkstra/A* with a geo heuristic), isochrone (reachability within a cost budget), and a nearest-neighbour + 2-opt TSP tour — the logistics/urban-planning primitives. eg-geo (+ reuses graph traversal).
CONCEPT:EG-KG.domains.geo-task Map-based task tracking Geo-anchored tasks: a :GeoTask with a location + status + optional service-area geometry, spatial queries (tasks within a bbox/polygon, nearest-N to a point, tasks along a route), and assignment to the nearest resource — the field-ops/urban-planning task layer over the spatial store. eg-geo (+ node model).
CONCEPT:EG-KG.compute.graph-data-science-algorithms Graph data-science algorithms (GDS) A pure-Rust graph-algorithms library (Neo4j GDS parity) in eg-compute: PageRank, weakly/strongly-connected components, Louvain community detection, betweenness + degree centrality, single-source weighted shortest path (Dijkstra), and node similarity (Jaccard/cosine over neighborhoods). Consumed by Cypher CALL gds.* (wiring = follow-up). eg-compute.
CONCEPT:EG-KG.ontology.wired-into-commit-write Integrity Constraint Validation (ICV) Stardog-style ICV: interpret SHACL shapes (EG-KG.ontology.concept-6) under the CLOSED-world / unique-name assumption as database integrity constraints — validate the graph against them, report violations with focus-node + failing-constraint + a SPARQL 'explain' witness, and an optional guard mode that REJECTS a write/transaction that would violate a constraint. eg-rdf (+ reuse eg-shacl). Surpasses Stardog ICV by also running in the OWL-reasoned view.
CONCEPT:EG-KG.ontology.concept-7 RCC8 + Egenhofer topological relation families Extends the EG-KG.ontology.concept-10 GeoSPARQL baseline with the full OGC topological-relation function families: RCC8 (geof:rcc8eq/dc/ec/po/tpp/ntpp/tppi/ntppi) and Egenhofer (geof:ehEquals/disjoint/meet/overlap/covers/coveredBy/inside/contains), each lowered onto eg-geo DE-9IM intersection-matrix predicates. eg-rdf geosparql feature; extends EG-261.
CONCEPT:EG-KG.query.bolt-wire-protocol Neo4j Bolt wire protocol A native Bolt v4.4 protocol listener (src/server/bolt_wire/) so Neo4j drivers (neo4j-python/js/go, cypher-shell) connect directly: PackStream v2 chunked framing, HELLO/LOGON handshake, RUN/PULL/DISCARD/BEGIN/COMMIT/ROLLBACK messages routed to the existing Cypher engine, RECORD/SUCCESS/FAILURE responses. Behind the EG-KG.compute.subsystems-reference WireProtocol seam, feature bolt-wire, env EPISTEMIC_GRAPH_BOLT_ADDR. Out of pi (node/full/cluster).
CONCEPT:EG-KG.query.route-create-view-create pg_catalog + information_schema system views Postgres system-catalog compatibility so psql (\d, \dt, \l), ORMs, and BI tools introspect the engine: synthesize pg_catalog.pg_class/pg_namespace/pg_attribute/pg_type/pg_index/pg_proc + information_schema.tables/columns/schemata/views/routines from the live table/view/function catalogs, answerable as normal SELECTs (incl. common pg_catalog funcs like pg_table_is_visible, format_type, current_schema). eg-query. Extends EG-102.
CONCEPT:EG-KG.query.greatest-least-int4range-tsrange PostgreSQL array/range types + common functions Array types (int[]/text[]: literals, subscript, ANY/ALL, unnest, array_agg, array_length, || concat, @>/&& overlap), range types (int4range/tsrange with @>/&&/<@), and the common scalar functions ORMs/BI emit (coalesce/nullif already; add string_agg, split_part, regexp_replace, to_char/to_timestamp, date_trunc, extract, greatest/least, generate_series). eg-query (DataFusion UDF/UDTF registration + classify). Extends EG-KG.query.create-drop-extension-over/103.
CONCEPT:EG-KG.query.postgres-family-extension-plan Apache AGE cypher() over pgwire Postgres AGE compatibility: SELECT * FROM cypher('graph', $$ MATCH ... RETURN ... $$) AS (a agtype) — classify.rs recognizes the cypher(graph, $$...$$) set-returning function, routes the inner Cypher to the eg-query cypher engine, and projects results as the AS-clause columns. eg-query/pgwire.
CONCEPT:EG-KG.query.real-ann-top-k pgvector index pushdown Accelerate pgvector ORDER BY embedding <-> $1 LIMIT k by pushing the ANN search down to the eg-ann HNSW index (instead of a full scan+sort); CREATE INDEX ... USING hnsw/ivfflat DDL registers the index; distance operators <->/<#>/<=> map to L2/inner/cosine. eg-query/pgwire + eg-ann. Extends EG-115.
CONCEPT:EG-KG.query.continuous-aggregate-lowering TimescaleDB hypertables + continuous aggregates create_hypertable(), time_bucket() gap-fill, and CREATE MATERIALIZED VIEW ... WITH (timescaledb.continuous) continuous aggregates lowered onto the eg-tsdb time-series store + EG-KG.query.streaming-execution Op::Window. eg-query/eg-tsdb.
CONCEPT:EG-KG.query.create-drop-function SQL stored functions (CREATE FUNCTION) CREATE FUNCTION name(args) RETURNS t AS $$ … $$ LANGUAGE sql — parse + persist user-defined SQL functions in a durable catalog and invoke them in queries (scalar + table functions; SQL-language bodies substituted/executed via the existing SQL exec). PL/pgSQL control-flow (IF/LOOP/RETURN) is a documented follow-up (WASM/interpreter). eg-query. Extends EG-KG.query.create-drop-extension-over CREATE EXTENSION catalog work.
CONCEPT:EG-KG.query.paradedb-bm25 ParadeDB @@@ BM25 full-text The ParadeDB @@@ search operator + paradedb.* functions lowered onto the eg-text BM25 index (SELECT ... WHERE col @@@ 'query'), with score()/snippet(). eg-query/pgwire + eg-text.
CONCEPT:EG-KG.ontology.resp2-resp3-codec-round Redis RESP wire + core structures A hand-rolled RESP2/RESP3 protocol listener (src/server/redis_wire/) serving the core Redis command set (GET/SET/DEL/EXPIRE/INCR, HSET/HGET, LPUSH/LRANGE, SADD/SMEMBERS, ZADD/ZRANGE, keyspace scan) over the engine KV surface. Behind redis-wire feature + env EPISTEMIC_GRAPH_REDIS_ADDR. Out of pi.
CONCEPT:EG-KG.ontology.object-put-get-head S3 REST serving surface An S3-compatible REST API (src/server/s3/): bucket + object CRUD (PUT/GET/DELETE/HEAD/List) with SigV4-lite auth over the blob store, so S3 clients read/write blobs. Behind s3-api feature. Out of pi.
CONCEPT:EG-KG.query.mqtt-packet-codec MQTT broker wire An MQTT 3.1.1/5.0 protocol listener (src/server/mqtt_wire/) mapping CONNECT/PUBLISH/SUBSCRIBE/PINGREQ/DISCONNECT onto the EG-275 broker (topic exchange + bindings, QoS 0/1), so MQTT/IoT clients pub/sub over the native broker. Behind mqtt-wire feature + env EPISTEMIC_GRAPH_MQTT_ADDR. Out of pi. Extends EG-275.
CONCEPT:EG-KG.ontology.stomp-frame-codec-unit STOMP broker wire A STOMP 1.2 text-frame listener (src/server/stomp_wire/) mapping CONNECT/SEND/SUBSCRIBE/ACK/DISCONNECT onto the EG-275 broker, so STOMP clients pub/sub over the native broker. Behind stomp-wire feature + env EPISTEMIC_GRAPH_STOMP_ADDR. Out of pi. Extends EG-275.
CONCEPT:EG-KG.compute.replayable-append-log Broker streams (replayable append-log) Kafka/RabbitMQ-Streams-style persistent append-only stream queues on the broker: messages are retained (not deleted on consume) in an ordered offset-indexed log per stream, consumers read from an explicit offset (earliest/latest/N) and replay, with retention by size/age. Complements the EG-275 work-queue (claim/delete) semantics. eg-core/broker.rs.
CONCEPT:EG-KG.compute.publisher-confirms-consumer-qos Publisher confirms + consumer QoS acks RabbitMQ publisher-confirms (broker returns a confirm/nack per published message once durably enqueued, with a monotonic delivery-tag) + consumer manual-ack/nack-with-requeue over the claim path, so producers get at-least-once delivery guarantees. eg-core/broker.rs. Extends EG-275/280.
CONCEPT:EG-KG.compute.dead-letter-queues Broker dead-letter queues Per-queue DLQ policy: messages that exceed max-delivery-attempts or are rejected are routed to a configured dead-letter exchange/queue with the original routing metadata preserved. Extends EG-275. eg-core/broker.rs.
CONCEPT:EG-KG.compute.message-ttl-expiry Broker message TTL + queue expiry Per-message and per-queue TTL; expired messages are dead-lettered (or dropped) on a lazy sweep at claim time + a periodic reaper. Extends EG-275/276. eg-core/broker.rs.
CONCEPT:EG-KG.compute.priority-queues Broker priority queues Priority-ordered delivery within a queue (higher priority claimed first, FIFO within a priority band), on the EG-KG.compute.atomically-claim-oldest-pending claim path. Extends EG-275. eg-core/broker.rs.
CONCEPT:EG-KG.compute.delayed-scheduled-delivery Broker delayed/scheduled delivery Deliver-after (delay) + deliver-at (scheduled) messages held invisible until their eta, then made claimable — reusing the EG-KG.compute.priority-queues ordering + a due-time index. Extends EG-275. eg-core/broker.rs.
CONCEPT:EG-KG.compute.groups-qos-prefetch-honoring Broker consumer groups + QoS/prefetch Named consumer groups sharing a queue with per-consumer prefetch (unacked-in-flight limit) + fair round-robin dispatch + per-consumer visibility leases. Extends EG-275. eg-core/broker.rs.
CONCEPT:EG-KG.domains.graphql-enterprise-hardening GraphQL enterprise hardening Production GraphQL controls in eg-graphql: automatic persisted queries (APQ — sha256 hash registry + hash-only requests), query depth + complexity/cost analysis with configurable limits (reject over-budget queries before execution), field/node count caps, and an introspection on/off toggle. Protects the federated subgraph (EG-295) in production. eg-graphql.
CONCEPT:EG-KG.query.apollo-federation-subgraph GraphQL Apollo Federation subgraph Apollo Federation v2 subgraph support: _service { sdl } + _entities(representations:[_Any!]!) resolvers, @key/@shareable/@external directive parsing in the emitted SDL, so the engine is a federated subgraph in an Apollo supergraph. eg-graphql. Extends EG-064..066.
CONCEPT:EG-OS.observability.trace-assembly Distributed traces + trace search OTLP/OTLP-JSON span ingest on the obs listener (/v1/traces) into a span store (spans as nodes/rows keyed by trace_id/span_id with parent links, attributes, timings), a trace-assembly + search API (by service/operation/duration/tag, returning span trees), and service-dependency-graph derivation. Completes the logs(EG-KG.retrieval.observability-search)+metrics(EG-172)+traces observability trilogy; surpasses OpenObserve traces. Gated obs, out of pi.
CONCEPT:EG-KG.backend.is-configured-so-co KV-cache server endpoint + vLLM/LMCache connector A gated HTTP surface over the eg-kvcache SharedKvBackend (EG-185/186): GET/PUT/EXISTS a KV block by token-hash + stats, so parallel-deployed vLLM/LMCache instances share KV blocks through the engine (OOM-offload + cross-instance reuse). Plus a thin connector doc/adapter shaping the LMCache remote-backend contract. server + eg-kvcache path-dep. Behind a feature; out of pi.
CONCEPT:EG-KG.memory.eg-batch-decay-caller Scene/trajectory/memory wire-Op + MCP surface Expose the eg-core agent-memory + scene-graph + trajectory library APIs (EG-087/099/220/221/222) over the wire: additive Methods (CreateSummary/Consolidate/Maintain/SceneObject/Trajectory ops) + dispatch handlers, so AU/MCP can drive them remotely (today in-process only). eg-core + protocol/dispatch. Extends EG-087/099/220-222.
CONCEPT:EG-KG.query.scatter-knn-merge Cross-shard kNN full scatter Scatter a vector kNN query across per-shard eg-ann indexes and merge to a global top-k via the existing merge_topk leaf (EG-KG.retrieval.scatter-gather), for cluster-wide vector search. eg-ann + server (raft/router). Extends EG-069.
CONCEPT:EG-KG.coordination.backpressure-busy-signal Real-time QoS/SLO scheduler A QoS/SLO-aware request scheduler (server/transport): per-tenant/priority admission + deadline scheduling + backpressure so latency-critical requests meet SLOs under load. server. Realizes EG-097.
CONCEPT:AU-KG.compute.numeric-kernel Numeric kernel — one kernel, two surfaces (Analytics Program P1) Slim, BLAS/LAPACK-free numeric kernel (crates/eg-numeric: ndarray arrays/reductions/element-wise + faer pure-Rust LAPACK-class linalg — svd/eigh/solve/pinv/lstsq/qr/cholesky/det/matrix_power). Surface A = a Python extension module epistemic_graph.numeric (feature python: zero-copy rust-numpy + allow_threads GIL release) consumed by agent_utilities.numeric.xp (AU-KG.backend.lmcache-native-connector), so migrating off numpy is mechanical. Surface B = the SAME pure kernel the engine links as an rlib for in-database analytics (DataFusion UDFs / graph / vector / timeseries ops — compute-near-data, no FFI). Gated behind a numeric cargo feature (in full, OUT of pi/pi-max/node/release-tiny): the Python-extension FFI is behind eg-numeric's OWN python feature (off in the engine), so a numeric engine build links faer/ndarray but NO Python extension — the Plan-01 no-Python-extension-in-engine contract holds. Parity-proven np.allclose vs numpy (847 checks incl. nan/inf/singular/empty edge cases). P1 of the Analytics Program (plans/epistemic-graph-analytics_program.md); P2–P5 (the 598-site migration + full Surface B) follow. See architecture/numeric_kernel.md.
CONCEPT:EG-KG.sharding.follower-pull-loop Cross-region async read-replica tier A bounded monotone-LSN ReplicationLog the primary appends every committed mutation to + serves over /replicate?since=<lsn>, and an async follower pull-loop (run_replica_follower) that applies the tail via the canonical wal::apply path — a geographically-distant region gets a LOCAL, eventually-consistent read copy that never pays a cross-region Raft round-trip. A follower past the retained ring is told to re-snapshot (ReplicaLag::Behind). src/server/replica.rs, feature federation-search (reuses the pure-Rust ureq stack; out of pi). Beyond the synchronous multi-Raft groups + EG-KG.ontology.federation-client federated read.
CONCEPT:EG-KG.coordination.circuit-breaker Capacity guardrails (breaker/quota/backpressure) Three composable, pure, unit-tested guards fronting the replica-serve + transport paths: a per-target CircuitBreaker (Closed→Open→HalfOpen, opens after N consecutive failures for a cooldown, one half-open trial) so a dead region fails fast; a CapacityGuard per-tenant hard concurrency quota (bounds blast radius) + a global high-water backpressure shed (GuardDecision::Backpressure). Complements the EG-320 QoS scheduler (which reorders by priority) with absolute ceilings + fail-fast. src/server/replica.rs, feature federation-search; out of pi.
CONCEPT:EG-KG.txn.calvin-deterministic-ordering Full Calvin deterministic-ordering commit A THIRD cross-shard commit branch alongside 2PC + Paxos-Commit-lite (EG-KG.txn.cross-shard/082): a global CalvinSequencer stamps each txn with a monotone total-order GlobalSeq, that ORDER is Raft-replicated (the Calvin "input log"), and participants EXECUTE it deterministically with NO vote round and NO abort — so agreement on the order IS agreement on the outcome, and a crashed coordinator is resolved by any node replaying the replicated sequence (recover_sequenced). deterministic_order is the pure replay-stable order kernel. src/raft/cross_shard_txn.rs, feature calvin (implies nonblocking); opt-in per call, default path unchanged. Honest scope: OLLP read-lock reconnaissance + multi-node sequencer fan-in now SHIP as EG-KG.txn.concept-2/EG-343.
CONCEPT:EG-KG.txn.concept-2 Calvin OLLP reconnaissance + deterministic ordered read-lock phase Closes the EG-KG.txn.calvin-deterministic-ordering serializability gap for CONFLICTING sequenced txns. For a txn whose read/write set is DATA-DEPENDENT (not statically known), an OLLP (Optimistic Lock Location Prediction) reconnaissance read (CrossShardCoordinator::reconnoiter / predict_rwset) reads the current committed state at the seed records to PREDICT the full RwSet; a deterministic ordered lock manager (OrderedLockManager, keyed by RecordKey, granted STRICTLY in the sequencer's GlobalSeq total order — shared reads co-grant, writes exclusive) then acquires read+write locks in sequence order, so two conflicting sequenced txns serialize in the sequence order (the lower-seq txn's committed write is observed by the higher-seq one) — full serializable isolation. The deterministic-execution phase then runs lock-free. Deadlock-free because a request only ever waits on strictly-lower-GlobalSeq requests (identical order on every record ⇒ acyclic wait-for). src/raft/cross_shard_txn.rs, gated calvin/test/harness (out of full). Honest scope: registration MUST be in sequence order (the sequencer/epoch scheduler supplies that — admit_batch); recon_still_valid implements OLLP staleness DETECTION and the automatic re-sequence/restart loop now SHIPS as EG-348. Proven by calvin_ollp_ordered_readlock_serializes_conflicting_txns. Extends EG-324.
CONCEPT:EG-KG.txn.concept-4 Calvin OLLP recon-staleness restart Wires the automatic re-sequence/RESTART loop EG-KG.txn.concept-2 documented but left open. EG-KG.txn.concept-2 supplied only staleness DETECTION (recon_still_valid); CrossShardCoordinator::acquire_ollp_with_restart closes it: each attempt reconnoiters the seeds, derives the predicted RwSet, draws a FRESH GlobalSeq, acquires the ordered locks, and re-validates under them — if a lower-sequence txn changed a dependency after the recon (recon_still_valid = false) it RELEASES the wrongly-predicted locks and re-submits the txn at a new (strictly higher) sequence, bounded by max_restarts (exhaustion ⇒ deterministic abort Err). Returns an OllpAcquired (validated seq/rwset + held LockGuard + attempts). Determinism (honest scope): the restart DECISION is a pure function of committed, Raft-replicated state observed UNDER the sequence-ordered lock, so every replica replaying the same ordered log makes the identical stale/valid decision, the same restart count, and the same final abort-or-commit — a full guarantee within the single-active-sequencer ordering domain (EG-KG.txn.calvin-deterministic-ordering). Routing a restarted txn into a specific epoch of the multi-node fan-in is now WIRED (EG-349, acquire_ollp_with_restart_epoch, commit 88c9f34 / re-affirmed surpass-6mo WS-3): the cross-node epoch PLACEMENT of a restart is deterministic (attempt N → epoch_for_restart(base_epoch, N) = base_epoch + (N-1), packed cross-epoch-comparably by epoch_packed_seq), so every node agrees on the restart's final epoch + seq — proven by the live calvin_ollp_epoch_routing_restart_agrees_across_nodes harness test. The one genuine remainder is the cross-node GOSSIP TRANSPORT that exchanges each node's per-epoch batch (see EG-KG.txn.concept-3), not the routing decision itself. src/raft/cross_shard_txn.rs, gated calvin/test/harness (out of full). Proven by calvin_ollp_stale_recon_is_restarted_and_commits_serializably + calvin_ollp_epoch_routing_restart_agrees_across_nodes. Extends EG-342.
CONCEPT:EG-KG.txn.concept-3 Multi-node sequencer epoch fan-in The multi-node generalization of EG-KG.txn.calvin-deterministic-ordering's single CalvinSequencer: each node's local sequencer produces a per-epoch batch of NodeInputs; epoch_fan_in(epoch, per_node) deterministically MERGES every node's inputs into ONE global total order per epoch (EpochBatch of SequencedInput, ascending GlobalSeq) via a pure round-robin interleave sorted by (local_seq, node, txn_id). Because the merge is a pure, data-only function, every node derives the byte-IDENTICAL global order from the same per-node batches — the property the vote-free deterministic execution relies on (cross-epoch total order is lexicographic (epoch, global_seq)). src/raft/cross_shard_txn.rs, same gating as EG-342. Proven by calvin_two_node_epoch_fan_in_derives_identical_order. Extends EG-324.
CONCEPT:EG-KG.domains.robotics-gpu-distribution ROS2 bridge over rosbridge-WebSocket Bridges engine CDC events ↔ ROS2 topics by talking the standard rosbridge_suite JSON-over-WebSocket protocol (advertise/subscribe/publish/unsubscribe) to a rosbridge_server — NO CycloneDDS/rmw/DDS C stack, just a pure-Rust tokio-tungstenite client. Engine mutations → ROS2 publishes (cdc_to_publish); inbound ROS2 messages → engine AddNode via wal::apply (publish_to_method). src/server/ros2_bridge.rs, feature ros2-bridge; out of pi/default/node.
CONCEPT:EG-KG.ingest.dds-transport Native DDS/RTPS transport seam + ros2-dds/ros2-rmw legs The DdsTransport trait (src/server/dds.rs) unifies the EG-KG.domains.robotics-gpu-distribution rosbridge-WebSocket leg and TWO native DDS legs behind ONE interface, so the engine's CDC↔ROS2 topic path can target ANY of them — all put the IDENTICAL std_msgs/String payload on the wire (the pure EG-KG.domains.robotics-gpu-distribution cdc_to_publish/publish_to_method shaping is reused; advertise/publish/subscribe/poll_inbound + a default publish_cdc + inbound_to_method). The first native impl (NativeDdsTransport, feature ros2-dds) speaks REAL RTPS via the pure-Rust rustdds crate (native Rust DDS+RTPS — mio/pnet/speedy/cdr-encoding), so it links NO CycloneDDS/rmw/ros C toolchain and genuinely builds in CI everywhere; a #[cfg(feature="ros2-dds")] loopback test publishes→subscribes over an actual RTPS wire. The SECOND native impl (CycloneDdsTransport, feature ros2-rmw, S5 — see EG-KG.ingest.rmw-cyclonedds-leg below) links the REAL CycloneDDS-C stack for genuine zero-config live-ros2 interop. Both are kept OUT of pi/default/node/full — folded ONLY into the full-extras bundle (a pi/full build links no rustdds/cyclonedds, asserted by cargo tree). The rmw topic/type name-mangling for zero-config live-ros2 interop is implemented as EG-KG.ingest.rmw-topic-prefix (below) and shared by both legs.
CONCEPT:EG-KG.ingest.rmw-topic-prefix rmw topic/type name-mangling for live-ros2 interop Makes a topic published by either EG-KG.ingest.dds-transport native DDS leg discoverable/subscribable by a REAL ros2 topic echo with zero config (ROS 2 Humble+, rmw_cyclonedds/rmw_fastrtps). Pure functions mangle_topic_name (a ROS topic /chatter → the DDS topic rt/chatter — fully-qualify then prepend the rt prefix) and mangle_type_name (std_msgs/String or std_msgs/msg/Stringstd_msgs::msg::dds_::String_, the <pkg>::<ns>::dds_::<Msg>_ descriptor) are applied INTERNALLY at DDS-Topic creation (NativeDdsTransport::topic for ros2-dds, CycloneDdsTransport::topic for ros2-rmw), so the DdsTransport interface stays ROS-topic-oriented (callers pass /chatter; writer/reader map keys + poll_inbound still surface the un-mangled name). rustdds' CDRSerializerAdapter frames every sample with exactly the 4-byte CDR_LE encapsulation header (00 01 00 00, CDR_LE_ENCAPSULATION_HEADER) ros2 decodes; the CycloneDDS-C leg gets the same framing for free from the C stack itself. src/server/dds.rs, features ros2-dds/ros2-rmw (extras-only, Pi contract unchanged). Residual: the ROS 2 Iron/Jazzy type-hash / typesupport RIHS01_… descriptor is deferred on BOTH legs (Humble matches by mangled type name only, which this satisfies). Tests: eg349_mangle_topic_name_matches_rmw_convention, eg349_mangle_type_name_matches_rmw_convention, eg349_ros2_cdr_le_encapsulation_header, eg347_cyclone_type_name_matches_mangling, + the mangled-name RTPS loopback on each leg.
CONCEPT:EG-KG.ingest.rmw-cyclonedds-leg CycloneDDS-C-backed rmw transport leg (S5) The THIRD DdsTransport impl (CycloneDdsTransport, src/server/dds.rs, feature ros2-rmw) links the REAL rmw_cyclonedds/CycloneDDS-C stack via the safe cyclonedds Rust crate (cycloneddscyclonedds-rust-syscyclonedds-src), so a topic published here is genuine zero-config live-ros2 interop — a real ros2 node discovers/pubs/subs with NO bridge, not merely wire-compatible framing. cyclonedds-src VENDORS the CycloneDDS C sources in the crate tarball (no network fetch at build time); cyclonedds-rust-sys's build script configures+builds them with cmake (static lib, security/SSL/IDLC/examples OFF) and links PREBUILT bindgen output shipped in the crate (no libclang/bindgen invocation). The DDS sample type (CycloneRos2String, a single DdsString field) is a HAND-WRITTEN DdsType impl (no IDL compiler) using the crate's public topic::{adr, TYPE_STR} ops-array API, with an overridden clone_out (a dds_string_dup, not a bitwise ptr::read) to avoid a double-free across the DDS loan boundary. The Topic<T> wrapper (which holds an Rc, so keeping it would make the transport !Send/!Sync) is std::mem::forget'd right after creating the writer/reader from it — the underlying DDS entity lives on transitively under the DomainParticipant, torn down only when the participant itself drops. Toolchain-gated (needs cc/cmake), kept OUT of pi/default/node/full — folded ONLY into full-extras (a pi/full build links no cyclonedds/cyclonedds-rust-sys/cyclonedds-src, asserted by cargo tree). Test: eg347_cyclone_dds_loopback_pub_sub_roundtrip (a real RTPS loopback over the CycloneDDS-C wire, mirroring the ros2-dds leg's loopback test).
CONCEPT:EG-KG.compute.gpu-distance-seam GPU distance/tensor dispatch seam A DistanceBackend (eg-ann) / TensorBackend (eg-tensor) trait seam factoring the batch-distance + elementwise kernels behind a swappable compute device. The pure-Rust CpuBackend is ALWAYS the compiled-in fallback and byte-for-byte the ground truth; FlatIndex::search + Tensor::elementwise route through the batch_distances/elementwise_dispatch dispatch. crates/eg-ann/src/distance.rs + crates/eg-tensor/src/gpu.rs, feature gpu; the seam itself is pure-Rust but kept out of pi/full by the facade.
CONCEPT:EG-KG.backend.real-cuda-tensor-backend Real CUDA distance/tensor backend The GPU half of the EG-KG.compute.gpu-distance-seam seam: a real cudarc-backed CUDA backend that NVRTC-compiles the batch-distance / elementwise CUDA-C kernel at first use, ships buffers to the device, and launches one thread per row/element — selected when a device initialises, else the seam falls back to CPU at runtime (a gpu-cuda binary is fully functional on a GPU-less host). cudarc is dynamic-loading (libcuda/libnvrtc dlopen'd at runtime) so the leg BUILDS with NO CUDA toolkit; feature gpu-cuda, OUT of pi/default/node/full (extras-only) — a pi build links no cudarc (asserted by cargo tree --features pi).
CONCEPT:EG-KG.storage.rle-codec-default Real KV warm-tier compression (zstd) Replace the eg-kvcache warm-tier RLE-fallback with real zstd/lz4 compression (behind a feature; out of pi), for effective RAM offload of KV blocks. eg-kvcache. Extends EG-185.
CONCEPT:EG-OS.observability.prometheus-ingest OTel export + Prometheus remote-write + OTLP Export the engine's own metrics/traces to an external OTel collector (OTLP push) + a Prometheus remote-write receiver, closing the observability loop (the engine ingests AND emits). Adds a protobuf (prost) dep behind an otel-export feature; out of pi. server/obs. Extends EG-091/163/173.
CONCEPT:EG-KG.storage.lsn-as-snapshot-returns LTAP lakehouse interop (Parquet/Delta/Iceberg) An async columnar-materialization tier (new eg-lake crate) transcoding engine table/columnar data → Parquet-on-object-store with Delta+Iceberg transaction logs + an Iceberg-REST catalog + LSN-style as-of snapshots (reusing versioned snapshots/Op::AsOf), so external lakehouse engines read our tables with zero ETL — making epistemic-graph an LTAP superset. arrow/parquet + delta/iceberg deps behind a lake feature; out of pi.
CONCEPT:EG-KG.ingest.broker-streams-namespaces Multi-language client drivers (broker/streams/RBAC/backup/NL) B1.7 thin client bindings for the Program-B engine Methods that had no client surface: the native broker + append-log streams (EG-275..284/314), RBAC admin (EG-KG.compute.feature), online backup/restore (EG-090), and NL→query (EG-080). A FULL Python surface on EpistemicGraphClient (.broker/.rbac/.admin namespaces + query.nl_query) matching the existing composition style, plus THIN, generated-from-the-Method-list JS (clients/js) and Go (clients/go) bindings over the SAME framed-MessagePack transport (4-byte BE length + {id,graph,auth_token,method,params}, HMAC-SHA256 auth). Thin clients only — Pi-contract preserved (no heavy deps, nothing in pi). See docs/interfaces/clients.md. Binds EG-275..284/314/EG-KG.compute.feature/EG-090/EG-080 to the wire.
CONCEPT:EG-KG.query.surface-b-numeric-operators Surface-B numeric SQL analytics operators (Analytics Program P4) The eg-numeric kernel (AU-KG.compute.numeric-kernel) exposed as DataFusion SQL UDFs/UDAFs so analytics run IN-ENGINE over resident columns (compute-near-data, no fetch-to-Python, no FFI). Scalar UDFs cosine_sim(a,b) (kernel dot/norm cosine similarity — the raw-similarity complement to EG-115's vector_cosine distance), l2_normalize(v) (unit vector v/‖v‖List<Float32> pgvector, feeds cosine_sim/ANN in-query), zscore(col) (standardize (x-mean)/std, population ddof=0, over the materialized batch — a global two-pass is the stddev(x) OVER () window form); UDAF covariance(a,b) (sample ddof=1, kernel means, List<Float64> merge state). Registered on both the graph exec and obs-tables SQL paths. Gated behind eg-query's numeric feature (implies sql, pulls the pure faer/ndarray kernel — NO pyo3); the engine numeric feature turns it on via eg-query?/numeric (in full, OUT of pi/node: cargo tree --features pi links neither eg-numeric nor faer). crates/eg-query/src/sql/numeric.rs. The column→Array2 marshalling operators svd/pca land as EG-KG.query.svd-eg-pca-column/EG-335. P4 of the Analytics Program; see architecture/numeric_kernel.md.
CONCEPT:EG-KG.compute.l2-normalize-batch-vectors Kernel-backed in-engine batch vector op (BatchL2Normalize) A new engine Method::BatchL2Normalize { vectors } that L2-normalizes a batch of vectors IN-ENGINE via the eg-numeric kernel (linalg::batch_l2_normalize, compute-near-data) — the kernel-backed successor to the deprecated BatchCosineSimilarity on the same client/Method path, with a client.batch_l2_normalize() Python binding. Handler gated #[cfg(feature = "numeric")] (a no-numeric build, e.g. pi, reports the op requires the feature); the enum variant is always present (like the finance Methods) so the wire stays stable. src/server/handlers/graph_ops.rs + crates/eg-numeric/src/linalg.rs. Extends/replaces the EG-era BatchCosineSimilarity.
CONCEPT:EG-KG.storage.content-addressed-put KV-cache data-version invalidation Data-version invalidation for the eg-kvcache shared + tiered KV cache (EG-185/186/187), mirroring eg-core's version-keyed result cache (KG-2.233): each cached entry captures the DataVersion (a GraphCore::version() epoch, KG-2.180) it was DERIVED at, and a graph write that advances the store's current version RETIRES every now-stale entry so no stale agent/LLM context is ever served. A new DataVersion { Agnostic, At(u64) } type (crates/eg-kvcache/src/version.rs) — Agnostic (the default) marks a pure content-addressed KV page that is NEVER invalidated, so the EG-186 dedup / EG-185 tiering (LRU, pinning) paths are untouched and the feature is opt-in / zero-cost. SharedKvBackend::put_block gains a derived_at arg + set_data_version; TieredCache layers a side-map of per-key versions on top of the tiers + put_versioned/set_data_version (retires stale across HOT/WARM/COLD, pinned or not). The EG-KG.backend.is-configured-so-co HTTP surface wires it: an optional X-EG-Data-Version header version-tags a PUT, PUT /kv/version/<n> advances the surface's version (the hook a graph write drives), and a stale entry reads as 404. Chose the simple version EPOCH (over a per-dependency-set) — correct-by-construction, O(1) to trigger, exactly the proven ResultCache pattern; over-invalidation is a latency cost only. Extends EG-185/186/187, mirrors KG-2.233.
CONCEPT:EG-KG.backend.shipped-pip-installable-python Python LMCache remote-backend driver The shipped, pip-installable Python driver (epistemic_graph.kvcache) that lets vLLM/LMCache use the EG-KG.backend.is-configured-so-co KV-cache endpoint as a remote KV backend. RemoteKVConnector implements the LMCache remote-backend shape (get/put/contains/exists/stats) over GET\|PUT\|HEAD /kv/<hash> + /kv/<hash>/exists + /kv/stats with the bearer token (EPISTEMIC_GRAPH_KVCACHE_TOKEN) and a configurable base URL, degrading every error to a cache miss on the hot path. RemoteKVL2Connector is the LMCache native_plugin L2-adapter native client (event_fd/submit_batch_*/drain_completions) wrapping it so the decoupled lmcache server offloads its L2 tier to the content-addressed surface (dedup + /kv/stats apply, unlike the generic resp/Redis L2 adapter). Stdlib-only import (urllib); optional httpx behind the [lmcache] extra. Realizes the deferred Python-connector half of EG-187. epistemic_graph/kvcache/.
CONCEPT:EG-KG.storage.eg-iceberg-avro-manifest Iceberg v2 Avro manifest writer Replaces the LTAP tier's (EG-KG.storage.lsn-as-snapshot-returns) Iceberg Avro manifest stub with a real, spec-compliant format-version-2 manifest writer: manifest_entry Avro records (status/snapshot_id/null-inherited sequence numbers + a data_file with content/file_path/file_format/partition/record_count/file_size_in_bytes), Iceberg field-ids on every Avro field, and the required manifest metadata keys (schema/schema-id/partition-spec/partition-spec-id/format-version/content) in the Avro container header. Written at the exact path metadata.json's snapshot references, so a committed Iceberg snapshot resolves to real Avro a stock reader (Spark/Trino/DuckDB) follows. Pure-Rust apache-avro (null codec, no C *-sys) behind eg-lake's lake feature — OUT of pi. crates/eg-lake/src/iceberg_avro.rs. Per-column stats are now emitted (CONCEPT:EG-KG.storage.iceberg-avro-manifest-carries). Extends EG-317.
CONCEPT:EG-KG.storage.multi-graph-batch-write Batched cross-graph write op A new Method::MultiGraphBatchUpdate { batches_msgpack } that carries a BatchUpdate-shaped op list for MANY named graphs in ONE round-trip; the server (src/server/dispatch.rs::multi_graph_batch_update) decodes Vec<(graph_name, operations_msgpack)> and applies each graph's sub-batch through the ORDINARY per-graph write path (dispatch_graph_opMethod::BatchUpdate) CONCURRENTLY on a tokio::JoinSet, so N distinct graphs commit across N of the K redb shard writers in parallel instead of the client serializing N round-trips that each re-acquire one per-graph write lock. REUSES the existing per-graph batch_update primitive (no new per-op op), so persistence/WAL/Raft/CDC/access-control all apply per sub-batch exactly as a normal batch, and a single-graph multi-graph write is byte-for-byte a plain BatchUpdate. Carries its graphs in the METHOD (like NlQuery), routed BEFORE the single-req.graph graph-op path. Partial-success: {"results": {graph: <batch_result>}, "errors": {graph: msg}} — one graph's failure never aborts the others. Python binding client.lifecycle.multi_graph_batch_update(batches). The write-side complement to per-shard content-keyed graph routing (agent-utilities AU-KG.ingest.batched-cross-graph-writer): fanning a hot source across src:x#0..#K-1 sub-graphs + writing them in one multi-graph batch makes the memory-gen write stage scale with the distinct-graph width K. crates/eg-types/src/protocol.rs, src/server/dispatch.rs, epistemic_graph/client.py.
CONCEPT:EG-KG.storage.iceberg-manifest-list Iceberg manifest-list writer The manifest-list half of EG-KG.storage.eg-iceberg-avro-manifest: a v2 manifest_file Avro record (manifest_path/length, partition_spec_id, content, sequence_number/min_sequence_number, added·existing·deleted file & row counts, nullable partition field-summaries) written as the Avro file the snapshot's manifest-list points at, referencing the EG-KG.storage.eg-iceberg-avro-manifest manifest. Same lake-gated apache-avro path; OUT of pi. crates/eg-lake/src/iceberg_avro.rs. Extends EG-KG.storage.lsn-as-snapshot-returns/EG-333.
CONCEPT:EG-KG.storage.iceberg-avro-manifest-carries Iceberg per-column stats + field summaries The deferred EG-KG.storage.eg-iceberg-avro-manifest follow-on: the LTAP tier now gathers per-column statistics as each data file is materialized and emits them into the Iceberg Avro data_file manifest entry, so an external reader (Spark/Trino/DuckDB) does predicate pushdown / file skipping. parquet_io::materialize_with_column_stats walks the batch once for per-column value_count/null_count/nan_count and typed min/max, and reads the Parquet FileMetaData for each column's compressed column_size; the neutral ColumnStat (bounds kept as typed CellValue, no Iceberg coupling in the pure snapshot layer) rides on FileEntry.column_stats. The manifest writer serializes them into the six spec stats maps — column_sizes (108) / value_counts (109) / null_value_counts (110) / nan_value_counts (137) / lower_bounds (125) / upper_bounds (128) — as Avro logicalType: map arrays keyed by the column's Iceberg field-id, bounds in Iceberg single-value binary (little-endian scalars, raw UTF-8 for strings, untruncated so always spec-valid). A file recorded without stats (bare record_file) emits the maps as null. The manifest_file.partitions field_summary list stays null — correctly, because eg-lake materializes an unpartitioned spec (zero partition-spec fields ⇒ zero summaries; data-file bounds are the file-skipping mechanism for an unpartitioned table). Same lake-gated apache-avro/parquet path; OUT of pi. crates/eg-lake/src/{parquet_io,snapshot,iceberg_avro}.rs. Extends EG-KG.storage.eg-iceberg-avro-manifest/EG-KG.storage.iceberg-manifest-list/EG-317.
CONCEPT:EG-KG.query.concept-6 Surface-B pca(vec_col, k) UDAF (column→matrix PCA) Completes the deferred half of P4: a DataFusion aggregate that marshals a COLUMN OF VECTORS (same operand forms as cosine_sim — a List<Float{32,64}> column or '[..]' text) into a dense n×d ndarray::Array2, mean-centers each column, eigendecomposes the d×d sample covariance (ddof=1, eg_numeric::reductions::mean + linalg::eigh), and returns the top-k principal-component DIRECTIONS (loadings) descending by explained variance as List<List<Float64>> (k unit vectors of length d; sign arbitrary, projected coords = X_centered·componentsᵀ downstream, k clamped to d). Accumulator buffers rows row-major (flat Vec<f64>+dim, k from the 2nd arg) so multi-phase grouping merges losslessly (state = flat List<Float64>+dim/k Int64). Registered on both the graph exec and obs-tables SQL paths via register_numeric, gated behind the numeric feature (out of pi). crates/eg-query/src/sql/numeric.rs. Extends EG-KG.query.surface-b-numeric-operators; see architecture/numeric_kernel.md.
CONCEPT:EG-KG.query.svd-eg-pca-column Surface-B svd(vec_col) UDAF (column→matrix SVD) The sibling of EG-KG.query.concept-6: a DataFusion aggregate that stacks a COLUMN OF VECTORS into an n×d matrix (each row = one matrix row) and returns its singular values (descending) as List<Float64> via eg_numeric::linalg::svdvals (faer, BLAS/LAPACK-free). Shares the MatrixAcc row-buffering accumulator + cell_to_json numeric-list rendering with EG-KG.query.concept-6; registered on both SQL paths, gated numeric (out of pi). crates/eg-query/src/sql/numeric.rs. Extends EG-KG.query.surface-b-numeric-operators/EG-321.
CONCEPT:EG-KG.query.kmeans-clustering-half-one Surface-B kmeans(vec_col, k) UDAF (column→matrix clustering) The clustering sibling of EG-KG.query.concept-6/EG-KG.query.svd-eg-pca-column: a DataFusion aggregate that marshals a COLUMN OF VECTORS (same operand forms as cosine_sim — a List<Float{32,64}> column or '[..]' text) into a dense n×d ndarray::Array2 and returns one hard cluster label (0..k) per aggregated ROW, in ingestion order, as List<Int64>. Backed by a NEW pure-Rust k-means kernel eg_numeric::cluster::{kmeans, kmeans_labels} (Lloyd's algorithm + k-means++ seeding, seedable via the kernel's ChaCha20 random::GeneratorNO linfa/BLAS, so the Pi contract holds: cargo tree --features pi links no eg-numeric/faer/ndarray/linfa). k clamped to n, empty clusters re-seeded to the farthest point, deterministic given the fixed default seed (KMEANS_DEFAULT_SEED=42) so a SQL aggregate is reproducible. Reuses EG-KG.query.concept-6's MatrixAcc row-buffering accumulator + k-from-2nd-arg state (generalized via MatrixOp::takes_k); cell_to_json extended to render List<Int64>. Registered on both SQL paths via register_numeric, gated numeric (out of pi). crates/eg-numeric/src/cluster.rs + crates/eg-query/src/sql/numeric.rs. Extends EG-KG.query.surface-b-numeric-operators/EG-321.
CONCEPT:EG-KG.query.eg-3 Cross-modal join→analytics in-engine (the numpy-surpassing differentiator) The Analytics-Program P4 differentiator: join graph + vector + timeseries, then run pca/kmeans/covariance over the JOINED result set IN-ENGINE (compute-near-data, no fetch-to-Python, no data-layer round-trip) — impossible in numpy, which has no data layer to join across. One SQL statement joins the resident nodes table (graph/relational modality + a per-node vector embedding prop) to a per-node timeseries aggregate (AVG(reading) over a readings relation) on node id, and computes: kmeans(emb, k) clustering the joined vectors, pca(emb, k) over the joined vectors, and covariance(graph_scalar, timeseries_avg) — a statistic that spans TWO modalities aligned by the join. Proven end-to-end by crates/eg-query/tests/cross_modal_analytics.rs on synthetic data with hand-computed results (cross-modal cov = 7.0; two balanced k-means clusters; PC1 = ±[1/√2,1/√2]). Builds on EG-KG.query.kmeans-clustering-half-one/335/336/329. See architecture/numeric_kernel.md + architecture/analytics_program.md.
CONCEPT:AU-KG.compute.is-installed-kernel-discovery eg-numeric Surface-A pyo3 wheel + numpy-parity CI gate The Surface-A (AU-KG.compute.numeric-kernel) Python extension shipped as an installable pyo3 cdylib wheel so agent_utilities.numeric.xp runs kernel-LIVE (AU-KG.compute.shim-goes-kernel-live), not just the numpy fallback. Built with maturin build --release -m crates/eg-numeric/Cargo.toml --features python — the pymodule is numeric with __kernel__ = "eg-numeric", discovered by the shim as epistemic_graph.numeric (folded build) or top-level numeric (standalone wheel). On a Python newer than pyo3-0.22's max (e.g. 3.14) set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to build anyway; on ≤3.13 no flag is needed. This is a SEPARATE cdylib wheel from the bindings="bin" server facade, so scripts/check_no_pyo3.sh (which guards the facade + src/, never crates/) and the Pi contract still hold — cargo tree --features pi \| grep -ci pyo3 = 0. Gated by the numeric-parity CI job (.github/workflows/rust-ci.yml): it maturin-builds the wheel, installs it, and runs crates/eg-numeric/tests/test_kernel_parity.py — a self-contained corpus asserting every kernel op np.allclose its numpy reference across the full op-surface incl. edge cases (nan/inf, singular matrix, empty), so CI FAILS if the Rust kernel diverges from numpy. See architecture/numeric_kernel.md. Extends EG-321.
CONCEPT:EG-KG.compute.tensor-gpu-distance eg-numeric PyPI publish leg + GPU-gated CUDA parity test Two closes: (A) the eg-numeric wheel actually PUBLISHES. The AU-KG.compute.is-installed-kernel-discovery Surface-A wheel was built + numpy-parity-gated in CI but never shipped to PyPI — the ONE blocker to agent_utilities fully dropping numpy (downstreams cannot hard-depend on an unpublished kernel). .github/workflows/release-build.yml gains a wheels-numeric matrix (maturin build --release -m crates/eg-numeric/Cargo.toml --features python, linux x86_64 + aarch64, manylinux 2_28) + a sdist-numeric leg, uploaded as wheel-numeric-* / sdist-numeric, and wired into the tag-gated publish-pypi job's needs + artifact download (the wheel-* glob already sweeps in wheel-numeric-*), so a v* tag publishes eg-numeric ALONGSIDE epistemic-graph. Publishing stays TAG-ONLY (the existing if: startsWith(github.ref, 'refs/tags/v') gate is unchanged) — a branch push/PR builds + smoke-tests but never publishes. (B) the CUDA kernels get LIVE GPU validation. The EG-KG.compute.gpu-distance-seam/EG-KG.backend.real-cuda-tensor-backend CUDA distance (eg-ann) + elementwise (eg-tensor) backends were correctness-matched to CPU but never run on a device (no GPU in CI). A GPU-gated parity test in each crate (eg351_cuda_batch_distance_matches_cpu_ground_truth, eg351_cuda_elementwise_matches_cpu_ground_truth, #[cfg(feature = "gpu-cuda")]) detects a device at runtime via cuda::backend(): when a GPU is present it asserts the real cudarc kernel == the CPU ground truth (f32 relative tolerance for the accumulating distance kernel, BITWISE-exact for the single-op f64 elementwise kernel); when none is present it SKIPS cleanly (eprintln! + early return). So it compiles under --features gpu-cuda, no-ops in GPU-less CI, and auto-validates wherever a GPU exists (e.g. the GB10 box) without breaking CI. Pi contract untouched (cargo tree --features pi \| grep -ciE 'cudarc\|pyo3' = 0). Extends AU-KG.compute.is-installed-kernel-discovery/EG-327.
CONCEPT:EG-KG.query.eg-3 Cross-modal join→analytics in-engine (the numpy-surpassing differentiator) The Analytics-Program P4 differentiator: join graph + vector + timeseries, then run pca/kmeans/covariance over the JOINED result set IN-ENGINE (compute-near-data, no fetch-to-Python, no data-layer round-trip) — impossible in numpy, which has no data layer to join across. One SQL statement joins the resident nodes table (graph/relational modality + a per-node vector embedding prop) to a per-node timeseries aggregate (AVG(reading) over a readings relation) on node id, and computes: kmeans(emb, k) clustering the joined vectors, pca(emb, k) over the joined vectors, and covariance(graph_scalar, timeseries_avg) — a statistic that spans TWO modalities aligned by the join. Proven end-to-end by crates/eg-query/tests/cross_modal_analytics.rs on synthetic data with hand-computed results (cross-modal cov = 7.0; two balanced k-means clusters; PC1 = ±[1/√2,1/√2]). Builds on EG-KG.query.kmeans-clustering-half-one/335/336/329. See architecture/numeric-kernel.md + architecture/analytics-program.md.
CONCEPT:AU-KG.compute.is-installed-kernel-discovery eg-numeric Surface-A pyo3 wheel + numpy-parity CI gate The Surface-A (AU-KG.compute.numeric-kernel) Python extension shipped as an installable pyo3 cdylib wheel so agent_utilities.numeric.xp runs kernel-LIVE (AU-KG.compute.shim-goes-kernel-live), not just the numpy fallback. Built with maturin build --release -m crates/eg-numeric/Cargo.toml --features python — the pymodule is numeric with __kernel__ = "eg-numeric", discovered by the shim as epistemic_graph.numeric (folded build) or top-level numeric (standalone wheel). On a Python newer than pyo3-0.22's max (e.g. 3.14) set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to build anyway; on ≤3.13 no flag is needed. This is a SEPARATE cdylib wheel from the bindings="bin" server facade, so scripts/check_no_pyo3.sh (which guards the facade + src/, never crates/) and the Pi contract still hold — cargo tree --features pi \| grep -ci pyo3 = 0. Gated by the numeric-parity CI job (.github/workflows/rust-ci.yml): it maturin-builds the wheel, installs it, and runs crates/eg-numeric/tests/test_kernel_parity.py — a self-contained corpus asserting every kernel op np.allclose its numpy reference across the full op-surface incl. edge cases (nan/inf, singular matrix, empty), so CI FAILS if the Rust kernel diverges from numpy. See architecture/numeric-kernel.md. Extends EG-321.
CONCEPT:EG-KG.compute.tensor-gpu-distance eg-numeric folded into the epistemic-graph wheel + GPU-gated CUDA parity test Two closes: (A) the numeric kernel ships in ONE package — epistemic-graph[numeric]. The AU-KG.compute.is-installed-kernel-discovery Surface-A wheel was built + numpy-parity-gated in CI but there must be only ONE published binary (a separately-published eg-numeric==0.1.0 re-upload also aborted the whole twine upload dist/* batch and starved the epistemic-graph upload). So the kernel .so is now FOLDED INTO the epistemic-graph node wheel as epistemic_graph/numeric.abi3.so: the wheels job in .github/workflows/release-build.yml builds the pyo3 cdylib (maturin build -m crates/eg-numeric/Cargo.toml --features python) for the same target and injects its .so into the server wheel via scripts/inject_numeric_kernel.py (recomputing RECORD). pip install epistemic-graph[numeric] then provides epistemic_graph.numeric (the [numeric] extra adds numpy for interop) — the module the AU xp shim probes (AU-KG.compute.shim-goes-kernel-live). The old wheels-numeric + sdist-numeric publish legs are REMOVED; publish-pypi uploads only the epistemic-graph wheels + sdist via the same token twine upload --skip-existing dist/* (tag-gated, unchanged). The AU-KG.compute.is-installed-kernel-discovery numeric-parity gate in rust-ci.yml still builds/imports the kernel from the crate — it does not publish. (B) the CUDA kernels get LIVE GPU validation. The EG-KG.compute.gpu-distance-seam/EG-KG.backend.real-cuda-tensor-backend CUDA distance (eg-ann) + elementwise (eg-tensor) backends were correctness-matched to CPU but never run on a device (no GPU in CI). A GPU-gated parity test in each crate (eg351_cuda_batch_distance_matches_cpu_ground_truth, eg351_cuda_elementwise_matches_cpu_ground_truth, #[cfg(feature = "gpu-cuda")]) detects a device at runtime via cuda::backend(): when a GPU is present it asserts the real cudarc kernel == the CPU ground truth (f32 relative tolerance for the accumulating distance kernel, BITWISE-exact for the single-op f64 elementwise kernel); when none is present it SKIPS cleanly (eprintln! + early return). So it compiles under --features gpu-cuda, no-ops in GPU-less CI, and auto-validates wherever a GPU exists (e.g. the GB10 box) without breaking CI. Pi contract untouched (cargo tree --features pi \| grep -ciE 'cudarc\|pyo3' = 0). Extends AU-KG.compute.is-installed-kernel-discovery/EG-327.
CONCEPT:EG-KG.compute.capability-reference pgwire in node/full tier + wheel-install Docker image Two build changes making the published single-node wheel a COMPLETE Postgres-wire DB, and the release image install-not-compile. (A) pgwire folded from cluster-only into the node + full feature tiers (cluster now inherits it via node; the redundant explicit pgwire is dropped from the cluster list). pgwire is pure-Rust (pgwire + futures over the wire-neutral wire core, CONCEPT:EG-KG.compute.subsystems-reference) and the SINGLE-NODE node tier already links query/DataFusion (+ ring via federation), so it adds NO new heavy/C dep CLASS to node — a stock node wheel is therefore a COMPLETE Postgres-wire-capable single-node DB (no raft). The Pi contract is preserved: pgwire IMPLIES query (DataFusion) and pulls the _ring crypto backend, both forbidden on pi/pi-max, so it can never enter a Pi build — proven byte-for-byte identical cargo tree -e normal for pi/pi-max before vs after the move (pgwire/datafusion crate count in pi = 0, in pi-max = 0 for pgwire). (B) the release Docker image installs the prebuilt wheel. docker/Dockerfile switched from a two-stage cargo build --release (~25 min/arch) to a single python:3.11-slim + uv pip install --system of the published node-tier wheel (the agents/* fleet standard) — uv auto-selects the per-platform manylinux wheel so one docker buildx --platform linux/amd64,linux/arm64 is multi-arch clean with no Rust toolchain, seconds not minutes; EG_INSTALL_SPEC allows a local-wheel sanity build, and the in-build epistemic-graph-server --help is the smoke. release-build.yml's docker-image + publish-image now needs: publish-pypi, are tag-gated, and thread the tag-derived version via an EG_VERSION build-arg. Extends AU-KG.query.raw-python/EG-074.
CONCEPT:EG-KG.query.concept-8 Analytics-through-UQL on ALL SQL paths The "analytical database with native analytical tools" story, closed on EVERY SQL entry point: the eg-numeric-backed Surface-B operators (cosine_sim/l2_normalize/zscore/covariance/svd/pca/kmeans, EG-KG.query.surface-b-numeric-operators/335/336/344) run IN-ENGINE over resident columns (compute-near-data, no numpy round-trip) not just on the native RPC Method::Sql path (exec_sql) but on the WIRE SQL path too — a DBeaver/psql/DBI client's SELECT pca(...) hits them because pgwire/mysql/sqlite/mssql all drive the shared WireSessionexec_sql_typed_with_tablesrun_typedbuild_ctx, and register_numeric is called ONCE inside build_ctx (plus on the obs-tables exec_sql_over_tables context), so there is a single shared registration site and no privileged path. AUDIT finding: all paths were ALREADY covered by that shared build_ctx site (nothing needed re-wiring). GAP finding: the common statistical aggregates (corr/stddev/var/median/approx_percentile_cont/covar_samp/covar_pop) are DataFusion built-ins ALWAYS registered on every context (verified live), so no kernel-backed duplicates were added — the kernel earns its place only for what DataFusion has no built-in for (PCA/SVD/k-means/cosine). Proven end-to-end by crates/eg-query/tests/analytics_through_uql.rs: every operator resolves through the wire read path AND the RPC path with identical results, plus a cross-modal JOIN→analytics query (EG-KG.query.eg-3) through the wire path. Gated numeric (out of pi: cargo tree --features pi \| grep -ciE 'eg-numeric\|faer\|ndarray' = 0). Docs: analytics_in_uql.md. Extends EG-KG.query.surface-b-numeric-operators/EG-345.
CONCEPT:EG-KG.query.eg-feature SQLite .db file IMPORT Method::ImportSqliteFile { path } reads every user table (+ its rows) from a real on-disk sqlite3 .db file into the engine's process-global user-table store (eg_query::TableStore — the SAME store the Method::Sql DDL/DML + pgwire paths write, so an imported table is instantly visible to SELECT … FROM <table>). The documented EG-KG.query.concept-3 follow-up, DISTINCT from the sqlite-wire NDJSON dialect surface. Each column's declared type maps by SQLite affinity → an engine ColumnType (INTEGER→BigInt, REAL→Double, TEXT→Text, BLOB/none→Bytes, numeric→Double); columns import NULLABLE + non-PK so exact values (incl. NULL/BLOB) pass through coercion (constraints are not mirrored, VALUES are). A same-name table is REPLACED (drop-then-recreate) so the import mirrors the file. BATCH: ONE insert_rows per table, never per-row. Self-routed in dispatch (file-scoped, like Blob/Kv), run on the blocking pool. Gated sqlite-file, which pulls rusqlite with the BUNDLED C sqlite3 (no pure-Rust SQLite-file writer exists) — kept OUT of pi/default, folded into full/node (Pi contract: cargo tree --features pi links no rusqlite/libsqlite3-sys). src/server/handlers/sqlite_file.rs + client.query.import_sqlite_file(). The EG-KG.query.concept-3 .db-file follow-up.
CONCEPT:EG-KG.query.full-protocol SQLite .db file EXPORT Method::ExportSqliteFile { path, tables } writes selected user tables OUT to a FRESH, valid sqlite3 .db file a stock sqlite3 CLI can open. tables empty ⇒ every user table; else exactly the named tables (each must exist). Any pre-existing file (+ stray -wal/-shm/-journal siblings) is removed first so the export is a clean database. Each engine ColumnType maps back to a SQLite declared type (INTEGER/REAL/TEXT/BLOB) so a re-import round-trips; each Cell → a SQLite runtime value. BATCH: ONE scan per table (engine round-trip), then a single bulk sqlite transaction of inserts. The inverse of EG-KG.query.eg-feature, sharing the same handler/feature/gating (rusqlite bundled C sqlite3, out of pi, in full/node). Round-trip proven by test_sqlite_file_roundtrip_eg331_eg332 (re-opens the exported file with the sqlite library — proving sqlite3-readability — and asserts every value incl. a NULL + a BLOB survived). src/server/handlers/sqlite_file.rs + client.query.export_sqlite_file(). The EG-KG.query.concept-3 .db-file follow-up.
CONCEPT:EG-KG.domains.raster-build Raster tile pyramid build The raster complement to EG-KG.domains.map-tiles's vector-tile (MVT) server: eg_geo::raster models a georeferenced coverage as a Raster (a Web-Mercator EPSG:3857 Bbox + a width×height × bands pixel-interleaved, row-major byte grid, row 0 = north; serde-serialisable so a coverage persists as a typed value in the redb per-graph store, like a Geometry) — e.g. an ingested GeoTIFF/GeoParquet coverage decoded to bands. Raster::build_pyramid(z_min, z_max, opts) emits the full XYZ tile pyramid: for each zoom it computes the inclusive tile-index range intersecting the coverage (tile_range, exclusive east/south edges via an epsilon shrink so a boundary-aligned extent does not spill into the next tile) and resamples each Tile via EG-KG.domains.raster-fetch, returning a Pyramid { tiles: Vec<(Tile, RasterTile)>, counts } (per-zoom tile counts). Pure eg-geo library capability (like encode_mvt), reachable from the geo-tier executor / a geo serving surface. Reuses EG-KG.domains.map-tiles Tile::bounds/ORIGIN_SHIFT + EG-KG.domains.geo-registry Web-Mercator. crates/eg-geo/src/raster.rs. Extends EG-265.
CONCEPT:EG-KG.domains.raster-fetch Raster tile fetch + hand-rolled PNG codec The per-tile half of EG-KG.domains.raster-build: Raster::tile(z, x, y, opts) fetches one XYZ tile — a TILE_SIZE×TILE_SIZE (256²) RasterTile with the source band count, nearest-neighbour-resampled from the coverage over the tile's Web-Mercator bounds (correctly downsamples as z decreases, upsamples as it increases; pixels outside the coverage take ResampleOptions.nodata, transparent for RGBA). RasterTile::to_png() (and the free encode_png/decode_png) is a hand-rolled, dependency-free PNG codec for 8-bit grayscale/GA/RGB/RGBA — stored (uncompressed) DEFLATE blocks in a zlib wrapper + hand-computed CRC-32 (chunk) and Adler-32 (zlib), filter-0 scanlines. The same "no codegen, no C" ethos as EG-KG.domains.map-tiles's hand-rolled MVT protobuf: NO image/png/flate2, zero new dependencies, so eg-geo stays trivially inside the Pi contract (it is already out of the pi tier; cargo tree --features pi links no image/png/eg-geo). Raw band tiles are also available directly via RasterTile::data. Round-trip proven by raster::tests (pyramid tile counts 1/4/16 over z=1..3, a known green/red band value in a tile, and a PNG encode→decode identity incl. a multi-block grayscale tile). crates/eg-geo/src/raster.rs. Extends EG-KG.domains.map-tiles/EG-338.
CONCEPT:EG-KG.query.real-pgvector-ann-top pgvector real ANN top-k pushdown Push ORDER BY col <-> $1 LIMIT k down to the eg-ann HNSW/IVF index (real top-k) instead of the brute-force fallback, when a matching CREATE INDEX USING hnsw/ivfflat (EG-KG.query.real-ann-top-k) exists. eg-query/pgwire + eg-ann. Extends EG-116.
CONCEPT:EG-KG.ingest.broker-reject-publish Broker exactly-once + AMQP/MQTT frame exposure Idempotent-producer dedup (producer id + sequence → drop duplicate publishes) for effectively-exactly-once, plus expose the EG-283/284 stream/confirm/ack ops over the AMQP/MQTT wire frames (today reachable only via engine Methods). eg-core/broker + amqp_wire/mqtt_wire. Extends EG-275/283/284.
CONCEPT:EG-KG.storage.path-index-store Durable JSONPath index persistence Persist the EG-084 inverted JSONPath index (path→ids) to redb + rehydrate at boot (in-memory today), and hook JSON-index counts into the planner cost Stats for selectivity. eg-core. Extends EG-084.
CONCEPT:EG-KG.query.schema-typed-fusion-sql Federated typed SQL/SPARQL result fusion Extend EG-KG.ontology.federation-client federated search with typed result fusion for SQL + SPARQL partials (schema-aware column union + typed dedup/merge, not just hashed-key union), so cross-instance SQL/SPARQL results combine correctly. server/federation. Extends EG-243.
CONCEPT:EG-KG.query.rename-table-moves-catalog ALTER TABLE beyond ADD COLUMN DROP COLUMN, RENAME COLUMN, RENAME TO, ALTER COLUMN TYPE on the durable user-table catalog (EG-KG.query.register-user-tables-alongside) with data migration, + DROP CONSTRAINT. eg-query/tables. Extends EG-018.
CONCEPT:EG-KG.query.bm25-ranking-snippets ParadeDB BM25 real ranking + snippets Real BM25 relevance scoring + highlighted snippets via the eg-text index behind the EG-KG.query.paradedb-bm25 @@@/paradedb.score()/snippet() surface (placeholder 1.0 today). eg-text (+ minimal eg-query lowering). Extends EG-119.
CONCEPT:EG-KG.domains.geo-partitioning Routing turn-restrictions + time-windows Extend EG-KG.domains.geo-routing routing with turn-restriction penalties (via edge/turn cost), one-way already supported, and time-window/time-dependent edge weights (cost as a function of departure time) for realistic logistics routing. eg-geo. Extends EG-266.
CONCEPT:EG-KG.compute.durable-rbac-identity-persistence Durable RBAC/identity persistence Persist the EG-KG.compute.feature RBAC roles/grants + agent identities to redb (in-memory today), so policy survives restart; load at boot, write-through on RbacAdmin mutations. eg-core (isolation + persistence). Extends EG-092.
CONCEPT:EG-KG.storage.derived-tensor-writeback-sink Tensor-op CAS write-back Persist derived tensors from Op::TensorOp/Op::TensorScan results into the EG-085 content-addressed tensor store (write-back on the exec path), so computed tensors are durable + dedup-shared. eg-tensor + eg-plan exec. Extends EG-085.
CONCEPT:EG-KG.ontology.iri-template-object-map OBDA full R2RML Turtle parse Parse standard R2RML mapping documents in Turtle (rr:TriplesMap/rr:logicalTable/rr:subjectMap/rr:predicateObjectMap/rr:template/rr:column) into the EG-101 VirtualGraph model, so a real R2RML file drives an OBDA virtual graph. eg-rdf. Extends EG-101.
CONCEPT:EG-KG.domains.geo-formats Geospatial format I/O: Shapefile/KML/GeoParquet Reader/writer for ESRI Shapefile (.shp/.dbf/.shx), KML/KMZ, and GeoParquet — round-tripping eg-geo geometries + attributes, completing the map-data ingest/export matrix alongside GeoJSON/WKB/GPX (EG-KG.domains.geojson-gpx-formats). eg-geo. Extends EG-264.
CONCEPT:EG-KG.txn.pubsub-transactions Redis pub/sub + S3 multipart completeness Redis SUBSCRIBE/PSUBSCRIBE/PUBLISH/UNSUBSCRIBE pub-sub + MULTI/EXEC transactions on the RESP wire (EG-KG.ontology.resp2-resp3-codec-round), and S3 multipart upload (Create/Upload-Part/Complete/Abort) + range GET on the S3 surface (EG-KG.ontology.object-put-get-head). server/redis_wire + server/s3. Extends EG-KG.ontology.resp2-resp3-codec-round/176.
CONCEPT:EG-KG.query.gds-call-procedures GDS algorithms over Cypher CALL gds.* Wires the EG-144 graph-data-science library (PageRank/Louvain/WCC/SCC/betweenness/Dijkstra/similarity) into the Cypher surface as CALL gds.<algo>(...) YIELD ..., projecting the current graph into the eg-compute adjacency + streaming results as Cypher rows. eg-query/cypher. Extends EG-144.
CONCEPT:EG-KG.query.gds-procedure-routing GDS breadth: labelPropagation/knn/dbscan/linkPrediction Track I (Cypher-25/GDS competitor-feedback fold-in): four more CALL gds.* procedures on the EG-KG.query.gds-call-procedures registry, each a thin binding (project → parse config → run kernel → YIELD rows), no algorithm reimplemented. gds.labelPropagation/gds.knn route to NEW always-on graph_algos kernels (EG-KG.compute.label-propagation; knn_similarity beside the existing all_pairs_similarity) — no new Cargo feature. gds.dbscan routes to the existing eg_compute::mining::cluster::dbscan (EG-KG.mining.dbscan-density) over a nodeProperty feature vector, gated behind the new cypher-mining eg-query feature (implies eg-compute/mining). gds.linkPrediction routes to the existing eg_compute::graphlearn::link_predict KAN link-predictor (EG-KG.graphlearn.link-predictor) — a fit-then-predict workflow folded into one CALL, unlike every other stateless GDS proc here — gated behind the new cypher-graphlearn eg-query feature (implies eg-compute/graphlearn). Both new features are OUT of the base cypher feature (Pi-lean posture, mirroring finance/numeric). crates/eg-query/src/cypher/gds.rs, crates/eg-compute/src/graph_algos/{label_propagation.rs,similarity.rs}. Extends EG-KG.query.gds-call-procedures.
CONCEPT:EG-KG.query.quantified-path-pattern Cypher 25 quantified path patterns (QPP) ((a)-[:REL]->(b)){min,max} — repeats a WHOLE inner sub-pattern (not just one relationship, generalizing *min..max) min..max times. EdgePat gains an optional group: Option<Box<QuantifiedGroup>> (a synthetic hop wrapping the inner Pattern + quantifier); the parser disambiguates a group's (( open from a plain node's ( by one token of lookahead; execution factors the existing hop-walker out into walk_hops and reuses it recursively for group_reachable/expand_group_once (a BFS over "meta-edges", each one full application of the inner pattern) — so nested groups and multi-hop inner patterns fall out for free. Honest scope (deferred): only the group's overall reachability + the FINAL repetition's end node participate in the outer MATCH/WHERE/RETURN scope — per-iteration variable bindings inside the group are not exposed as Cypher-25's full list values. CREATE rejects a group hop (no well-defined write semantics). crates/eg-query/src/cypher/{plan.rs,parser.rs,exec.rs}.
CONCEPT:EG-KG.query.protocol-types Live CEP standing-query subscription surface A server surface (Method + subscription stream) over the EG-KG.query.pipelined-execution live-CEP standing-query engine: register a CEP pattern, subscribe, and receive pushed matches fed by the EG-064 CDC broadcast bus. server + eg-stream. Extends EG-088.
CONCEPT:EG-KG.ontology.rdf-update-guard ICV write-path enforcement Wires EG-KG.ontology.wired-into-commit-write integrity-constraint-validation into the commit/write path: a guard evaluates the proposed change set against registered SHACL-as-constraints and REJECTS a transaction that would introduce a violation (constraint-enforced transactions), configurable enforce/warn. eg-rdf/eg-shacl + commit hook. Extends EG-146.
CONCEPT:EG-KG.retrieval.hnsw-vector-index HNSW vector index A hierarchical-navigable-small-world graph index in eg-ann for higher recall-per-probe than IVF-PQ, with insert/search/serde-persist + the EG-KG.query.concept-5 recall harness as the tuning signal. eg-ann. Extends EG-KG.retrieval.scatter-gather/297.
CONCEPT:EG-KG.query.bottomk-selection PromQL extended function set Extends the EG-172 PromQL evaluator with _over_time family (sum/avg/min/max/count/stddev/quantile over range), delta/idelta/deriv, topk/bottomk/quantile, label_replace/label_join, and clamp*. eg-tsdb. Extends EG-172.
CONCEPT:EG-KG.query.concept-5 Exact/flat vector index + recall harness Alongside the EG-KG.retrieval.scatter-gather IVF-PQ ANN: a brute-force exact kNN index (correct ground truth for small sets + a re-rank stage over ANN candidates for high precision), a hybrid re-rank combining ANN recall with exact-distance refinement, and a recall@k / precision self-evaluation harness (ANN vs exact ground truth). eg-ann. Extends EG-KG.retrieval.scatter-gather/070.
CONCEPT:EG-KG.enrichment.cross-modal-enrichment-hook VRL-style ingest pipelines OpenObserve/Vector-style transform pipelines applied at log/event ingest: a small pure-Rust pipeline DSL (parse/json-extract, filter/drop, set/rename/remove fields, coerce types, route-to-stream) compiled to a staged executor run over incoming records before they land in the tsdb/log store. eg-tsdb. Surpasses OpenObserve VRL by being cross-modal (can enrich from the graph).
CONCEPT:EG-KG.query.prometheus-http-query-api PromQL + Prometheus HTTP query API A PromQL evaluator (instant/range vectors, selectors, rate/sum/avg/max/min/histogram functions, binary ops) over the eg-tsdb metric series, exposed as a Prometheus-compatible /api/v1/query + /api/v1/query_range HTTP API on the obs listener. Gated obs. Depends on AU-KG.ingest.self-ingest/tsdb.
CONCEPT:EG-KG.query.core-query-input NL→query seam (NlPlanner) An engine seam that turns a natural-language string into a wire::Plan/UQL via an injected NlPlanner trait (feature nl-query, reuses ureq to call an OpenAI-compatible endpoint); the generated Plan executes through the existing deterministic pipeline, keeping the engine pure-Rust + LLM-optional. eg-plan.
CONCEPT:EG-KG.query.fence-stripper NL→query standalone + HTTP/UDF surface Method::NlQuery{text,graph} + a /nl HTTP route + a nl_query('…') SQL UDF entry; standalone config from agent-utilities config.json (endpoint/model/key-env; create if absent). Depends on EG-078.
CONCEPT:EG-KG.compute.json-deep-indexing Document/JSON deep indexing Deep JSONPath query + a durable inverted path-index (path→ids) in eg-core maintained on the mutation path; eg-query lowers Postgres ->/->>/@>/jsonb_path_query + a Mongo-style $match onto a new Pred::JsonPath; the planner uses the path index for selectivity.
CONCEPT:EG-KG.ontology.concept-10 GeoSPARQL baseline OGC GeoSPARQL over the SPARQL surface: the geo:/geof: vocab, WKT/GML geometry literals, and geof:sfWithin/sfIntersects/distance FILTER functions lowered onto the EG-KG.ontology.singles-concept/258 spatial predicates (eg-geo). eg-rdf. The baseline that EG-KG.ontology.concept-7 (RCC8/Egenhofer families) extends.
CONCEPT:EG-KG.query.pipelined-execution Event-stream + complex event processing New pure-Rust crate eg-stream (reuses tokio already pulled by server) for high-velocity windowed ingestion + CEP. Append events to the eg-tsdb columnar store (EG-KG.query.streaming-execution Op::Window is the windowing primitive); add Op::Cep { pattern } — a bounded NFA over a sliding/tumbling window (sequence/within/absence operators) emitting matches as a RowSet. The EG-064 CDC broadcast bus feeds live windows; standing CEP queries subscribe. Gated stream, folded into node/full, out of pi. Depends on EG-KG.query.streaming-execution, EG-064.
CONCEPT:EG-KG.sharding.reshard-on-restore Online backup / restore + PITR Consistent online backup: a Database::begin_read() MVCC snapshot per shard (EG-KG.storage.snapshot-read-off-writer) → a portable backup bundle reusing EG-030's verbatim raw-row copy (so encryption-at-rest + the EG-KG.sharding.row-level-security audit chain survive), a restore CLI, and Method::Backup/Restore admin RPCs (mirror the EG-038 admin surface). Point-in-time recovery rides the durable ledger/WAL replay. Redb-only; clean "not available" on non-redb builds. Foundation for the DR / low-RPO-RTO story.
CONCEPT:EG-KG.storage.content-addressed-dedup Array / tensor store New pure-Rust leaf crate eg-tensor (hand-written N-D array or pure-Rust ndarray behind the feature; NO BLAS/C in default). Stores dense/chunked N-D arrays (images/sensor frames/genomics/ML features) content-addressed in the existing blob CAS (reuse ChunkStore + EG-KG.storage.backward-manifest-read CDC), with a dtype/shape manifest as a node property. Adds Op::TensorScan { layer } + Op::TensorOp { kind } (slice/reduce/elementwise) to the eg-types::wire algebra (pure-serde, Pi-safe), executed in eg-plan/exec.rs behind a tensor feature. Distinct from eg-ann (fixed-D ANN vectors). Folded into node/full, out of pi.
CONCEPT:EG-KG.ontology.concept-4 SPARQL builtin-function library completion Extend the FILTER/expression evaluator (eg-rdf/src/sparql.rs, on top of EG-KG.ontology.rich-filter) with the remaining SPARQL 1.1 builtins: term constructors (IRI/URI,BNODE,STRDT,STRLANG,UUID,STRUUID), hashes (MD5/SHA1/SHA256/SHA384/SHA512 via RustCrypto already in-tree), date-time (NOW,YEAR..SECONDS,TIMEZONE/TZ), numeric (RAND,ABS,ROUND,CEIL,FLOOR), and string extras (STRBEFORE/STRAFTER/REPLACE/ENCODE_FOR_URI,LANGMATCHES,SAMETERM).
CONCEPT:EG-KG.ontology.concept-5 RDF-star / SPARQL-star (RDF 1.2) Quoted-triple term << s p o >> in the value model + star parsing/eval + annotation syntax {\| p o \|}, for statement-level metadata (reification's successor).
CONCEPT:EG-KG.ontology.feature RDF serialization matrix Reader/writer coverage beyond N-Triples/Turtle: N-Quads + TriG (quads), RDF/XML, JSON-LD 1.1 (contexts/expansion), and RDFa parse — in eg-rdf/src/mapping.rs, reusing oxrdf/oxttl where available (add pure-Rust oxrdfxml/oxjsonld only behind the rdf feature, out of pi).
CONCEPT:EG-KG.ontology.singles-concept Spatial / geospatial modality New pure-Rust leaf crate eg-geo (geo/geo-types + an in-house packed Hilbert R-tree; NO GEOS/PROJ C deps). Geometries (point/line/polygon) persist as a typed value in the redb per-graph store behind a geo feature; a durable R-tree index rides the same shard. Adds Pred::SpatialWithin/SpatialDWithin + source Op::SpatialScan { layer, bbox } to the eg-types::wire algebra (pure-serde, Pi-safe), executed in eg-plan/exec.rs, so a spatial filter composes with Traverse/Rank/Filter in one plan. SQL st_within/st_distance + a GeoSPARQL subset surface it. Folded into node/full, out of pi.
CONCEPT:EG-KG.retrieval.scatter-gather Cross-shard kNN scatter-gather A vector kNN fans out across shards/Raft groups and merges the per-shard top-k into a global top-k (server-layer scatter-gather over eg-ann indexes), replacing single-shard-only search.
CONCEPT:EG-KG.retrieval.hybrid-metadata-prefilter Hybrid metadata pre-filter for ANN Push a graph/SQL predicate INTO the ivfpq::search scan (a candidate-id allowlist / predicate arg) so filtering happens during the ANN probe instead of over-fetch-then-post-filter.
CONCEPT:EG-KG.ontology.order-by-values-exists SPARQL ORDER BY / VALUES / EXISTS execution Add the GraphPattern::OrderBy arm (sort solutions by the OrderExpression list, ASC/DESC) — a correctness fix (results are currently returned UNORDERED) — plus VALUES (inline data) and EXISTS/NOT EXISTS filter evaluation in eg-rdf/src/sparql.rs.
CONCEPT:EG-KG.storage.backward-manifest-read Content-defined blob chunking FastCDC/Gear rolling-hash chunker replacing fixed 2 MiB chunks, preserving the sha256 CAS dedup + refcount GC; variable chunk boundaries in BlobManifest.
CONCEPT:EG-KG.sharding.per-graph-write-coalescer Per-Graph Write Coalescer Concurrent single-op writes to ONE hot graph (the __commons__ ingestion firehose) batch onto a lazily-created per-graph writer (src/write_coalescer.rs) and apply under ONE topo.write() per batch — collapsing N lock acquisitions into ⌈N/batch⌉. Writers are keyed by graph name in a DashMap (auto per new graph/connector, no hardcoded list); dirty/WAL/gauge side-effects stay in the dispatch shell so durability and checkpoint contracts are unchanged. Default ON, batch auto-sized from cpu count; opt out with EPISTEMIC_GRAPH_WRITE_COALESCE=0. See write_coalescer.md.
CONCEPT:EG-KG.txn.multi-op-occ-acid Multi-op OCC ACID Transactions Optimistic, snapshot-isolation, server-staged transactions (src/server/txn.rs + handlers/txn.rs). BeginTxn returns a server-issued txn_id; Txn{AddNode,RemoveNode,AddEdge,RemoveEdge,Cas} STAGE durable mutations into a server-held write-set (nothing touches the graph or persistence until commit). Commit takes topo.write() ONCE — the serialization point — validates the OCC read-set (per-GraphCore AtomicU64 version + per-node fingerprints), applies the staged write-set atomically through ONE GraphTxn, bumps the version, and records each staged method through the configured PersistenceBackend; it returns Bool(false) on conflict (a true rollback — nothing applied). Rollback discards the staged state. A long-open txn never holds topo.write(); an idle-TTL sweep auto-rolls-back abandoned txns, and per-graph/per-agent open-txn caps bound memory. Staged ops bypass the write coalescer (no deadlock). Single-op CAS auto-commit is unchanged.
CONCEPT:EG-KG.query.dep-free-behind Dep-free Cypher query surface Read-only MATCH (a:Label)-[:REL]->(b:Label2) WHERE a.prop = 'x' RETURN a, b LIMIT k over ONE graph, behind the facade cypher feature. A hand-written recursive-descent parser (crates/eg-query/src/cypher/) compiles the subset to the engine's OWN primitives — label predicates → the eg-core label index, fixed-shape multi-hops → a synthesized pattern GraphView fed to vf2_match_views, variable-length paths *m..n → petgraph BFS — with NO DataFusion, so it ships in the lean Pi tier (pi feature). Routed through handlers/query.rs (Method::CypherQuery), returning the same QueryResult carrier as Sql.
CONCEPT:EG-KG.coordination.distributed-cache-coherence Caching hierarchy + result cache Three layers, all Pi-aware. (1) Version-keyed query-RESULT cache (result-cache feature): a bounded, pure-Rust LRU on GraphCore (crates/eg-core/src/result_cache.rs) keyed by (query-hash, version()) caches the serialized bytes of a read query (Sql/Cypher/Sparql/UnifiedQuery). A repeated identical query on an UNCHANGED graph HITS (no recompute); ANY write bumps the OCC version() (KG-2.180) → the next lookup keys on a new version → MISS → recompute, so staleness is impossible by construction. analysis_snapshot_versioned() pairs the snapshot + version atomically under the topo read lock; clear/hibernate invalidate directly. Pure LRU + 128-bit hash, no new dep → folded into pi/node/cluster/full. (2) CDC-driven distributed cache-coherence (cache_coherence over the streaming CDC feed, EG-KG.query.streaming-cdc-subscriptions): a replica tailing a peer's CDC feed calls GraphCore::invalidate_for_remote_change per event, so a write on A retires B's cached result for that graph. (3) Cold object-store tier seam (cold-tier default + cold-tier-s3): a ColdTier trait (RAM→redb→object-store, crates/eg-core/src/cold_tier.rs) with an in-memory/redb default and an S3 impl, so a cold graph offloads its whole serialized state and rehydrates on access — extending hibernation (EG-KG.storage.100m-tenant).
CONCEPT:EG-KG.sharding.row-level-security Engine-level security: per-agent RLS + encryption-at-rest + hash-chained audit (security feature, Lane O) Three pure-Rust security primitives, folded into node/cluster/full (OUT of bare default + the lean pi tier). (1) Per-agent Row-Level Security (crates/eg-core/src/isolation.rs IsolationLayer::filter_view): the read/plan-path GraphView snapshot is filtered down to the rows the caller may see (via the _owner/_visibility/_grants node-property convention + owner/grant/manager-of/System resolution) BEFORE it reaches any query surface (SQL/Cypher/SPARQL/unified), so no surface can exfiltrate a forbidden row. No-op until the first RegisterIdentity (single-tenant back-compat). (2) Encryption-at-rest (src/crypto.rs, RustCrypto ChaCha20-Poly1305 — pure-Rust, NO ring/openssl; threaded through src/redb_store.rs/redb_backend.rs/embedded/store.rs): the redb durable VALUE blobs are AEAD-sealed under EPISTEMIC_GRAPH_ENCRYPTION_KEY (keys stay plaintext so range scans work); a wrong key fails the read (never silent plaintext). Opt-in (changes the on-disk format). (3) Hash-chained tamper-evident audit log (src/audit.rs + an AUDIT redb table): every durable mutation links into a sha2 hash chain; Method::AuditVerify walks it and reports OK or the first break (AuditReport). RLS threaded into dispatch.rshandlers/query.rs + handlers/rdf.rs. Reconciled with the result cache (KG-2.233): the result-cache key folds in the caller's RLS context (rls_cache_hash — agent_id-salted when has_rules()), so agent A's RLS-filtered cached result is never served to agent B for the same query text.
CONCEPT:EG-KG.query.query-federation Query federation / foreign sources (federation feature, Lane P) A federated UnifiedQuery reads an EXTERNAL source as a RowSet and composes it with the local graph/vector/SQL ops in ONE plan, no Python round-trip. crates/eg-plan/src/federation.rs defines the ForeignSource trait + two impls: RemoteEngineSource (a remote epistemic-graph engine over the SAME length-prefixed-MessagePack + HMAC transport) and HttpJsonSource (a generic HTTP/JSON API). A new Op::ForeignScan { source, join } plan op runs inline inside the unified-query executor; Method::RegisterForeignSource { name, source } records a named ForeignSourceSpec in the process-global foreign_sources registry on ServerState (handlers/federation.rs) for reuse by name. Pulls eg-plan's federation (the rustls HTTP client ureq + hmac/sha2/hex), which IMPLIES query; OFF by default and folded into node/cluster/full — NEVER pi, so a default/pi build links no ureq/rustls/ring (the Pi contract holds). Proven end-to-end by a two-in-process-engine test: engine A ForeignScans engine B over TCP, joins B's rows with A's local graph, ranks + limits, and the fused result equals the manual join.
CONCEPT:EG-KG.txn.serializable-zero-cost Transaction isolation levels BeginTxn carries an isolation level (M6b); the OCC commit path validates the read-set under the requested level so a txn opts into the consistency it needs.
CONCEPT:AU-KG.ontology.owl-screen-bridge AT-SPI accessibility view Exposes AT-SPI accessible elements as a structural view the engine can ingest/query.
CONCEPT:EG-KG.backend.authoritative-dispatch Commit-before-ack durability A durable mutation is committed to redb (group-commit fsync) BEFORE its Response is acked; a commit failure becomes an ERROR response, so an acked write is always on disk. Read once at startup into ServerState.redb_authoritative.
CONCEPT:AU-KG.ingest.source-sync-canonical In-engine Raft replication Cluster-tier raft feature: the engine runs as a multi-node HA cluster replicating its authoritative redb state via openraft 0.9; durable mutations route through Raft consensus before apply+ack, with automatic leader failover. Off ⇒ the write path is byte-for-byte the single-node path.
CONCEPT:AU-KG.query.raw-python Postgres wire-protocol shim A pg-wire front-end (pgwire facade feature) that lets standard Postgres clients speak to the engine's SQL surface over the wire.
CONCEPT:EG-KG.storage.read-through-seam-exercised Read-through-safe eviction Under authoritative mode the per-graph node cap resumes enforcing (bounded memory) WITHOUT data loss: a ReadThrough seam (crates/eg-core/src/read_through.rs) serves an evicted node's stored blob from redb on a RAM miss, and a node is dropped from RAM only after a redb read confirms it is on disk.
CONCEPT:AU-KG.backend.backend-modes redb-authoritative default (THE FLIP) Built with the redb feature (folded into full/node/cluster/pi), the persist backend defaults to redb in authoritative mode whenever a persist dir is configured — the engine is a durable SOURCE OF TRUTH out of the box, with a one-time .mp/.wal → redb migration on first authoritative boot.
CONCEPT:EG-KG.query.describe pg-wire extended/prepared protocol The Describe step of the Postgres extended-query (prepared-statement) protocol over the pg-wire shim, where the shim can't statically know a result shape ahead of execution.
CONCEPT:EG-KG.query.follow-up pg-wire SQL DML completeness Completes INSERT/UPDATE/DELETE DML coverage over the pg-wire shim's SQL surface.
CONCEPT:EG-KG.storage.authoritative-flip Quiet common-restart recovery The common small restart path recovers without noisy warnings — only genuine anomalies log loudly.
CONCEPT:EG-KG.query.concept-13 pg-wire identity bridge Bridges pg-wire (SCRAM) authentication to the engine's AgentIdentity, so the post-login pg user drives IsolationLayer ACL checks.
CONCEPT:EG-KG.storage.one-fsync-covers-raft Durable redb Raft log The Raft log, vote, and applied state live in the SAME graph.redb Database as M2 graph data, keyed by (group_id, index)/(group_id, key); a log append and its graph mutation coalesce into ONE WriteTransaction/one fsync, and a restarted node recovers its log tail locally from redb. The separate raft.redb sidecar is gone.
CONCEPT:EG-KG.sharding.raft-resharding Multi-Raft scaffold A MultiRaft manager holds N openraft groups keyed by GroupId, sharing ONE TCP listener per node (frames tagged/demuxed by group id) and ONE shared graph.redb; a GroupRouter maps graph_name → GroupId. Group = transaction boundary (no cross-group txns yet). Runs one DEFAULT_GROUP while exercising routing/lifecycle/isolation in tests.
CONCEPT:EG-KG.storage.blob-namespace Content-addressed BLOB substrate A streamed, content-addressed BLOB store (blob feature) for large binary payloads, with begin/chunk/commit/fetch/ref/unref/gc methods served over the protocol.
CONCEPT:EG-KG.sharding.semantic-embedding-store-backed Native eg-ann vector index Pure-Rust CPU IVF-PQ + OPQ + SQ8-refine vector index (ann feature, folded into pi/node/cluster/full) as the SemanticStore backend, replacing rebuild-on-load HNSW; a persisted index reopens WITHOUT rebuilding from raw vectors (ann-redb also stores the codes in the redb durable tier).
CONCEPT:AU-KG.compute.vector Cross-modal query plan ONE cross-modal plan composing a DataFusion filter with vector/graph stages into a single execution pipeline.
CONCEPT:EG-KG.query.concept-14 Cost-based plan reorder The same cross-modal plan is reordered by cost (a selective vs. a broad predicate) so the most selective stage runs first.
CONCEPT:AU-KG.retrieval.god-nodes-communities Native time-series store A native time-series store (tsdb feature) with store + query primitives, present when the feature is compiled.
CONCEPT:EG-KG.compute.handled-outside-single-anchor Unified Ebbinghaus decay curve The ONE temporal-decay curve (eg_core::decay): the same Ebbinghaus function powers semantic-memory confidence decay and the tsdb time-series decay queries, so decay is defined once and shared.
CONCEPT:AU-KG.ontology.emits-database-ontology-entities Correctness + load harness The standing proof-engine that gates every distributed/durability claim (multi-Raft, M2 durability, distributed txns, replication). TEST/DEV-ONLY harness feature (pulls raft): a deterministic nemesis fault-injector (partition/kill/saturate) + load-gen + linearizability checker + a bounded gauntlet (cargo test --features harness raft) and an unbounded seeded nemesis soak bin. NEVER folded into any deployment tier — a production/pi/full build links NOTHING from it, and the raft-network partition hook is #[cfg(any(test, feature = "harness"))] so a non-harness build has the byte-for-byte production network path. See docs/architecture/correctness_harness.md.
CONCEPT:AU-KG.query.top-nodes-by-degree UQL text query front-end A pure text front-end (crates/eg-plan/src/uql/*: lexer + recursive-descent parser, NO DataFusion) that parses a piped query language into the SAME wire::Plan the UnifiedQuery method carries, then runs the IDENTICAL run_unified executor — Method::UnifiedQueryText is a front-end over the cross-modal planner, not a new execution path. A parse error renders a clear caret-annotated error Response. Ships in default/Pi alongside the algebra/cost/IR (only Plan EXECUTION is query-gated).
CONCEPT:EG-KG.backend.engine-modes In-process embedded library API A SQLite/DuckDB-style in-process handle (src/embedded.rs + src/embedded/*) that drives the SAME GraphCore + redb-authoritative durable rows the socket dispatch does (via wal::apply + the extracted server-INDEPENDENT src/redb_store.rs), with NO Tokio server/socket/HMAC — ONE core, two transports. Gated on embedded (→ redb, deliberately NOT server), so --features "embedded redb" builds with no Tokio runtime (the Pi/edge "a local engine each" story). Delete durably purges via the shared purge_graph_rows so a recreate of the same tenant name starts clean (EG-KG.backend.tenant-delete-recreate-same parity).
CONCEPT:EG-KG.backend.tenant-delete-recreate-same Tenant-delete durable purge DeleteGraph now durably PURGES the graph's redb rows (nodes/edges/ledger/semantic + the graph_meta identity), awaited commit-before-ack under authoritative mode, so a recreate of the SAME tenant name starts from a clean durable slate. Without it a deleted incarnation's rows survived (same sanitized key) and leaked into the recreated tenant via the read-through-on-RAM-miss path and via load_all — silently dropping/corrupting the new tenant's writes (tenant churn / resharding / hibernation rehydration). Default no-op for non-authoritative / non-redb backends.
CONCEPT:AU-KG.query.text-spatial-time Embedded BM25 full-text search + lexical Rank/RRF-hybrid Op A crates/eg-text Tantivy-backed (the Rust Lucene) inverted index over (node_id, text): BM25 top-k → Vec<TextHit>, incremental upsert/delete, persistent mmap segments that reopen WITHOUT rebuilding postings. Adds two ops to the eg-plan RowSet algebra: Op::RankText{query} (BM25 re-rank of the candidate set — a lexical sibling of the vector Rank, same (id, score) currency) and Op::FuseRrf{left, right, k} (reciprocal-rank fusion of two ranking sub-plans into one RowSet — the modern hybrid lexical+vector retrieval pattern; a fused query beats either modality alone). HARD feature-gated behind text (folded into node/cluster/full, NEVER pi/default) because Tantivy is heavy (≈140 crates incl. zstd-sys's C build) — a default/Pi build links NO Tantivy (the Pi contract, asserted by cargo tree). Dep-free TextHit + rrf_fuse ship in every build.
CONCEPT:EG-KG.ontology.kg-native-rdf-sparql Native RDF surface (Lane W) A crates/eg-rdf mapping that projects an RDF dataset onto the SAME property-graph the rest of the engine uses and serializes it back (Turtle/N-Triples via oxrdf/oxttl). LoadRdf ingests triples through the BatchUpdate durable path; GetRdf serializes a graph OUT; an opt-in lossless redb quads table (rdf-redb) holds multi-valued-literal predicates the property blob can't express, and GetRdf unions them back. PURE-RUST — links NO C/native dep (asserted by cargo tree) — feature-gated behind rdf (folded into node/cluster/full, kept OUT of pi/default).
CONCEPT:EG-KG.ontology.concept-11 Native SPARQL 1.1 surface (Lane W) A SPARQL SELECT engine (crates/eg-rdf/src/sparql.rs) that compiles spargebra's parsed algebra (BGP/FILTER/OPTIONAL) to GraphView scans over the mapped property-graph, returning a SparqlResult { vars, rows } over the Method::Sparql round-trip. IMPLIES rdf; pure-Rust (spargebra), no native dep. Read-only; gated behind sparql (folded into node/cluster/full, OUT of pi/default).
CONCEPT:EG-KG.ontology.incremental-materialization Native OWL 2 reasoner (Lane W) A pure-Rust OWL 2 EL⁺ completion reasoner (Baader/Brandt/Lutz CR1–CR4 + role chains — the ELK/CEL core) UNIONED with the OWL 2 RL property rules, in crates/eg-rdf/src/owl.rs. Parses OWL axioms from the SAME oxttl-parsed RDF as rdf (NO horned-owl/whelk-rs — links ZERO native dep), and provides classification (the full subclass hierarchy S(A) derived through existential restrictions ∃r.C ⊑ D, equivalentClass, conjunction, role chains, ⊤/⊥), consistency checking (a ⊥-derivation via owl:disjointWith ⇒ inconsistent + unsatisfiable-class list), incremental/differential materialization (monotone fixpoint resumes from the prior closure on an axiom delta — only new subsumers derived), and justifications (each inferred subsumption cites its rule + axioms + premises). Reaches entailments the RL eg-compute::reasoning cannot (e.g. HumanHeart ⊑ HumanComponent through ∃partOf.Body on the LHS). Surfaced as Method::OwlReason (returns OwlReasonResult { subclasses, instances, consistent, unsatisfiable }) + a Python RdfClient.owl_reason. Read-only; gated owl (implies sparqlrdf; PURE-RUST + Pi-safe — folded into pi/node/full).
CONCEPT:EG-KG.ontology.concept-12 Reason + SparqlBgp ops in the unified planner (Lane W) Two SOURCE ops added to the eg-plan RowSet algebra: Op::Reason { target_class, ontology } (seed the RowSet with the OWL-reasoner-inferred members of a class — incl. ids the property-graph stored NO explicit type edge for) and Op::SparqlBgp { query, var } (seed from a SPARQL SELECT's node bindings). Both produce a RowSet, so ONE Method::UnifiedQuery plan can do OWL/SPARQL candidate-set → graph Traverse / vector Rank / SQL Filter / Limit. Proven by a compose oracle (crates/eg-plan/src/owl_tests.rs): a fused Reason→Rank/SparqlBgp→Rank plan returns the BYTE-IDENTICAL ids as running the reasoner/evaluator then an independent kNN separately. The ops live inside the query-gated Op enum (the unified-plan executor needs DataFusion), so they are gated owl-plan (= owl + query) — folded into node/full, OUT of pi.
CONCEPT:EG-KG.storage.lane-n-increment Cross-shard 2PC distributed txn (Lane N) A CrossShardCoordinator (src/raft/cross_shard_txn.rs) that runs two-phase commit across multiple Raft groups when a txn spans shards (GroupRouter::is_cross_shard): each participant durably PREPAREs its staged slice (xshard_prepare table), the coordinator writes the atomic DECISION row (xshard_decision1=commit/0=abort, presumed-abort), then participants apply. In-doubt txns survive a coordinator/participant crash and are resolved deterministically from the durable prepare/decision rows on boot (recover_in_doubt, run unconditionally before serving). The pure xshard_* table machinery lives in the shared server-INDEPENDENT src/redb_store.rs (alongside NODES/EDGES/purge_graph_rows); the writer-thread Cmd arms in redb_backend call into it. Proven by the cross-shard nemesis gauntlet (src/raft/xshard_harness.rs, harness feature): atomicity-on-all-participants, killed-participant-no-partial-commit, recovery-commit/abort, single-group fast-path.
CONCEPT:EG-KG.backend.tiny-shared Reference-counted graceful shutdown + tier bundles The accept loop now select!s the next connection against a shared ShutdownCoordinator (an AtomicUsize active-connection refcount + a Notify signal), so it BREAKS and main() falls through to the persistence flush + final checkpoint (previously dead code). A SIGTERM/SIGINT handler fires the signal (a supervisor stop is a clean checkpointed shutdown). The optional --idle-shutdown-secs N (env EPISTEMIC_GRAPH_IDLE_SHUTDOWN_SECS) arms an idle watcher that triggers shutdown once the refcount has been 0 for N seconds — the auto-bundled tiny daemon self-terminates after its last client disconnects; absent/0 ⇒ long-living/persistent. Commit-before-ack is intact (shutdown checkpoints, never drops acked writes). The default maturin wheel builds the node tier bundle (max-functionality single binary); a pi-tier lean wheel is documented for the Raspberry-Pi-3.
CONCEPT:EG-KG.storage.100m-tenant Online resharding + cold-tenant hibernation src/raft/reshard.rs TenantManager: reshard_graph(A→B) = quiesce (per-tenant migration lock) → durability barrier (checkpoint the graph) → re-point the router (GroupRouter::assign) → resume — zero-downtime because one shared registry + one shared graph.redb keyed by graph name means a "move" is re-pointing ownership of future writes, not copying rows. Cold-tenant hibernation: GraphCore::hibernate() drops the in-RAM topology/props/vectors (the durable redb rows are retained, the read-through seam intact) and rehydrate_graph() rebuilds the core from a per-graph durable dump (redb_store::read_graph_dump). MultiRaft gains tenant_lock + ensure_group, now held in ServerState (no longer leaked). Proven by src/raft/reshard_harness.rs (harness feature): reshard-keeps-data-and-serves-after, reshard-data-durable-across-restart, hibernate-then-rehydrate-intact.
CONCEPT:EG-KG.txn.reader-never-sees-node Cross-modal ACID commit A single durable WriteTransaction lands a graph mutation + a vector upsert (TxnAddEmbedding) + a blob-reference (TxnBlobRef — a (node_id, digest) pair) ATOMICALLY across the modalities: either every modality commits in the one redb transaction or, on any failure, the WriteTransaction is dropped and NONE land (true rollback — no torn cross-modal write). Adds the TxnAddEmbedding/TxnBlobRef protocol ops + client methods. Proven by crossmodal-commits-all-modalities-atomically (+ durable reload) and crossmodal-rolls-back-all-modalities-on-failure.
CONCEPT:EG-KG.txn.routes-cross-shard-txn Lane-N multi-graph BeginTxn wire (user-facing 2PC) The user-facing Txn* ops gain an optional per-op graph; GraphTxnState accumulates a multi-graph write-set (extra_writes). Commit detects span via GroupRouter::is_cross_shard: a single-group txn stays the byte-for-byte fast path, while a ≥2-group span routes the staged write-set through CrossShardCoordinator::commit_cross_shard (the existing, fault-proven 2PC of CONCEPT:EG-KG.storage.lane-n-increment). No-cluster / single-group-collapse applies the slices locally. Proven by xshard_harness: user-multigraph-txn-commits-atomically-across-groups, user-multigraph-txn-atomic-under-participant-kill (no partial commit).
CONCEPT:EG-KG.query.streaming-cdc-subscriptions CDC feed + continuous queries (Lane L) A durable, ordered per-graph change stream layered over the engine's existing change record (the ledger): the dispatch write-side-effect block — the ONE centralized point that records every durable mutation — also emits an ordered CdcEvent (node/edge add/remove/update with before/after property blobs, a per-graph monotonic seq) into a CdcHub (src/server/cdc.rs, feature streaming). Method::CdcRead { graph, from_seq, limit } tails it from a cursor (poll/stream); re-reading from a later seq skips seen events; a cursor behind the bounded retained ring errors so a consumer re-seeds. On the SAME feed, Method::RegisterContinuousQuery/ReadContinuousQuery maintain a named aggregate (count / numeric-sum, label-filtered) INCREMENTALLY on each change (delta, not a re-run) — seeded from the graph's current state at registration so it equals a full re-run (proven by test_continuous_query_incremental_equals_full_rerun). Captures before/after by reading the GraphCore directly, so it is correct for BOTH the inline and the write-coalescer apply paths. PURE-Rust (a bounded VecDeque ring + tokio::Notify) — no heavy dep — so streaming folds into pi/node/cluster/full (the Pi contract holds: a --features pi build links no DataFusion/openraft).
CONCEPT:EG-KG.storage.feature Distributed graph compute (Pregel/GAS) + materialized views A cross-shard vertex-centric superstep engine (src/raft/pregel.rs, feature compute-dist which implies raft): runs PageRank / connected-components / BFS over a graph whose vertices span MULTIPLE Raft groups by exchanging per-superstep boundary messages, so a distributed run equals the single-graph result (proven by cross_shard_pagerank_matches_single_graph / cross_shard_connected_components_matches_single_graph). An incremental variant updates from a prior result instead of recomputing from scratch (incremental_cc_equals_from_scratch). Named results are persisted as MatViewStore materialized views in a redb MATVIEWS table (src/redb_store.rs, alongside NODES/EDGES) via matview_put/matview_scan on the redb backend, reloaded on boot by reload_matviews (src/main.rs) so GetMatView serves them immediately (matview_persists_and_reloads_from_redb). The DistributedCompute + CreateMatView/GetMatView/RefreshMatView Methods self-route in handlers::dist_compute. Cluster-tier only (needs the multi-Raft groups) — kept OUT of pi/default/node/full, so the Pi contract holds (no openraft).
CONCEPT:EG-KG.query.rowset-execution WASM-sandboxed UDF runtime A wasmtime sandbox (crates/eg-wasm, feature wasm-udf) that runs agent-supplied compute as a WASM module with NO host capabilities (no fs/net imports — a module importing one is rejected), bounded by a fuel budget (an infinite-loop UDF is fuel-killed, never a hang) and a memory cap (a memory-hog UDF is capped) — proven by the eg-wasm unit tests (identity round-trip, fuel-kill, memory-cap, host-import-rejected). Method::RegisterUdf { id, wasm } compiles+caches a UdfModule in a process-global UdfRegistry on ServerState; Method::RunUdf { id, input } runs it sandboxed off-reactor (round-trip + fuel-kill proven through the full server dispatch by run_udf_through_dispatch_runs_sandboxed_and_fuel_kills_infinite_loop). A wasm-udf-gated Op::Udf { id } plan op (crates/eg-plan) runs a registered UDF as a RowSet transform inside a unified query. Folded into node/cluster/full (the WASM runtime is a serving-tier capability) — kept OUT of pi/default, so a Pi/default build links neither wasmtime nor cranelift.
CONCEPT:EG-KG.query.wire-codec Subscriptions / watch + triggers (Lane L) A LISTEN/NOTIFY-style reactive surface over the CDC feed (CONCEPT:EG-KG.query.streaming-cdc-subscriptions), transport-compatible with the one-Response-per-Request engine: Method::Watch { graph, from_seq, label, timeout_ms } is a long-poll — it returns the matching changes since the cursor, else awaits the per-graph Notify up to timeout_ms for the first one, then returns a WatchBatch { events, next_seq } the client resumes from (one Request → one Response, NO side-channel socket; proven to wake on an in-window write by test_watch_long_poll_wakes_on_write). Plus simple triggers/reactions: Method::RegisterTrigger { name, graph, label, op, action } records a firing (an opaque action payload) whenever a change matches its label + op (add/remove/update/any); FiredTriggers polls the per-graph fired log (cursor-driven), ListTriggers/DropTrigger manage them. The reactive READ + REGISTER surface self-routes in handlers::streaming like tsdb/blob; the WRITE side (emitting + firing) lives in the dispatch shell. Same streaming feature/Pi-contract as KG-2.229.
CONCEPT:EG-KG.compute.lane-v Per-tenant memory budget + autoscale signals (Lane V — cost/efficiency) src/cost.rs (feature cost, folded into pi/node/cluster/full — PURE-RUST, Pi-safe). Tracks an approximate resident-RAM estimate per graph (GraphCore::memory_estimate — node/edge property-blob bytes + per-node/edge overhead + embedding-vector bytes via the new SemanticStore::embedding_bytes) and rolls it up per TENANT (tenant_of = the graph-name prefix before the first :). enforce_memory_budgets runs a periodic sweep: a tenant over its byte budget has its COLDEST graphs reclaimed — first durability-gated LRU eviction (reusing the EG-KG.storage.read-through-seam-exercised no-loss gate per graph), then GraphCore::hibernate (EG-KG.storage.100m-tenant) — until back under cap, with a global ceiling + fair per-tenant caps (ceiling / active_tenants) so one hot tenant can't starve others. ONE knob EPISTEMIC_GRAPH_MEMORY_BUDGET (k/m/g suffixes) turns it on; the default auto-sizes to 70% of system RAM (/proc/meminfo). Autoscale signals: Method::ResourceStats returns a structured ResourceSnapshot (per-graph + per-tenant + process aggregate: resident memory, node/edge counts, in-flight admission depth, hibernated-vs-resident counts, eviction/hibernation totals) for the agent-utilities autoscaler (AU-OS.config.health-gated-deploy-rollback), surfaced ALSO via the Prometheus /metrics endpoint (graph_memory_bytes, graph_hibernated, budget_evictions_total, budget_hibernations_total). capacity_estimate + docs/cost_model.md map the Pi→cluster tiers to footprints for capacity planning. Proven by src/cost.rs tests: over-budget-evicts-and-rehydrates (stays under cap + evicted data reads back), fair-cap-protects-small-tenant, resource-stats-counts-are-accurate.
CONCEPT:EG-KG.query.sparql-completeness Query-language polish: native GraphQL + SPARQL completeness + UQL surface clauses (Lane M) Three read-surface completions. (1) Native GraphQL (crates/eg-graphql, feature graphql, node/cluster/full — OUT of pi): a GraphQL query compiled to scans + BFS over the SAME off-lock GraphView the Cypher path uses, via a PURE-RUST hand-written parser + resolver + schema-from-graph (NO async-graphql — evaluated as ~80+ transitive crates + a proc-macro derive, too heavy for the Pi contract). Method::GraphQl { query } routes through handlers::query::try_handle, so GraphQL runs UNDER the SAME version-keyed, RLS-aware result-cache compose as SQL/Cypher/SPARQL (rls_cache_hash("graphql", …, caller, rls) + analysis_snapshot_versioned() + rls.filter_view(caller, …)): a GraphQL read can never leak rows across agents, and GraphQl == CypherQuery for the equivalent query. (2) SPARQL completeness (crates/eg-rdf/src/sparql.rs, feature sparql): aggregates (COUNT/SUM/…/GROUP BY), property paths, and GRAPH patterns over the GraphView scans. (3) UQL surface clauses (crates/eg-plan/src/uql/parser.rs + uql/mod.rs): AS OF @<ts>Op::AsOf, WINDOW <dur>Op::Window, FOREIGN "<name>"Op::Foreign (a name MARKER — the resolved federation executor is the federation-gated Op::ForeignScan of EG-KG.query.query-federation, which carries a full ForeignSourceSpec eg-plan cannot resolve from a bare name), REASON <Class>Op::Reason, TEXT "<q>"Op::RankText. The context ops are RowSet-preserving today (the temporal/windowed/federation pulls are downstream seams).
CONCEPT:EG-KG.ontology.concept-13 Probabilistic / confidence-weighted OWL reasoning (Lane I) Confidence-weighted, time-decayed OWL inference (crates/eg-rdf/src/owl.rs, feature owl). The EL⁺/RL closure propagates a per-entailment confidence in [0,1]: eg:confidence axiom annotations × per-node confidence × Ebbinghaus time-decay, taking the max over alternative derivations. Method::OwlReason gains min_confidence (τ threshold — only entailments with confidence ≥ τ are returned; 0.0 keeps everything, and a fully-HARD ontology yields all 1.0 so the surface is backwards-identical when nothing is uncertain); OwlReasonResult gains index-aligned subclass_conf/instance_conf. New Method::OwlReasonDistributed { graphs, ontology, target_class, min_confidence } runs ONE weighted closure over the UNION of the named graphs/shards' axioms + decayed-confidence facts (the cross-shard union-read seam, EG-KG.query.cross-graph-union) and returns the SAME OwlReasonResult a single-graph reason would over the same axioms. The unified-plan Op::Reason now scores each inferred member by its membership confidence as the RowSet score, so a bare REASON plan is already ranked by confidence and composes with a downstream vector Rank/Limit. Proven by eg-rdf owl tests + a distributed 2-graph dispatch proof.
CONCEPT:EG-KG.backend.many-repeated-create-delete Tenant-churn-safe in-memory teardown on DeleteGraph DeleteGraph now tears down the per-graph in-memory state keyed by NAME elsewhere in ServerState — not just the registry's live GraphCore — so a same-name recreate starts truly clean every cycle and never binds the deleted incarnation's orphaned state. The write-coalescer drops its cached per-graph writer (WriteCoalescer::drop_writer, src/write_coalescer.rs) so a recreate routes to the NEW GraphCore instead of the deleted one's orphaned worker; the dispatch DeleteGraph arm (src/server/dispatch.rs) sequences this in-memory teardown alongside the durable purge (src/server/persistence/redb_backend.rs). Closes a tenant-churn data-loss window where many repeated create→delete→recreate cycles on the SAME graph name could land writes on a stale coalescer.
CONCEPT:EG-KG.ontology.lpg-rdf-projection-vocabulary Configurable LPG→RDF projection vocabulary for native SPARQL Method::Sparql gains base_iri + type_convention that select how the live property graph is projected into RDF before SPARQL evaluation (eg_rdf::sparql::Projection::from_wire, src/server/handlers/rdf.rs, feature sparql). An empty base_iri ⇒ the identity projection (verbatim keys), so existing callers are byte-for-byte unchanged; a caller-supplied namespace + CamelCase convention projects keys/labels into that vocabulary. Runs under the SAME off-lock snapshot + version-keyed, RLS-aware result cache as SQL/Cypher (CONCEPT:EG-KG.coordination.distributed-cache-coherence × EG-KG.sharding.row-level-security); the cache key also folds in the projection vocabulary so a raw and a namespaced query of the same text never alias.
CONCEPT:EG-KG.ingest.resets-socket-so-assimilation Response-size overload backstop for the unbounded GetNodes full-graph dump The GetNodes handler (src/server/handlers/graph_ops.rs) checks the cheap GraphCore::node_count() BEFORE materializing the response: when the graph exceeds the cap (EPISTEMIC_GRAPH_MAX_RESPONSE_NODES, default 50_000, read once at startup via state::max_response_nodes()), it returns a typed RESULT_TOO_LARGE error Response INSTEAD of serializing the whole graph (e.g. __commons__ with 166K+ nodes × 1024-dim embeddings) into one gigabyte-scale frame that overruns/resets the client connection. The decision is a pure oversize_dump_error(count, cap) helper (unit-tested). The Python client surfaces it as a catchable ResultTooLargeError(RuntimeError). Cap 0 disables the guard; the bounded reads (GetNodesByLabel, per-id) are unaffected.
CONCEPT:AU-KG.ontology.manage-arbitrary Multi-Raft M2 hardening: pooled per-peer Raft connections A shared PeerPool (src/raft/network.rs) keeps a small bounded set of WARM TcpStreams per peer host:port, reused across append/vote/snapshot RPCs and across ALL groups on the node (one pool per MultiRaft, mirroring the already-shared inbound listener). Replaces the scaffold's connect-per-RPC churn. Strict request→response on one stream with no correlation id, so a pooled connection is handed out exclusively for one round-trip and only returned to the idle set if it SUCCEEDED; a stale idle entry surfaces as an IO error and the call transparently retries ONCE on a fresh connection, so openraft never observes a spurious failure. opens/reuses counters give metrics/test visibility.
CONCEPT:AU-KG.ingest.mirror-inbound Multi-Raft M2 hardening: group-per-tenant-range routing ring GroupRouter (src/raft/multi.rs) resolves graph_name → GroupId as override → tenant-range ring → DEFAULT_GROUP. The ring (set_group_ring) is a sorted, de-duplicated set of group ids that un-pinned graphs hash-distribute across via a stable FNV-1a hash (identical on every node, never persisted). MultiRaft::configure_group_ring(n) brings up groups 0..n on the node and installs the ring. An EMPTY ring (the default) collapses to single-group scaffold behavior byte-for-byte; explicit assign pins (reshard/tests) always win.
CONCEPT:AU-KG.ingest.staged Multi-Raft M2 hardening: per-group snapshot scoping A group's snapshot dump (EgStore::dump_graphs, src/raft/store.rs) is SCOPED to the graphs whose tenant range resolves to THIS group via the router carried on AppCtx.router, so a large tenant in one group never bloats another group's snapshot. A store opened WITHOUT a router (router: None — the direct/single-store path) dumps the whole registry, preserving the unscoped scaffold behavior.

Cross-Project References (from agent-utilities)

Concept ID Name Origin
CONCEPT:AU-ECO.messaging.native-backend-abstraction Unified Toolkit Ingestion agent-utilities
CONCEPT:AU-KG.query.object-graph-mapper Knowledge Graph Core Core Architecture agent-utilities
CONCEPT:AU-ORCH.execution.inject-signal-board-observations Multi-Agent Orchestration Abstraction agent-utilities
CONCEPT:AU-KG.query.vendor-agnostic-traversal Batch Materialization / Local SPARQL Fast Path agent-utilities
CONCEPT:EG-KG.storage.nonblocking-checkpoint Code/Test Enrichment & Interlinking (incl. 2.8r cross-file call/import resolution) agent-utilities
CONCEPT:EG-KG.query.cross-graph-union Cross-graph union reads (point/label/neighbor reads unioned across a graph set, deduped by id) epistemic-graph
CONCEPT:EG-KG.query.read-only-sql-query Internal SQL query surface (read-only SELECT … FROM nodes over one graph via DataFusion, behind the query feature) epistemic-graph
CONCEPT:EG-KG.compute.consult-lazy Lazy secondary label index (label → node ids) for O(1) label lookup, invalidated on write epistemic-graph
CONCEPT:EG-KG.storage.kg-kg Pluggable PersistenceBackend + durable redb write-through tier (behind the redb feature) epistemic-graph
CONCEPT:EG-KG.query.concept-12 Bounded, demand-driven secondary PROPERTY equality index (key → value → node ids) + DataFusion predicate pushdown epistemic-graph
CONCEPT:AU-KG.retrieval.architecture-report Unified IndexManager seam: ONE registry over the label/property/vector/ontology indexes (SecondaryIndex trait + index_for/descriptors_for_column); eg-query pushdown consults ONE PushdownRegistry. See docs/architecture/index_manager.md epistemic-graph
CONCEPT:EG-KG.retrieval.one-round-trip-discovery One-round-trip hybrid discovery (Method::Discover { keywords, query_embedding, k }): dense-retrieve candidates via the HNSW batch primitive, re-rank each by BOTH semantic similarity AND lexical keyword overlap over name/description/type, and return the top-k hydrated as [{id,name,description,type,score}]. Complements SemanticSearch (bare (id,score)) by folding in the keyword signal + hydrating result text in a single call; an empty embedding degrades to a bounded keyword-only scan epistemic-graph
CONCEPT:EG-KG.query.eg-validate-procedural-body PL/pgSQL procedural interpreter: CREATE FUNCTION … LANGUAGE plpgsql stored in the durable function catalog and executed on a bare SELECT fn(args) / CALL proc(args), with embedded SQL run back through the existing read path (crates/eg-query/src/sql/plpgsql.rs, folded into the sql feature, no new deps) epistemic-graph
CONCEPT:EG-KG.query.concept-7 PL/pgSQL control-flow: DECLARE vars, BEGIN..END, := assignment, IF/ELSIF/ELSE, LOOP/WHILE/FOR … LOOP (integer range), EXIT/CONTINUE, RETURN, RAISE, and SELECT … INTO var over a variable environment — a hand-written statement interpreter over the tokenized body epistemic-graph
\n## CONCEPT:AU-KG.memory.mementified-context\nRust-Native Finance Compute Suite
\n## CONCEPT:EG-KG.compute.rust-native-training-loss\nGraph Network Protocols
\n## CONCEPT:EG-KG.compute.rust-native-training-loss\nData Science Primitives
\n## CONCEPT:AU-KG.memory.working-set-eviction\nAST Ingestion Pipeline
CONCEPT:EG-KG.ingest.timeseries-source-mid-pipeline Cross-modal SEAM regression: planner mid-pipeline composition proofs (Reason→Traverse→Rank == reasoner∘BFS∘kNN oracle; SensorFuse→Rank / TensorScan→Rank downstream composition; reason-seeded cost reorder plan-shape) epistemic-graph
CONCEPT:EG-KG.query.query-plan-node Bitemporal SEAM: decay-sweep (Ebbinghaus) re-weights confidence while AsOf{Valid} re-selects liveness and an in-between UPDATE flips a validity window — asserts confidence values + liveness at two instants epistemic-graph
CONCEPT:EG-KG.compute.concept-6 Vector⇄reasoning cross-txn write→read consistency: a freshly written node (embedding + reasoner-subsumed type) is BOTH ANN-ranked AND reasoner-included by the next [Reason|>Rank] plan; an embedding UPDATE is reflected by the next ANN pass epistemic-graph
CONCEPT:EG-KG.compute.concept-7 SPARQL-UPDATE→reasoning cross-txn visibility: a DELETE/INSERT WHERE re-parenting a TBox axiom is visible to the very next OWL EL⁺ classification epistemic-graph
CONCEPT:EG-KG.query.concept-9 Cache-coherence SEAM: the version-keyed SQL result cache HITS at the old version and MISSES + returns fresh rows after a version-bumping write; cache ON == cache OFF byte-identical epistemic-graph
CONCEPT:EG-KG.compute.cross-modal-seam Served E2E cross-modal SEAM: add(embedding+type)→reason→discover returns the fresh node; a SPARQL-style axiom UPDATE is visible to BOTH owl_reason AND hybrid discover epistemic-graph
CONCEPT:EG-KG.query.txn-cross-modal-ryow In-txn cross-modal RYOW unified query: Method::TxnUnifiedQuery{txn_id,plan,…} + TxnUnifiedQueryText{txn_id,text,…} run the SAME wire::Plan/UQL over a committed-store snapshot OVERLAID with the txn's staged write-set, so staged nodes/edges/embeddings are visible to THIS txn pre-commit and invisible off-txn until commit (Gap 1, Lanes 0/A) epistemic-graph
CONCEPT:EG-KG.backend.cross-modal-atomic-commit In-txn time-series measurement staging: Method::TxnAddMeasurement{txn_id,series,points,graph} stages a tsdb batch that lands in the SAME redb WriteTransaction as the txn's graph/vector/blob writes at commit (Gap 2, Lanes 0/B) epistemic-graph
CONCEPT:EG-KG.txn.extended-cross-modal In-txn OWL axiom staging: Method::TxnAxiom{txn_id,turtle,graph} lowers staged Turtle axioms to graph node/edge writes in the SAME atomic commit WriteTransaction so the OWL reasoner sees them consistently (Gap 2, Lanes 0/B) epistemic-graph
CONCEPT:EG-KG.query.extended-cross-modal In-txn SPARQL CONSTRUCT staging: Method::TxnConstruct{txn_id,sparql,graph} lowers the CONSTRUCT's produced triples to graph node/edge writes in the SAME atomic commit WriteTransaction (Gap 2, Lanes 0/B) epistemic-graph
CONCEPT:EG-KG.query.native-time-series TSDB-in-plan source op: wire::Op::TsScan{series,from,to} seeds the cross-modal RowSet from the native eg-tsdb SeriesStore over [from,to) (via PlanCtx.tsdb/with_tsdb), so a downstream Rank/Limit/Filter fuses the tsdb leg with graph/vector/relational legs in ONE plan (Gap 3, Lanes 0/C) epistemic-graph
CONCEPT:EG-KG.sharding.deployment-tiers One full-featured build (tier collapse): default == fullcargo build links every MAIN feature that compiles without an external GPU/robotics toolchain; the former pi/pi-max/node deployment tiers (and the shared-wheel-filename collision) are removed. cluster (openraft HA) and full-extras (GPU/ROS2) remain opt-in build layers, NOT published wheels. Preserved invariants: no pyo3 (bindings=bin), no openraft in default, no cudarc/rustdds in default epistemic-graph
CONCEPT:EG-KG.txn.isolation-ryow-begin-set pgwire cross-modal txn seam: the PGWIRE TEXT entrypoint for the EG-359..363 in-txn cross-modal seam. The shared WireSession recognizes UQL …, SET EMBEDDING FOR <id> = <vec>, INSERT INTO series …, SPARQL UPDATE …, SPARQL CONSTRUCT … inside a BEGIN…COMMIT and routes them onto the committed RPC seam — staging via txn.rs GraphTxnState (vector/measurement + lowered axiom/CONSTRUCT methods), reading in-txn via the extracted run_unified overlay (read-your-own-writes across graph+vector+OWL), and committing every modality atomically via the extracted commit_cross_modal_txn (now in-memory-tolerant on a no-persistence engine). Off-txn cross-modal writes auto-commit as one atomic statement. No plan/txn logic duplicated; lives on the EG-KG.compute.subsystems-reference shared core so the multi-wire family inherits it epistemic-graph
CONCEPT:EG-KG.compute.eg North Star: Seamless — every discovered cross-modal seam is fully implemented at EVERY surface (RPC/native, all SQL wires, SPARQL, GraphQL), never merely flagged. docs/north_star.md tracks the seam backlog: RPC cross-modal txn (done, EG-359..363), pgwire cross-modal txn (done, EG-KG.txn.isolation-ryow-begin-set), GraphQL cross-modal txn (done in-memory tier, EG-379..383; durable-commit + tsdb tier = facade reconcile), mysql/mssql-wire wire-test pending, advanced cross-modal tests open epistemic-graph
CONCEPT:EG-KG.query.txn-tsdb-read-your In-txn tsdb read-your-own-writes staged-series overlay: an in-memory eg_plan::StagedSeries (series → staged (ts_ns, values) points) attached to PlanCtx via with_staged_series; Op::TsScan (tsdb_scan_op) MERGES the staged overlay (RYOW precedence on a ts collision) with the committed eg-tsdb SeriesStore, so an in-txn UQL TsScan reads the txn's own uncommitted GraphTxnState.measurements while an off-txn read sees committed only. run_unified_overlaid builds the overlay from the resolved txn's staged measurements. Closes the north_star in-txn tsdb-RYOW open row epistemic-graph
CONCEPT:EG-KG.query.reason-iri-parses-angle REASON <iri> mid-plan: the UQL lexer tokenizes an angle-bracketed IRI (<scheme:…>) as Tok::Iri and parse_reason accepts it, so REASON <http://…/Class> is expressible at the UQL/pgwire surface (previously only a bare label). Composes mid-pipeline via the existing reason_op FILTER branch — a MATCH … |> RANK … |> REASON <http://…/Class> |> TRAVERSE intersects the ranked candidate set with the reasoned members of that explicit IRI class epistemic-graph
CONCEPT:EG-KG.compute.no-embedder-bound-op Server-side embedder seam for UQL RANK BY ~ "text": a quoted rank ref lowers to wire::Op::RankEmbed { text }, resolved at exec time by a TextEmbedder (embed(text) -> Vec<f32>) bound on the PlanCtx via with_embedder; the executor turns the text into a query vector kNN-ranked over the SemanticStore exactly like a literal-vector Op::Rank. With NO embedder bound Op::RankEmbed is a clean typed error (never a panic). Dep-free trait; a deterministic HashEmbedder fallback exists for tests/offline. A bare-ident handle (~handle) stays a reserved forward seam epistemic-graph
CONCEPT:EG-KG.query.bind-server-side-text Facade injection point for the UQL embedder seam: run_unified (src/server/handlers/query.rs) binds the process-wide server-side TextEmbedder onto the PlanCtx (with_embedder) so a served RANK BY ~ "text" (Op::RankEmbed) resolves its vector. The engine stores embeddings but produces them client-side today, so no in-process model ships by default (unbound ⇒ clean typed error); a real embedding model is wired HERE, and the deterministic HashEmbedder fallback is opt-in via EG_UQL_TEXT_EMBEDDER=hash epistemic-graph
CONCEPT:EG-KG.compute.tsscan-series-window-60s Real WINDOW tumbling time-series aggregate: Op::Window { secs } no longer passes rows through — it CONSUMES a RowSet of (ts,value) rows (a graph-node row's valid_from+score/value, OR a time-series SOURCE row from Op::TsScan/prior Op::Window where id=ts, score=value; TsScan's ns ts are normalized to seconds) and PRODUCES one row per non-empty tumbling bucket (id=aligned bucket start, score=aggregate) via eg-tsdb time_bucket, composing downstream into Rank/Limit. Closes the TsScan → Window gap (a TsScan row's numeric id was not a node, so Window dropped every row). Wired under timeseries; a non-timeseries build passes through epistemic-graph
CONCEPT:EG-KG.compute.trailing-aggregate-selector-lowers Selectable WINDOW aggregate: wire::Op::WindowAgg { secs, agg } (UQL WINDOW <dur> <agg>) runs the SAME tumbling windower as Op::Window with a chosen eg-tsdb Aggmean/avg, sum, min, max, count, first, last (unknown ⇒ mean). The unit is matched before the aggregate, so WINDOW 30 min is 30 minutes; WINDOW 30 s min is a 30-second min-aggregate. Base query wire variant; the aggregate is wired under timeseries (passthrough otherwise) epistemic-graph
CONCEPT:EG-KG.compute.negative-vector-component-parses Negative vector components in UQL RANK BY ~[…]: the parser's parse_vector_ref accepts a leading - (a Tok::Dash) before a numeric component, so RANK BY ~[-0.1, 0.2, -0.3] parses to the SAME Op::Rank { query } the Rust builder / wire DTO always accepted — closing a lexer/parser asymmetry (builder took negatives, UQL rejected them). Proven by a UQL-vs-builder equality test epistemic-graph
CONCEPT:EG-KG.compute.fuse-stage-now-dispatches UQL FUSE stage dispatch: parse_stage now dispatches FUSE, and parse_fuse lowers FUSE [branch] [branch] … (each bracketed sub-pipeline a Vec<Op> branch) to the SAME Op::FuseRrf { branches, k: 0.0 } the builder/wire construct (k=0.0 ⇒ the canonical eg_text::RRF_K). RRF fusion was builder/wire-only though the grammar listed it — closing a surface asymmetry. Feature-gated to text (mirrors TEXT); a non-text build errors with a clear "not in this build" message. Proven by a UQL-vs-builder equality test epistemic-graph
CONCEPT:EG-KG.ontology.string-type-iri-class String-type ↔ IRI-class bridge for REASON: the OWL reasoner's membership resolution bridges a node's bare string type (e.g. {"type":"Widget"}) to the OWL class IRI. Rule — the bridge base is the NAMESPACE of the REASON target IRI (up to the last / or #); a bare local-name t maps to <base + t>. Applied in eg-rdf asserted_types_* (optional class_base) and in reason_source (base derived from the target IRI), so REASON <http://ex/Device> includes a {"type":"Sensor"} node when <http://ex/Sensor> ⊑ <http://ex/Device>. Base absent ⇒ byte-for-byte the prior bare-string behavior epistemic-graph
CONCEPT:EG-KG.query.eg-8 mysql-wire cross-modal txn roundtrip TEST (per-surface parity, north-star EG-KG.compute.eg): tests/mysql_roundtrip.rs hand-rolls the MySQL Handshake-v10 / COM_QUERY text-protocol client over a raw TcpStream (no mysql client crate — the Pi-contract idiom) and drives the real mysql_wire::serve_with_auth listener, proving the MySQL wire inherits the shared-WireSession cross-modal seam (EG-KG.compute.subsystems-reference/EG-KG.txn.isolation-ryow-begin-set). Mirrors the pgwire cases — mysql_wire_txn_update_then_cross_modal_read (in-txn UPDATE + SET EMBEDDING both read back by an in-txn UQL cross-modal read, RYOW) and mysql_wire_txn_crossmodal_ryow_isolated_until_commit (BEGIN; SET EMBEDDING; INSERT INTO series; <UQL join>; COMMIT reads its own writes while a second connection sees none until COMMIT). Test-only; no src change epistemic-graph
CONCEPT:EG-KG.query.per-surface-parity mssql-wire (TDS) cross-modal txn roundtrip TEST (per-surface parity, north-star EG-KG.compute.eg): extends tests/mssql_roundtrip.rs with a reusable connect (PRELOGIN+LOGIN7 trust) + batch (SQLBatch → walk COLMETADATA/ROW/DONE, loud panic on an ERROR token) helper over the existing hand-rolled raw-TcpStream TDS client, proving the MSSQL TDS wire inherits the shared-WireSession cross-modal seam (EG-KG.compute.subsystems-reference/EG-KG.txn.isolation-ryow-begin-set). Mirrors the pgwire cases — tds_txn_update_then_cross_modal_read (in-txn UPDATE + SET EMBEDDING read back by an in-txn UQL cross-modal read, RYOW) and tds_txn_crossmodal_ryow_isolated_until_commit (BEGIN; SET EMBEDDING; INSERT INTO series; <UQL join>; COMMIT reads its own writes while a second TDS connection sees none until COMMIT). Test-only; no src change epistemic-graph
CONCEPT:EG-KG.query.eg-9 GraphQL cross-modal txn seam: eg_graphql::crossmodal gives the GraphQL surface the EG-359..363 in-txn cross-modal verbs — stageEmbedding / addMeasurement / sparqlUpdate / sparqlConstruct staging + an in-txn unifiedQuery read + commitTransaction. Because eg-graphql sits BELOW the facade in the crate DAG, it routes onto the SAME LOWER primitives the facade's GraphTxnState/run_unified_overlaid are built on — GraphView::overlay_* + eg_core::compute::semantic::semantic_overlay + eg_plan::execute over an eg_plan::uql-parsed plan + eg_rdf CONSTRUCT/UPDATE lowering — not the facade wrappers (which are ABOVE it, unreachable without a dependency cycle). No plan/lowering logic duplicated. Gated by the crossmodal sub-feature (default-on; rides into graphql/node/full via the facade's default-features pull) epistemic-graph
CONCEPT:EG-KG.compute.eg-178 GraphQL cross-modal multi-request txnId handle: GraphQL over HTTP has no session, so beginTransaction mints a txnId into an eg_graphql::CrossModalTxnRegistry (a Mutex<HashMap>, the SAME stateful-registry idiom as ApqRegistry) that a server carrier holds across a connection's requests; later stage* / unifiedQuery / commitTransaction / rollbackTransaction mutations carry the id. In-txn reads see the txn's own staged writes (RYOW); an off-txn / other-txn read sees the committed store only (isolation). This is the achievable multi-request form (the crate PROVIDES the session via the registry, so no batched single-op fallback is needed) epistemic-graph
CONCEPT:EG-KG.compute.series-name-its GraphQL cross-modal tsdb leg: addMeasurement stages a time-series point into the txn and the in-txn unifiedQuery threads an eg_plan::StagedSeries overlay (CONCEPT:EG-KG.query.txn-tsdb-read-your parity) so an Op::TsScan reads the txn's own uncommitted points. Gated by the crossmodal-tsdb sub-feature (default-OFF, parity with the facade's tsdb-gated TxnAddMeasurement); without it addMeasurement returns a clear "not built" error, never a silent drop. Enabled from the facade's tsdb graphql tier at reconcile epistemic-graph
CONCEPT:EG-KG.compute.eg-179 GraphQL cross-modal roundtrip proof: eg-graphql/src/crossmodal.rs tests drive a full multi-request txn — begin → stageEmbedding + sparqlUpdate → in-txn unifiedQuery returns read-your-own-writes (the staged embedding scores ≈1.0 under MATCH (:Person) |> RANK BY ~[…]) → a second (empty) txn's read + the committed store are ISOLATED from the staged writes → commitTransaction makes both the vector and graph modalities visible. A crossmodal-tsdb-gated test proves the addMeasurement StagedSeries overlay is read by a TsScan (RYOW). Rollback discards epistemic-graph
CONCEPT:EG-KG.compute.eg-180 GraphQL cross-modal facade reconcile hooks (engine 2.10.0): (1) durable atomic commit — the GraphQL HTTP carrier's commitTransaction converts the staged CrossModalTxn into a facade GraphTxnState and calls commit_cross_modal_txn (one redb WriteTransaction across ALL modalities incl. tsdb SERIES), exactly as pgwire's commit_txn_state; until then the in-crate commit is the in-memory tier (graph+vector to GraphCore); (2) the facade enables eg-graphql/crossmodal-tsdb from its tsdb graphql tier so addMeasurement rides full; (3) hoist the facade's pub(crate) handlers::txn::triples_to_methods into eg-rdf as pub so eg-graphql and handlers::txn share ONE triples→property-graph lowering instead of the mirror kept in crossmodal::lower_triples epistemic-graph
CONCEPT:EG-KG.compute.asof-valid-t-sources Advanced cross-modal test — bitemporal multi-hop: AsOf(source) → Rank → Reason<iri> → Traverse (temporal × vector × OWL × graph, 2-hop subclass, string→IRI bridge) over a hand-built PlanCtx. Only members INFERRED-and-LIVE-at-t reach their CITES targets in vector-rank order; a later instant re-selects a smaller live member set (crates/eg-plan/src/advanced_crossmodal_tests.rs) epistemic-graph
CONCEPT:EG-KG.compute.one-plan-fuses-four Advanced cross-modal test — federation fusion: Scan → Filter(DataFusion SQL) → ForeignScan(join) → Rank (graph × relational × foreign × vector) equals the manual join; plus a fail-closed proof that a Named foreign source with no registry attached errors cleanly (the SSRF/fail-closed analog at the plan seam; host-level SSRF is enforced+tested at the server surface) epistemic-graph
CONCEPT:EG-KG.compute.spatialscan-packed-hilbert-r Advanced cross-modal test — geo × vector × temporal: SpatialScan(Hilbert-R-tree bbox) → Filter(SpatialDWithin) → Rank → AsOf{Valid} narrows spatially, refines by planar distance, ranks by similarity, then keeps only temporally-live places; a later instant drops the expired one epistemic-graph
CONCEPT:EG-KG.compute.tensorscan-frame-seeds Advanced cross-modal test — tensor × graph × vector + CAS dedup: TensorScan → TensorOp(Reduce, CAS write-back) → Traverse<DERIVES> → Rank; the derived frames traverse into their docs (vector-ranked) while three identical reductions content-address to ONE dedup'd CAS blob (EG-304) epistemic-graph
CONCEPT:EG-KG.compute.scan-event-seeds-stream Advanced cross-modal test — CEP × graph × tsdb: Scan(Event) → Cep(login→logout sequence) → Traverse<TRIGGERS> → Window — the matched session's events traverse to their sensor readings, then a native eg-tsdb window aggregates them (a dangling non-matching event's reading is excluded from the window) — all in ONE plan epistemic-graph
CONCEPT:EG-KG.compute.reason-scholarlywork-sources Advanced cross-modal test — probabilistic × OWL × vector + MMR: Reason<iri> → Rank → Probabilistic(Expectation) → RankMmr reranks an OWL-reasoned, vector-filtered set by each row's stored distribution mean, then MMR promotes an orthogonal member above a near-duplicate (uncertainty × OWL × vector × diversity) epistemic-graph
CONCEPT:EG-KG.txn.one-transaction-stage-five Advanced cross-modal roundtrip (capstone) — 5-modality in-txn RYOW → atomic commit → consistent re-read: a graph node + vector embedding + graph edge + native tsdb measurement + OWL axiom staged in ONE txn, all visible to in-txn cross-modal queries (RYOW) and invisible off-txn, committed atomically (graph/vector/axiom durable-in-memory), then re-read off-txn incl REASON over the committed TBox (tests/advanced_crossmodal_roundtrip.rs, native dispatch harness) epistemic-graph
CONCEPT:EG-KG.txn.serializable-txn-captures-predicate Advanced cross-modal roundtrip — concurrent SERIALIZABLE across modalities: a serializable txn captures a predicate read-set (label=Sensor), a concurrent txn commits a phantom Sensor, and the serializable txn's commit re-evaluates the predicate, detects the phantom, and rolls back (Commit → false) — its staged cross-modal write never lands (tests/advanced_crossmodal_roundtrip.rs) epistemic-graph
CONCEPT:EG-KG.compute.eg-181 Advanced cross-modal roundtrip (OPEN, #[ignore]) — RLS per-agent visibility on a fused Reason→Rank + the overlay path. filter_view exists (crates/eg-core/src/isolation.rs); the RLS-fixture + caller-threading harness is the deferred lift epistemic-graph
CONCEPT:EG-KG.compute.eg-182 Advanced cross-modal roundtrip (OPEN, #[ignore]) — pgwire + /sparql + native consistent snapshot after a mixed cross-modal commit; needs the multi-listener (pgwire + sparql-http) test harness. Committed machinery exists (EG-KG.txn.isolation-ryow-begin-set) epistemic-graph
CONCEPT:EG-KG.compute.eg-183 Advanced cross-modal roundtrip (OPEN, #[ignore]) — encryption-at-rest + cross-modal read; a wrong key FAILS (no silent plaintext). Needs a keyed RedbBackend open/reopen roundtrip; the security-encrypted backend exists epistemic-graph
CONCEPT:EG-KG.compute.eg-184 Advanced cross-modal roundtrip (OPEN, #[ignore]) — streaming/CDC → live materialized cross-modal view rebuild; needs a CDC subscription (CdcHub) + MatViewStore rebuild assertion after a cross-modal commit. Both surfaces exist epistemic-graph
CONCEPT:EG-KG.compute.eg-185 Advanced cross-modal roundtrip (OPEN, #[ignore]) — cross-shard txn spanning modalities under Raft, kill coordinator mid-2PC → single decision. GENUINE GAP: no in-process multi-node openraft cluster + 2PC-kill test harness exists in tests/ epistemic-graph
CONCEPT:EG-KG.compute.eg-186 Advanced cross-modal roundtrip (OPEN, #[ignore]) — KV-cache warm-fork fan-out reusing cross-modal context. CROSS-REPO GAP: the warm-fork sandbox primitive (ForkableSandbox/WarmParentRegistry, ORCH-1.86..93) lives in agent-utilities, not epistemic-graph epistemic-graph
CONCEPT:EG-KG.compute.eg-187 Native-RPC routing for the extended cross-modal staging methods (leak closed, surfaced by the EG-KG.txn.one-transaction-stage-five capstone): TxnAddMeasurement/TxnAxiom/TxnConstruct handlers existed in handlers/txn.rs (EG-KG.backend.cross-modal-atomic-commit/361/362) but dispatch.rs never routed them → they errored on the native RPC surface while working over pgwire (EG-KG.txn.isolation-ryow-begin-set). Fix: three cfg-gated (tsdb/owl/sparql) dispatch arms routing each to handlers::txn::try_handle. Closes EG-KG.backend.cross-modal-atomic-commit/361/362 at the RPC surface (the "seamless — every surface" principle) epistemic-graph
CONCEPT:EG-KG.compute.highest-roi-proof-from Differential multi-surface oracle (query-surface harness): the highest-ROI review item — a curated suite of hybrid queries each executed via MULTIPLE surfaces (UQL text eg_plan::uql::parse, the Rust Plan/Op builder, and the separate-surfaces oracle eg_plan::oracle::separate_surfaces) asserting IDENTICAL plan AST + byte-identical RowSet (ids + vector scores), catching any seam where one surface lowers to a different plan/result. Covers MATCH→WHERE→TRAVERSE→RANK→AS OF→LIMIT and REASON→RANK→TRAVERSE. crates/eg-plan/tests/differential_oracle.rs epistemic-graph
CONCEPT:EG-KG.compute.systematically-exercise Cross-modal composition matrix: systematically exercises source-op × transform-op combinations + 3–5-op mixed chains with the review's edge cases (empty set, single row, high cardinality, low-selectivity RANK, deep TRAVERSE, REASON feeding TRAVERSE/RANK, timeseries sources feeding downstream) — each asserts no panic + a structurally sane RowSet (unique ids, LIMIT bounds the count, ids in the node universe). crates/eg-plan/tests/composition_matrix.rs epistemic-graph
CONCEPT:EG-KG.compute.eg-188 Property-based query-surface proofs: a constrained proptest generator emits RANDOM but VALID UQL pipelines and asserts the engine invariants — no panic / well-formed RowSet, determinism (same plan → same RowSet, the read side of snapshot isolation), LIMIT after RANK is a stable prefix, and a safe FILTER/AS OF commute in the non-emptying regime. crates/eg-plan/tests/plan_proptest.rs (proptest dev-dep) epistemic-graph
CONCEPT:EG-KG.storage.eg Plan-snapshot regression: insta snapshots of the logical plan for ~8 curated hybrid queries (each also asserted UQL-vs-builder identical) + the cost-reordered plans, so BOTH a lowering regression AND an optimizer-quality regression (a selective filter silently stops being pushed ahead of RANK) trip a snapshot diff. crates/eg-plan/tests/plan_snapshots.rs (insta dev-dep) epistemic-graph
CONCEPT:EG-KG.compute.eg-189 FINDING (open, north_star): the UQL RANK BY ~[…] vector-literal parser rejects a NEGATIVE component (expected a vector component, found -) — no unary-minus path in parse_vector_ref/the number lexer — while the Rust builder and wire plan DTO accept negatives. A surface expressiveness asymmetry: a query vector with any negative dimension is expressible via builder/RPC but NOT via UQL text. Surfaced by plan_proptest; tracked as an open row in docs/north_star.md epistemic-graph
CONCEPT:EG-KG.compute.eg-190 FINDING (open, north_star): op composition is NOT freely commutative — the executor's "empty candidate RowSet ⇒ a downstream Filter/AsOf/Reason re-seeds as a SOURCE" convention means [Filter, AsOf][AsOf, Filter] when the first op empties the set. Documented design (empty = source, so a bare Reason/AsOf is a leaf), but the algebraic commute law holds only in the non-emptying regime — a real caveat for any op-reordering optimizer. Pinned by plan_proptest::empty_intermediate_reseeds_source_breaks_commute; open row in docs/north_star.md epistemic-graph
CONCEPT:EG-KG.query.pgwire-leg Differential wire-surface oracle (pgwire leg): reuses the live pgwire listener in tests/pgwire_roundtrip.rs to prove the SQL-wire and UQL-wire surfaces agree on the SAME filter (type='Agent' AND rank>1 == MATCH (:Agent) WHERE rank > 1) and that the UQL wire surface is deterministic — the wire leg of the EG-KG.compute.highest-roi-proof-from multi-surface interchangeability proof epistemic-graph
CONCEPT:EG-KG.query.facade-reconcile-hook GraphQL cross-modal facade DURABLE-commit wiring — closes the EG-KG.compute.eg-180 reconcile hook the eg-graphql crate left open. Because eg-graphql sits BELOW the facade in the crate DAG, its in-crate commitTransaction lands graph+vector in-memory only; the facade GraphQL carrier (src/server/handlers/query.rs Method::GraphQl) now routes a commitTransaction to the DURABLE path. New crate API: CrossModalTxnRegistry::take + pub CrossModalTxn/GraphWrite accessors + classify_crossmodal(src) -> CrossModalRoute {Commit(txnId)|Staging|NotCrossModal} (one parse). New facade fn handlers::txn::commit_graphql_cross_modal converts the staged CrossModalTxn into a GraphTxnState (graph → AddNode/AddEdge, embeddings → stage_vector, tsdb batches → stage_measurement) and lands ALL modalities in ONE redb WriteTransaction via commit_cross_modal_txn — exactly as pgwire's commit_txn_state. begin/stage/read/rollback verbs run in-memory over a process-wide OnceLock registry; ordinary mutations keep execute_mutation. The facade full tier enables eg-graphql/crossmodal-tsdb so addMeasurement rides full. Proven durable (survives a reopen) by tests/graphql_crossmodal_durable.rs epistemic-graph
CONCEPT:ECO-4.0 Unified Toolkit Ingestion agent-utilities
CONCEPT:KG-2.0 Knowledge Graph Core Core Architecture agent-utilities
CONCEPT:ORCH-1.0 Multi-Agent Orchestration Abstraction agent-utilities
CONCEPT:KG-2.7 Batch Materialization / Local SPARQL Fast Path agent-utilities
CONCEPT:KG-2.8 Code/Test Enrichment & Interlinking (incl. 2.8r cross-file call/import resolution) agent-utilities
CONCEPT:KG-2.171 Cross-graph union reads (point/label/neighbor reads unioned across a graph set, deduped by id) epistemic-graph
CONCEPT:KG-2.178 Internal SQL query surface (read-only SELECT … FROM nodes over one graph via DataFusion, behind the query feature) epistemic-graph
CONCEPT:KG-2.176 Lazy secondary label index (label → node ids) for O(1) label lookup, invalidated on write epistemic-graph
CONCEPT:KG-2.177 Pluggable PersistenceBackend + durable redb write-through tier (behind the redb feature) epistemic-graph
CONCEPT:KG-2.199 Bounded, demand-driven secondary PROPERTY equality index (key → value → node ids) + DataFusion predicate pushdown epistemic-graph
CONCEPT:KG-2.213 Unified IndexManager seam: ONE registry over the label/property/vector/ontology indexes (SecondaryIndex trait + index_for/descriptors_for_column); eg-query pushdown consults ONE PushdownRegistry. See docs/architecture/index_manager.md epistemic-graph
CONCEPT:KG-2.132 One-round-trip hybrid discovery (Method::Discover { keywords, query_embedding, k }): dense-retrieve candidates via the HNSW batch primitive, re-rank each by BOTH semantic similarity AND lexical keyword overlap over name/description/type, and return the top-k hydrated as [{id,name,description,type,score}]. Complements SemanticSearch (bare (id,score)) by folding in the keyword signal + hydrating result text in a single call; an empty embedding degrades to a bounded keyword-only scan epistemic-graph
CONCEPT:EG-340 PL/pgSQL procedural interpreter: CREATE FUNCTION … LANGUAGE plpgsql stored in the durable function catalog and executed on a bare SELECT fn(args) / CALL proc(args), with embedded SQL run back through the existing read path (crates/eg-query/src/sql/plpgsql.rs, folded into the sql feature, no new deps) epistemic-graph
CONCEPT:EG-341 PL/pgSQL control-flow: DECLARE vars, BEGIN..END, := assignment, IF/ELSIF/ELSE, LOOP/WHILE/FOR … LOOP (integer range), EXIT/CONTINUE, RETURN, RAISE, and SELECT … INTO var over a variable environment — a hand-written statement interpreter over the tokenized body epistemic-graph
\n## CONCEPT:KG-2.20\nRust-Native Finance Compute Suite
\n## CONCEPT:KG-2.22\nGraph Network Protocols
\n## CONCEPT:KG-2.22\nData Science Primitives
\n## CONCEPT:KG-2.21\nAST Ingestion Pipeline
CONCEPT:EG-365 Cross-modal SEAM regression: planner mid-pipeline composition proofs (Reason→Traverse→Rank == reasoner∘BFS∘kNN oracle; SensorFuse→Rank / TensorScan→Rank downstream composition; reason-seeded cost reorder plan-shape) epistemic-graph
CONCEPT:EG-366 Bitemporal SEAM: decay-sweep (Ebbinghaus) re-weights confidence while AsOf{Valid} re-selects liveness and an in-between UPDATE flips a validity window — asserts confidence values + liveness at two instants epistemic-graph
CONCEPT:EG-367 Vector⇄reasoning cross-txn write→read consistency: a freshly written node (embedding + reasoner-subsumed type) is BOTH ANN-ranked AND reasoner-included by the next [Reason|>Rank] plan; an embedding UPDATE is reflected by the next ANN pass epistemic-graph
CONCEPT:EG-368 SPARQL-UPDATE→reasoning cross-txn visibility: a DELETE/INSERT WHERE re-parenting a TBox axiom is visible to the very next OWL EL⁺ classification epistemic-graph
CONCEPT:EG-369 Cache-coherence SEAM: the version-keyed SQL result cache HITS at the old version and MISSES + returns fresh rows after a version-bumping write; cache ON == cache OFF byte-identical epistemic-graph
CONCEPT:EG-370 Served E2E cross-modal SEAM: add(embedding+type)→reason→discover returns the fresh node; a SPARQL-style axiom UPDATE is visible to BOTH owl_reason AND hybrid discover epistemic-graph
CONCEPT:EG-359 In-txn cross-modal RYOW unified query: Method::TxnUnifiedQuery{txn_id,plan,…} + TxnUnifiedQueryText{txn_id,text,…} run the SAME wire::Plan/UQL over a committed-store snapshot OVERLAID with the txn's staged write-set, so staged nodes/edges/embeddings are visible to THIS txn pre-commit and invisible off-txn until commit (Gap 1, Lanes 0/A) epistemic-graph
CONCEPT:EG-360 In-txn time-series measurement staging: Method::TxnAddMeasurement{txn_id,series,points,graph} stages a tsdb batch that lands in the SAME redb WriteTransaction as the txn's graph/vector/blob writes at commit (Gap 2, Lanes 0/B) epistemic-graph
CONCEPT:EG-361 In-txn OWL axiom staging: Method::TxnAxiom{txn_id,turtle,graph} lowers staged Turtle axioms to graph node/edge writes in the SAME atomic commit WriteTransaction so the OWL reasoner sees them consistently (Gap 2, Lanes 0/B) epistemic-graph
CONCEPT:EG-362 In-txn SPARQL CONSTRUCT staging: Method::TxnConstruct{txn_id,sparql,graph} lowers the CONSTRUCT's produced triples to graph node/edge writes in the SAME atomic commit WriteTransaction (Gap 2, Lanes 0/B) epistemic-graph
CONCEPT:EG-363 TSDB-in-plan source op: wire::Op::TsScan{series,from,to} seeds the cross-modal RowSet from the native eg-tsdb SeriesStore over [from,to) (via PlanCtx.tsdb/with_tsdb), so a downstream Rank/Limit/Filter fuses the tsdb leg with graph/vector/relational legs in ONE plan (Gap 3, Lanes 0/C) epistemic-graph
CONCEPT:EG-371 One full-featured build (tier collapse): default == fullcargo build links every MAIN feature that compiles without an external GPU/robotics toolchain; the former pi/pi-max/node deployment tiers (and the shared-wheel-filename collision) are removed. cluster (openraft HA) and full-extras (GPU/ROS2) remain opt-in build layers, NOT published wheels. Preserved invariants: no pyo3 (bindings=bin), no openraft in default, no cudarc/rustdds in default epistemic-graph
CONCEPT:EG-372 pgwire cross-modal txn seam: the PGWIRE TEXT entrypoint for the EG-359..363 in-txn cross-modal seam. The shared WireSession recognizes UQL …, SET EMBEDDING FOR <id> = <vec>, INSERT INTO series …, SPARQL UPDATE …, SPARQL CONSTRUCT … inside a BEGIN…COMMIT and routes them onto the committed RPC seam — staging via txn.rs GraphTxnState (vector/measurement + lowered axiom/CONSTRUCT methods), reading in-txn via the extracted run_unified overlay (read-your-own-writes across graph+vector+OWL), and committing every modality atomically via the extracted commit_cross_modal_txn (now in-memory-tolerant on a no-persistence engine). Off-txn cross-modal writes auto-commit as one atomic statement. No plan/txn logic duplicated; lives on the EG-074 shared core so the multi-wire family inherits it epistemic-graph
CONCEPT:EG-373 North Star: Seamless — every discovered cross-modal seam is fully implemented at EVERY surface (RPC/native, all SQL wires, SPARQL, GraphQL), never merely flagged. docs/north_star.md tracks the seam backlog: RPC cross-modal txn (done, EG-359..363), pgwire cross-modal txn (done, EG-372), GraphQL cross-modal txn (done in-memory tier, EG-379..383; durable-commit + tsdb tier = facade reconcile), mysql/mssql-wire wire-test pending, advanced cross-modal tests open epistemic-graph
CONCEPT:EG-374 In-txn tsdb read-your-own-writes staged-series overlay: an in-memory eg_plan::StagedSeries (series → staged (ts_ns, values) points) attached to PlanCtx via with_staged_series; Op::TsScan (tsdb_scan_op) MERGES the staged overlay (RYOW precedence on a ts collision) with the committed eg-tsdb SeriesStore, so an in-txn UQL TsScan reads the txn's own uncommitted GraphTxnState.measurements while an off-txn read sees committed only. run_unified_overlaid builds the overlay from the resolved txn's staged measurements. Closes the north_star in-txn tsdb-RYOW open row epistemic-graph
CONCEPT:EG-375 REASON <iri> mid-plan: the UQL lexer tokenizes an angle-bracketed IRI (<scheme:…>) as Tok::Iri and parse_reason accepts it, so REASON <http://…/Class> is expressible at the UQL/pgwire surface (previously only a bare label). Composes mid-pipeline via the existing reason_op FILTER branch — a MATCH … |> RANK … |> REASON <http://…/Class> |> TRAVERSE intersects the ranked candidate set with the reasoned members of that explicit IRI class epistemic-graph
CONCEPT:EG-411 Server-side embedder seam for UQL RANK BY ~ "text": a quoted rank ref lowers to wire::Op::RankEmbed { text }, resolved at exec time by a TextEmbedder (embed(text) -> Vec<f32>) bound on the PlanCtx via with_embedder; the executor turns the text into a query vector kNN-ranked over the SemanticStore exactly like a literal-vector Op::Rank. With NO embedder bound Op::RankEmbed is a clean typed error (never a panic). Dep-free trait; a deterministic HashEmbedder fallback exists for tests/offline. A bare-ident handle (~handle) stays a reserved forward seam epistemic-graph
CONCEPT:EG-412 Facade injection point for the UQL embedder seam: run_unified (src/server/handlers/query.rs) binds the process-wide server-side TextEmbedder onto the PlanCtx (with_embedder) so a served RANK BY ~ "text" (Op::RankEmbed) resolves its vector. The engine stores embeddings but produces them client-side today, so no in-process model ships by default (unbound ⇒ clean typed error); a real embedding model is wired HERE, and the deterministic HashEmbedder fallback is opt-in via EG_UQL_TEXT_EMBEDDER=hash epistemic-graph
CONCEPT:EG-413 Real WINDOW tumbling time-series aggregate: Op::Window { secs } no longer passes rows through — it CONSUMES a RowSet of (ts,value) rows (a graph-node row's valid_from+score/value, OR a time-series SOURCE row from Op::TsScan/prior Op::Window where id=ts, score=value; TsScan's ns ts are normalized to seconds) and PRODUCES one row per non-empty tumbling bucket (id=aligned bucket start, score=aggregate) via eg-tsdb time_bucket, composing downstream into Rank/Limit. Closes the TsScan → Window gap (a TsScan row's numeric id was not a node, so Window dropped every row). Wired under timeseries; a non-timeseries build passes through epistemic-graph
CONCEPT:EG-414 Selectable WINDOW aggregate: wire::Op::WindowAgg { secs, agg } (UQL WINDOW <dur> <agg>) runs the SAME tumbling windower as Op::Window with a chosen eg-tsdb Aggmean/avg, sum, min, max, count, first, last (unknown ⇒ mean). The unit is matched before the aggregate, so WINDOW 30 min is 30 minutes; WINDOW 30 s min is a 30-second min-aggregate. Base query wire variant; the aggregate is wired under timeseries (passthrough otherwise) epistemic-graph
CONCEPT:EG-417 Negative vector components in UQL RANK BY ~[…]: the parser's parse_vector_ref accepts a leading - (a Tok::Dash) before a numeric component, so RANK BY ~[-0.1, 0.2, -0.3] parses to the SAME Op::Rank { query } the Rust builder / wire DTO always accepted — closing a lexer/parser asymmetry (builder took negatives, UQL rejected them). Proven by a UQL-vs-builder equality test epistemic-graph
CONCEPT:EG-418 UQL FUSE stage dispatch: parse_stage now dispatches FUSE, and parse_fuse lowers FUSE [branch] [branch] … (each bracketed sub-pipeline a Vec<Op> branch) to the SAME Op::FuseRrf { branches, k: 0.0 } the builder/wire construct (k=0.0 ⇒ the canonical eg_text::RRF_K). RRF fusion was builder/wire-only though the grammar listed it — closing a surface asymmetry. Feature-gated to text (mirrors TEXT); a non-text build errors with a clear "not in this build" message. Proven by a UQL-vs-builder equality test epistemic-graph
CONCEPT:EG-376 String-type ↔ IRI-class bridge for REASON: the OWL reasoner's membership resolution bridges a node's bare string type (e.g. {"type":"Widget"}) to the OWL class IRI. Rule — the bridge base is the NAMESPACE of the REASON target IRI (up to the last / or #); a bare local-name t maps to <base + t>. Applied in eg-rdf asserted_types_* (optional class_base) and in reason_source (base derived from the target IRI), so REASON <http://ex/Device> includes a {"type":"Sensor"} node when <http://ex/Sensor> ⊑ <http://ex/Device>. Base absent ⇒ byte-for-byte the prior bare-string behavior epistemic-graph
CONCEPT:EG-377 mysql-wire cross-modal txn roundtrip TEST (per-surface parity, north-star EG-373): tests/mysql_roundtrip.rs hand-rolls the MySQL Handshake-v10 / COM_QUERY text-protocol client over a raw TcpStream (no mysql client crate — the Pi-contract idiom) and drives the real mysql_wire::serve_with_auth listener, proving the MySQL wire inherits the shared-WireSession cross-modal seam (EG-074/EG-372). Mirrors the pgwire cases — mysql_wire_txn_update_then_cross_modal_read (in-txn UPDATE + SET EMBEDDING both read back by an in-txn UQL cross-modal read, RYOW) and mysql_wire_txn_crossmodal_ryow_isolated_until_commit (BEGIN; SET EMBEDDING; INSERT INTO series; <UQL join>; COMMIT reads its own writes while a second connection sees none until COMMIT). Test-only; no src change epistemic-graph
CONCEPT:EG-378 mssql-wire (TDS) cross-modal txn roundtrip TEST (per-surface parity, north-star EG-373): extends tests/mssql_roundtrip.rs with a reusable connect (PRELOGIN+LOGIN7 trust) + batch (SQLBatch → walk COLMETADATA/ROW/DONE, loud panic on an ERROR token) helper over the existing hand-rolled raw-TcpStream TDS client, proving the MSSQL TDS wire inherits the shared-WireSession cross-modal seam (EG-074/EG-372). Mirrors the pgwire cases — tds_txn_update_then_cross_modal_read (in-txn UPDATE + SET EMBEDDING read back by an in-txn UQL cross-modal read, RYOW) and tds_txn_crossmodal_ryow_isolated_until_commit (BEGIN; SET EMBEDDING; INSERT INTO series; <UQL join>; COMMIT reads its own writes while a second TDS connection sees none until COMMIT). Test-only; no src change epistemic-graph
CONCEPT:EG-379 GraphQL cross-modal txn seam: eg_graphql::crossmodal gives the GraphQL surface the EG-359..363 in-txn cross-modal verbs — stageEmbedding / addMeasurement / sparqlUpdate / sparqlConstruct staging + an in-txn unifiedQuery read + commitTransaction. Because eg-graphql sits BELOW the facade in the crate DAG, it routes onto the SAME LOWER primitives the facade's GraphTxnState/run_unified_overlaid are built on — GraphView::overlay_* + eg_core::compute::semantic::semantic_overlay + eg_plan::execute over an eg_plan::uql-parsed plan + eg_rdf CONSTRUCT/UPDATE lowering — not the facade wrappers (which are ABOVE it, unreachable without a dependency cycle). No plan/lowering logic duplicated. Gated by the crossmodal sub-feature (default-on; rides into graphql/node/full via the facade's default-features pull) epistemic-graph
CONCEPT:EG-380 GraphQL cross-modal multi-request txnId handle: GraphQL over HTTP has no session, so beginTransaction mints a txnId into an eg_graphql::CrossModalTxnRegistry (a Mutex<HashMap>, the SAME stateful-registry idiom as ApqRegistry) that a server carrier holds across a connection's requests; later stage* / unifiedQuery / commitTransaction / rollbackTransaction mutations carry the id. In-txn reads see the txn's own staged writes (RYOW); an off-txn / other-txn read sees the committed store only (isolation). This is the achievable multi-request form (the crate PROVIDES the session via the registry, so no batched single-op fallback is needed) epistemic-graph
CONCEPT:EG-381 GraphQL cross-modal tsdb leg: addMeasurement stages a time-series point into the txn and the in-txn unifiedQuery threads an eg_plan::StagedSeries overlay (CONCEPT:EG-374 parity) so an Op::TsScan reads the txn's own uncommitted points. Gated by the crossmodal-tsdb sub-feature (default-OFF, parity with the facade's tsdb-gated TxnAddMeasurement); without it addMeasurement returns a clear "not built" error, never a silent drop. Enabled from the facade's tsdb graphql tier at reconcile epistemic-graph
CONCEPT:EG-382 GraphQL cross-modal roundtrip proof: eg-graphql/src/crossmodal.rs tests drive a full multi-request txn — begin → stageEmbedding + sparqlUpdate → in-txn unifiedQuery returns read-your-own-writes (the staged embedding scores ≈1.0 under MATCH (:Person) |> RANK BY ~[…]) → a second (empty) txn's read + the committed store are ISOLATED from the staged writes → commitTransaction makes both the vector and graph modalities visible. A crossmodal-tsdb-gated test proves the addMeasurement StagedSeries overlay is read by a TsScan (RYOW). Rollback discards epistemic-graph
CONCEPT:EG-383 GraphQL cross-modal facade reconcile hooks (engine 2.10.0): (1) durable atomic commit — the GraphQL HTTP carrier's commitTransaction converts the staged CrossModalTxn into a facade GraphTxnState and calls commit_cross_modal_txn (one redb WriteTransaction across ALL modalities incl. tsdb SERIES), exactly as pgwire's commit_txn_state; until then the in-crate commit is the in-memory tier (graph+vector to GraphCore); (2) the facade enables eg-graphql/crossmodal-tsdb from its tsdb graphql tier so addMeasurement rides full; (3) hoist the facade's pub(crate) handlers::txn::triples_to_methods into eg-rdf as pub so eg-graphql and handlers::txn share ONE triples→property-graph lowering instead of the mirror kept in crossmodal::lower_triples epistemic-graph
CONCEPT:EG-384 Advanced cross-modal test — bitemporal multi-hop: AsOf(source) → Rank → Reason<iri> → Traverse (temporal × vector × OWL × graph, 2-hop subclass, string→IRI bridge) over a hand-built PlanCtx. Only members INFERRED-and-LIVE-at-t reach their CITES targets in vector-rank order; a later instant re-selects a smaller live member set (crates/eg-plan/src/advanced_crossmodal_tests.rs) epistemic-graph
CONCEPT:EG-385 Advanced cross-modal test — federation fusion: Scan → Filter(DataFusion SQL) → ForeignScan(join) → Rank (graph × relational × foreign × vector) equals the manual join; plus a fail-closed proof that a Named foreign source with no registry attached errors cleanly (the SSRF/fail-closed analog at the plan seam; host-level SSRF is enforced+tested at the server surface) epistemic-graph
CONCEPT:EG-386 Advanced cross-modal test — geo × vector × temporal: SpatialScan(Hilbert-R-tree bbox) → Filter(SpatialDWithin) → Rank → AsOf{Valid} narrows spatially, refines by planar distance, ranks by similarity, then keeps only temporally-live places; a later instant drops the expired one epistemic-graph
CONCEPT:EG-387 Advanced cross-modal test — tensor × graph × vector + CAS dedup: TensorScan → TensorOp(Reduce, CAS write-back) → Traverse<DERIVES> → Rank; the derived frames traverse into their docs (vector-ranked) while three identical reductions content-address to ONE dedup'd CAS blob (EG-304) epistemic-graph
CONCEPT:EG-388 Advanced cross-modal test — CEP × graph × tsdb: Scan(Event) → Cep(login→logout sequence) → Traverse<TRIGGERS> → Window — the matched session's events traverse to their sensor readings, then a native eg-tsdb window aggregates them (a dangling non-matching event's reading is excluded from the window) — all in ONE plan epistemic-graph
CONCEPT:EG-389 Advanced cross-modal test — probabilistic × OWL × vector + MMR: Reason<iri> → Rank → Probabilistic(Expectation) → RankMmr reranks an OWL-reasoned, vector-filtered set by each row's stored distribution mean, then MMR promotes an orthogonal member above a near-duplicate (uncertainty × OWL × vector × diversity) epistemic-graph
CONCEPT:EG-390 Advanced cross-modal roundtrip (capstone) — 5-modality in-txn RYOW → atomic commit → consistent re-read: a graph node + vector embedding + graph edge + native tsdb measurement + OWL axiom staged in ONE txn, all visible to in-txn cross-modal queries (RYOW) and invisible off-txn, committed atomically (graph/vector/axiom durable-in-memory), then re-read off-txn incl REASON over the committed TBox (tests/advanced_crossmodal_roundtrip.rs, native dispatch harness) epistemic-graph
CONCEPT:EG-392 Advanced cross-modal roundtrip — concurrent SERIALIZABLE across modalities: a serializable txn captures a predicate read-set (label=Sensor), a concurrent txn commits a phantom Sensor, and the serializable txn's commit re-evaluates the predicate, detects the phantom, and rolls back (Commit → false) — its staged cross-modal write never lands (tests/advanced_crossmodal_roundtrip.rs) epistemic-graph
CONCEPT:EG-391 Advanced cross-modal roundtrip (GREEN) — RLS per-agent visibility on a fused Reason→Rank + the overlay path. Closed by the RLS fixture (identities + _owner/_visibility node blobs) + caller-threaded agent_id in tests/advanced_crossmodal_roundtrip.rs; the overlay-leg seam is EG-KG.query.overlay-leg-rls-filter epistemic-graph
CONCEPT:EG-393 Advanced cross-modal roundtrip (GREEN) — pgwire + /sparql + native consistent snapshot after a mixed cross-modal commit. Closed by the multi-listener harness (EG-KG.query.tri-surface-snapshot-harness): pgwire + sparql-http + native dispatch on ONE ServerState, all three observe the same committed snapshot epistemic-graph
CONCEPT:EG-394 Advanced cross-modal roundtrip (GREEN) — encryption-at-rest + cross-modal read; a wrong key FAILS (no silent plaintext). Closed by the keyed RedbBackend open/reopen roundtrip (EG-KG.storage.encryption-reopen-roundtrip) epistemic-graph
CONCEPT:EG-395 Advanced cross-modal roundtrip (GREEN) — streaming/CDC → live materialized cross-modal view rebuild. Closed by a CdcHub continuous-query subscription maintained incrementally off a cross-modal write's change stream (EG-KG.query.cdc-live-view-rebuild); the MatViewStore variant needs the cluster-tier compute-dist/raft layer (not in full) epistemic-graph
CONCEPT:EG-396 Advanced cross-modal roundtrip — cross-shard txn spanning modalities under Raft, kill coordinator mid-2PC → single decision. GREEN under --features cluster via the in-crate raft::xshard_modality_harness (CONCEPT:EG-KG.txn.crossshard-2pc-modality-harness); the advanced-roundtrip spec is now cfg(feature="cluster")-gated, no longer #[ignore]d. Real-hardware remainder (cross-node participants, ANN/TSDB cross-shard modalities) documented on the new concept epistemic-graph
CONCEPT:EG-397 Advanced cross-modal roundtrip (OPEN, #[ignore]) — KV-cache warm-fork fan-out reusing cross-modal context. CROSS-REPO GAP: the warm-fork sandbox primitive (ForkableSandbox/WarmParentRegistry, ORCH-1.86..93) lives in agent-utilities, not epistemic-graph (refined in EG-KG.query.warmfork-fanout-open-reason) epistemic-graph
CONCEPT:EG-398 Native-RPC routing for the extended cross-modal staging methods (leak closed, surfaced by the EG-390 capstone): TxnAddMeasurement/TxnAxiom/TxnConstruct handlers existed in handlers/txn.rs (EG-360/361/362) but dispatch.rs never routed them → they errored on the native RPC surface while working over pgwire (EG-372). Fix: three cfg-gated (tsdb/owl/sparql) dispatch arms routing each to handlers::txn::try_handle. Closes EG-360/361/362 at the RPC surface (the "seamless — every surface" principle) epistemic-graph
CONCEPT:EG-400 Differential multi-surface oracle (query-surface harness): the highest-ROI review item — a curated suite of hybrid queries each executed via MULTIPLE surfaces (UQL text eg_plan::uql::parse, the Rust Plan/Op builder, and the separate-surfaces oracle eg_plan::oracle::separate_surfaces) asserting IDENTICAL plan AST + byte-identical RowSet (ids + vector scores), catching any seam where one surface lowers to a different plan/result. Covers MATCH→WHERE→TRAVERSE→RANK→AS OF→LIMIT and REASON→RANK→TRAVERSE. crates/eg-plan/tests/differential_oracle.rs epistemic-graph
CONCEPT:EG-401 Cross-modal composition matrix: systematically exercises source-op × transform-op combinations + 3–5-op mixed chains with the review's edge cases (empty set, single row, high cardinality, low-selectivity RANK, deep TRAVERSE, REASON feeding TRAVERSE/RANK, timeseries sources feeding downstream) — each asserts no panic + a structurally sane RowSet (unique ids, LIMIT bounds the count, ids in the node universe). crates/eg-plan/tests/composition_matrix.rs epistemic-graph
CONCEPT:EG-402 Property-based query-surface proofs: a constrained proptest generator emits RANDOM but VALID UQL pipelines and asserts the engine invariants — no panic / well-formed RowSet, determinism (same plan → same RowSet, the read side of snapshot isolation), LIMIT after RANK is a stable prefix, and a safe FILTER/AS OF commute in the non-emptying regime. crates/eg-plan/tests/plan_proptest.rs (proptest dev-dep) epistemic-graph
CONCEPT:EG-403 Plan-snapshot regression: insta snapshots of the logical plan for ~8 curated hybrid queries (each also asserted UQL-vs-builder identical) + the cost-reordered plans, so BOTH a lowering regression AND an optimizer-quality regression (a selective filter silently stops being pushed ahead of RANK) trip a snapshot diff. crates/eg-plan/tests/plan_snapshots.rs (insta dev-dep) epistemic-graph
CONCEPT:EG-404 FINDING (open, north_star): the UQL RANK BY ~[…] vector-literal parser rejects a NEGATIVE component (expected a vector component, found -) — no unary-minus path in parse_vector_ref/the number lexer — while the Rust builder and wire plan DTO accept negatives. A surface expressiveness asymmetry: a query vector with any negative dimension is expressible via builder/RPC but NOT via UQL text. Surfaced by plan_proptest; tracked as an open row in docs/north_star.md epistemic-graph
CONCEPT:EG-405 FINDING (open, north_star): op composition is NOT freely commutative — the executor's "empty candidate RowSet ⇒ a downstream Filter/AsOf/Reason re-seeds as a SOURCE" convention means [Filter, AsOf][AsOf, Filter] when the first op empties the set. Documented design (empty = source, so a bare Reason/AsOf is a leaf), but the algebraic commute law holds only in the non-emptying regime — a real caveat for any op-reordering optimizer. Pinned by plan_proptest::empty_intermediate_reseeds_source_breaks_commute; open row in docs/north_star.md epistemic-graph
CONCEPT:EG-406 Differential wire-surface oracle (pgwire leg): reuses the live pgwire listener in tests/pgwire_roundtrip.rs to prove the SQL-wire and UQL-wire surfaces agree on the SAME filter (type='Agent' AND rank>1 == MATCH (:Agent) WHERE rank > 1) and that the UQL wire surface is deterministic — the wire leg of the EG-400 multi-surface interchangeability proof epistemic-graph
CONCEPT:EG-419 GraphQL cross-modal facade DURABLE-commit wiring — closes the EG-383 reconcile hook the eg-graphql crate left open. Because eg-graphql sits BELOW the facade in the crate DAG, its in-crate commitTransaction lands graph+vector in-memory only; the facade GraphQL carrier (src/server/handlers/query.rs Method::GraphQl) now routes a commitTransaction to the DURABLE path. New crate API: CrossModalTxnRegistry::take + pub CrossModalTxn/GraphWrite accessors + classify_crossmodal(src) -> CrossModalRoute {Commit(txnId)|Staging|NotCrossModal} (one parse). New facade fn handlers::txn::commit_graphql_cross_modal converts the staged CrossModalTxn into a GraphTxnState (graph → AddNode/AddEdge, embeddings → stage_vector, tsdb batches → stage_measurement) and lands ALL modalities in ONE redb WriteTransaction via commit_cross_modal_txn — exactly as pgwire's commit_txn_state. begin/stage/read/rollback verbs run in-memory over a process-wide OnceLock registry; ordinary mutations keep execute_mutation. The facade full tier enables eg-graphql/crossmodal-tsdb so addMeasurement rides full. Proven durable (survives a reopen) by tests/graphql_crossmodal_durable.rs epistemic-graph
CONCEPT:EG-KG.query.hybrid-latency-benches Hybrid-query latency benchmarks Criterion (p50/mean) over the representative fused pipelines — MATCH→TRAVERSE→RANK, REASON→RANK→TRAVERSE (owl), TsScan→WINDOW (timeseries), FUSE RRF (text) — on a fixed synthetic dataset. harness=false bench, required-features=["query"] so a default cargo bench skips it. crates/eg-plan/benches/hybrid_queries.rs
CONCEPT:EG-KG.query.recall-oracle-bench Vector-leg recall@k vs brute-force oracle The ANN/HNSW semantic_search top-k vs the true brute-force cosine top-k over the SAME vectors; the bench ASSERTS recall ≥ a committed floor (0.80) and writes the value to EG_BENCH_RECALL_OUT for the CI gate. Deterministic (fixed data + HNSW params), so a drop is a real regression, not noise. crates/eg-plan/benches/hybrid_queries.rs
CONCEPT:EG-KG.query.synthetic-bench-dataset Deterministic bench dataset builder A seeded LCG (no rand dep, no checked-in corpus) generates N Doc nodes (CITES chain + skip fan) with dim-d embeddings + a fixed query vector, so latency and recall are byte-reproducible across runs. crates/eg-plan/benches/hybrid_queries.rs
CONCEPT:EG-KG.query.perf-recall-ci-gate CI perf/recall regression gate A benchmarks job in .github/workflows/rust-ci.yml runs the bench, then scripts/bench_gate.py compares each bench's criterion p50 (target/criterion/<b>/new/estimates.json) to generous committed ceilings + the measured recall to the floor (crates/eg-plan/benches/thresholds.json). Recall floor = the tight deterministic check; latency ceilings = a catastrophic-regression net (non-flaky). Absent benches skipped.
CONCEPT:EG-KG.query.pipeline-fuzz Query-pipeline fuzzing A proptest generator emits random VALID UQL pipelines up to 16 stages (longer than plan_proptest) over 512 cases against a denser graph, asserting no-panic/no-error, unique ids (consistent state), trailing-LIMIT bound, and determinism (same plan+snapshot → byte-identical RowSet = read-side snapshot isolation). crates/eg-plan/tests/fuzz_pipelines.rs
CONCEPT:EG-KG.query.concurrency-chaos-fuzz Concurrency chaos variant A batch of random pipelines executed serially (reference) and concurrently across 8 threads sharing ONE immutable GraphView snapshot + SemanticStore (thread::scope, no Arc); every concurrent result must equal its serial reference — reads are isolated under contention (incl. the lazy HNSW-build race). The query-workload nemesis run. crates/eg-plan/tests/fuzz_pipelines.rs
CONCEPT:EG-KG.query.libfuzzer-parser-deferred libFuzzer UQL-parser target — DONE (was deferred). Now built as EG-KG.query.uql-libfuzzer-parse-target (crates/eg-plan/fuzz/); the UNSTRUCTURED byte-level complement to the STABLE proptest VALID-pipeline harness (EG-KG.query.pipeline-fuzz). epistemic-graph
CONCEPT:EG-KG.query.uql-libfuzzer-parse-target libFuzzer UQL-parser fuzz target (BUILT) — a cargo-fuzz/libFuzzer bin (crates/eg-plan/fuzz/fuzz_targets/uql_parse.rs, target uql_parse) feeds ARBITRARY BYTES (UTF-8-guarded) to eg_plan::uql::parse, asserting the hand-written lexer/parser is total (Ok(Plan)|Err(UqlError), never panic/hang) on malformed input. The fuzz crate is a DETACHED workspace ([workspace] table, libfuzzer-sys 0.4 + eg-plan path dep) so it stays OFF the stable cargo test gate (libFuzzer needs nightly). Run: cargo install cargo-fuzz; cargo +nightly fuzz run uql_parse -- -runs=10000. Documented in docs/benchmarks.md. Closes EG-KG.query.libfuzzer-parser-deferred epistemic-graph
CONCEPT:EG-KG.query.overlay-leg-rls-filter RLS on the STAGED-OVERLAY leg of an in-txn fused read (closes EG-391): run_unified_overlaid (src/server/handlers/query.rs) already filter_viewd the COMMITTED base snapshot before overlaying the txn's staged write-set, so a staged owned+private node (_owner/_visibility blob) leaked through an in-txn Reason→Rank. Fix: a second rls.filter_view(caller, &mut view) AFTER overlay_write_set, so BOTH the committed and staged legs honor per-agent row visibility. No-op when no RLS rules are registered (has_rules()==false short-circuits), so the single-tenant RYOW path is byte-for-byte unchanged epistemic-graph
CONCEPT:EG-KG.query.tri-surface-snapshot-harness Tri-surface consistent-snapshot harness (closes EG-393): a pgwire listener (+ real tokio-postgres client), a /sparql HTTP listener, and the native dispatch path bound to ONE in-process ServerState. After a single mixed cross-modal BEGIN…COMMIT (graph node + embedding + a second node + a subClassOf graph edge + an OWL axiom) all three surfaces observe the SAME committed snapshot, and before the commit none of the three sees any of it (no surface sees a torn/partial state). tests/advanced_crossmodal_roundtrip.rs, gated pgwire+sparql-http epistemic-graph
CONCEPT:EG-KG.storage.encryption-reopen-roundtrip Encryption-at-rest keyed open/reopen cross-modal roundtrip (closes EG-394): EPISTEMIC_GRAPH_ENCRYPTION_KEY=K1 opens a keyed RedbBackend, a commit_crossmodal (node + edge + vector embedding) seals value blobs with ChaCha20-Poly1305; reopened with K1 the fused read_graph_dump (nodes+edges+semantic) DECRYPTS, and reopened with a WRONG key K2 read_node ERRORS (unseal wrong-key) rather than returning plaintext. The raw .redb bytes never contain the plaintext secret. A serialized env toggle (tokio Mutex) so the process-global key never races. tests/advanced_crossmodal_roundtrip.rs, gated security+redb epistemic-graph
CONCEPT:EG-KG.query.cdc-live-view-rebuild Streaming/CDC → live materialized cross-modal view rebuild (closes EG-395): a CdcHub continuous query (the streaming-native live view available in the full build) subscribed to state.cdc is maintained INCREMENTALLY off the CDC change stream a cross-modal write (graph nodes + vector embedding + graph edge) emits, and a subsequent native cross-modal read reflects the same new state. The MatViewStore variant needs the cluster-tier compute-dist/raft layer (NOT in full), so the continuous-query is the live materialized view under full. tests/advanced_crossmodal_roundtrip.rs, gated streaming epistemic-graph
CONCEPT:EG-KG.query.crossshard-2pc-open-reason RESOLVED by EG-KG.txn.crossshard-2pc-modality-harness — EG-396 is now GREEN under --features cluster; the advanced-roundtrip spec is cfg(feature="cluster")-gated (compiles out of --features full, runs under --features cluster), no longer #[ignore]d epistemic-graph
CONCEPT:EG-KG.txn.crossshard-2pc-modality-harness In-process cross-shard 2PC modality-spanning coordinator-kill harness (GREEN, closes EG-396) — src/raft/xshard_modality_harness.rs (cfg compute-dist, which cluster implies) stands up a live in-process node running TWO openraft groups on one redb-authoritative store and runs a cross-shard txn SPANNING TWO MODALITIES across the groups: property-graph (group A, Method::AddNode) + RDF/triple (group B, Method::AddTriples projected via wal::applyeg_rdf::mapping::load_triples). Four scenarios prove a SINGLE all-or-nothing decision: (1) happy commit_cross_shard lands both; (2) participant killed during PREPARE → abort, no partial commit; (3) coordinator killed AFTER the COMMIT decision (full node+backend drop) → recover_in_doubt re-applies BOTH (COMMIT); (4) coordinator killed BEFORE any decision → presumed-abort → NEITHER. Uses the phase-granular prepare_only/decide_only (cfg widened to compute-dist) + recover_in_doubt; exposed as pub async fn prove_crossshard_modality_2pc_single_decision(), driven by the cfg(feature="cluster") roundtrip spec + crate #[cfg(test)] scenarios (cargo test --features cluster). REAL-HARDWARE REMAINDER: (a) cross-NODE participants (all groups in ONE process — cross-host RPC/clock-skew soak needs multi-node hardware, cf. scripts/validate-raft-cluster.sh + AGENTS.md); (b) ANN vector + TSDB measurement modalities span shards only once the coordinator carries staged vector/measurement batches into the slice apply — today a SINGLE-graph cross-modal redb-WriteTransaction barrier (handlers::txn::commit_cross_modal) epistemic-graph
CONCEPT:EG-KG.query.warmfork-fanout-open-reason Refined OPEN reason for EG-397 (KV-cache warm-fork fan-out): a CROSS-REPO gap — the warm-FORK sandbox primitive (ForkableSandbox/WarmParentRegistry/forkserver, ORCH-1.86..93) lives in agent-utilities, not this engine. Reachable engine-side seams are the KV page store (eg-kvcache EG-364) + the cross-modal read surface, but NEITHER is a fork primitive (no in-engine os.fork/forkserver/CoW child sharing a warm parent's read context while isolating divergent writes). The fan-out test belongs to the agent-utilities warm-fork surface CONSUMING this engine's cross-modal context. Kept #[ignore]d epistemic-graph
CONCEPT:EG-KG.query.advanced-specs-green-umbrella Handoff-1 track F umbrella — the achievable #[ignore]d advanced cross-modal roundtrip specs (EG-391/393/394/395) turned GREEN by building the missing test-surface fixtures/harnesses (EG-KG.query.overlay-leg-rls-filter..EG-KG.query.cdc-live-view-rebuild), the two genuinely-hard rows (EG-396/EG-397) kept #[ignore]d with refined precise reasons (EG-KG.query.crossshard-2pc-open-reason/EG-KG.query.warmfork-fanout-open-reason). tests/advanced_crossmodal_roundtrip.rs + docs/north_star.md epistemic-graph
CONCEPT:EG-KG.query.usecase-agent-memory High-value use-case validation suite #1 — agent-memory retrieval (tests/usecase_agent_memory.rs). Episodes/events ingested as graph nodes + embeddings + a small OWL ontology, then ONE fused unified-query plan through the real eg_plan::execute engine honoring semantic (Op::Rank), lexical BM25 (Op::RankText over a live eg_text::TextIndex), graph expansion (Op::Traverse/Op::RankNodeDistance), OWL inference (Op::Reason), bi-temporal AS OF (Op::AsOf{Valid}), and RRF fusion (Op::FuseRrf) — asserting the fused rank honors every signal and beats any single modality. Confidence-decay proven deterministically via eg_rdf::owl::fact_confidence; its non-foldability into one plan is EG-KG.query.decay-not-foldable-finding epistemic-graph
CONCEPT:EG-KG.query.usecase-codebase-intel High-value use-case validation suite #2 — codebase / repository intelligence (tests/usecase_codebase_intel.rs). Code structure (File/Function nodes, CALLS/IMPORTS edges) as a graph + signature embeddings + a small language/framework ontology + changed-at valid-time metadata, then a hybrid plan "functions similar to X (Op::Rank), related via call-graph to Y (Op::Traverse CALLS), changed recently (Op::AsOf{Valid}), matching ontology concept Z (Op::Reason)" — asserting the fused answer honors vector + call-graph + temporal + OWL simultaneously epistemic-graph
CONCEPT:EG-KG.query.usecase-hybrid-rag-analytics High-value use-case validation suite #3 — hybrid RAG + in-engine analytics (tests/usecase_hybrid_rag_analytics.rs). Retrieve candidates via a graph+vector+text fused plan (Op::FuseRrf) through eg_plan::execute, hydrate the joined embeddings from the in-engine SemanticStore, then run in-engine PCA + kmeans via the real server dispatch (Method::DsPca/DsKMeans) — compute-near-data, no data leaves the process. Asserts the Rank→analytics seam: analytics run over exactly the fused-retrieval result set, kmeans labels separate the two semantic groups, PCA explained-variance concentrates on the dominant axis epistemic-graph
CONCEPT:EG-KG.query.usecase-observability-rca High-value use-case validation suite #4 — observability / root-cause analysis (tests/usecase_observability_rca.rs). Logs/traces as Document nodes + error-log embeddings + a Service dependency graph (DEPENDS_ON) + native metrics in an eg_tsdb::store::SeriesStore, then a fused plan traversing the dependency graph while ranking by error-log vector similarity (Op::Traverse + Op::FuseRrf[Op::Rank,Op::RankNodeDistance]) to surface the culprit service, plus a native Op::TsScanOp::WindowAgg window over the incident that flags the metric anomaly — asserting graph traversal + vector similarity + temporal windowing all contribute epistemic-graph
CONCEPT:EG-KG.query.usecase-kg-lifecycle High-value use-case validation suite #5 — KG lifecycle with validation & inference (tests/usecase_kg_lifecycle.rs). SHACL validation (Method::ShaclValidate — bad shape rejected, conformant one passes) + OWL reasoning + graph mutations + vector-index maintenance staged in ONE ACID txn (BeginTxnTxnAddNode/TxnAddEdge/TxnAddEmbedding/TxnAxiomCommit) over the real dispatch server, then asserting inference closure (Op::Reason infers the new instance a member of the committed super-class) AND vector re-indexing (new embedding kNN-retrievable) hold off-txn after commit, all under a CONCURRENT hybrid read that sees a consistent (never torn) snapshot epistemic-graph
CONCEPT:EG-KG.query.decay-not-foldable-finding FINDING (open, north_star) surfaced by use-case suite #1 (EG-KG.query.usecase-agent-memory): confidence time-decay cannot be folded into a single fused unified-query plan. The plan executor's Op::Reason passes now=0 to the confidence reasoner (decay-neutral by design so a bare Reason is a stable leaf), so bi-temporal AS OF liveness AND Ebbinghaus confidence-decay re-weighting cannot BOTH be expressed in one Op chain — decay must run via the separate OwlReason Method / eg_rdf::owl::instances_of_weighted surface threading a wall-clock now. Fix: a plan op/field carrying an explicit now/half_life. RESOLVED by EG-KG.query.reason-decay-in-plan (row now done in docs/north_star.md) epistemic-graph
CONCEPT:EG-KG.query.served-text-index-unbound-finding FINDING (north_star) surfaced by use-case suites #1/#3 (EG-KG.query.usecase-agent-memory/EG-KG.query.usecase-hybrid-rag-analytics): the served run_unified bound no BM25 TextIndex onto the PlanCtx, so a served UnifiedQuery/UnifiedQueryText whose plan contains Op::RankText or an Op::FuseRrf text branch degraded to ZERO lexical hits (documented at query.rs:705, KG-2.215 increment 2). The lexical leg is fully wired at the eg_plan::execute engine surface (PlanCtx::with_text) but was a no-op through the served RPC surface. RESOLVED by EG-KG.query.served-text-index-binding (row now done in docs/north_star.md) epistemic-graph
CONCEPT:EG-KG.query.reason-decay-in-plan Confidence time-decay folded IN-PLAN (resolves EG-KG.query.decay-not-foldable-finding). PlanCtx gains an owl-gated decay: Option<(now, half_life)> bound via PlanCtx::with_decay; the reason path (exec.rs::reason_op/reason_source) threads it into eg_rdf::owl::asserted_types_with_confidence_from_view/instances_of_weighted, so an Op::Reason computes Ebbinghaus-decayed OWL membership confidence in-plan and composes alongside Op::AsOf in ONE fused plan. None (default) ⇒ decay-neutral (now=0,half_life=0) — a bare Reason stays a stable leaf (byte-for-byte prior behavior). The SAME plan at two now values yields different decayed scores. crates/eg-plan/src/exec.rs. epistemic-graph
CONCEPT:EG-KG.query.served-text-index-binding Served lexical index binding (resolves EG-KG.query.served-text-index-unbound-finding). src/server/handlers/query.rs::run_unified builds a snapshot-derived eg_text::TextIndex from the off-lock GraphView node blobs (concatenating every string leaf) and binds it via PlanCtx::with_text, only when the plan references a text op (plan_needs_text), so a served Op::RankText/Op::FuseRrf text branch returns REAL BM25 lexical hits instead of ZERO. Snapshot-consistent + deterministic (BM25 over a doc set), so byte-identical ranking to a persistent index; a persistent index-on-write is a pure-perf follow-up. src/server/handlers/query.rs. epistemic-graph
CONCEPT:EG-KG.query.served-plan-optimize-routing Served plan routed through the FULL optimizer (No-Legacy migration). run_unified no longer runs the bespoke single CostModel::reorder_filter_rank swap; it hands the plan to eg_plan::execute, whose plan_optimize runs the full cross-modal optimizer (filter/AsOf-before-Rank + Reason↔Rank + FuseRrf branch reorder). reorder_filter_selectivity becomes a legacy no-op hint the optimizer's own derived selectivity supersedes (kept in the wire DTO for compatibility). Every rule is answer-preserving within the EG-405 guard, so served RESULTS are unchanged (differential). src/server/handlers/query.rs. epistemic-graph
CONCEPT:EG-KG.query.empty-set-commutativity INTENTIONAL semantics (resolves EG-405/EG-KG.compute.eg-190): the executor's "an EMPTY RowSet flips a downstream Filter/AsOf/Reason into SOURCE mode" convention is what lets a BARE op be a leaf source — NOT a bug. The commute law holds only in the non-emptying regime; the cost optimizer already guards it (adjacent narrower-vs-Rank only, input from a source, both intermediates ≥ 1 row, never narrower-vs-narrower). Documented in docs/uql.md; the witnesses plan_proptest::empty_intermediate_reseeds_source_breaks_commute + filter_and_asof_commute_in_nonempty_regime are kept as the spec. crates/eg-plan/src/optimizer.rs + crates/eg-plan/tests/plan_proptest.rs. epistemic-graph
CONCEPT:EG-KG.ontology.owl-proof-tree-explanation OWL proof-tree explanation service (Track J) Stardog's flagship "explanation" feature, native here. eg_rdf::owl::Classification::explain(sub, sup) -> Option<ProofNode> reconstructs the FULL recursive proof tree for a derived named-class subsumption from the Justifications the EL⁺/RL forward-chaining completion already records during saturation (rule + axiom labels + premise subsumptions) — NO re-derivation, pure reconstruction of the closure's own justification DAG, recursively down to an asserted LEAF (a reflexive seed A⊑A/A⊑⊤ or a base fact with no recorded justification). explain_instance composes an asserted ABox type fact with the TBox subsumption proof for an instance-membership explanation (rule = "CR-instance"). Surfaced as Method::OwlExplain { ontology, sub, sup }OwlExplainResult { found, tree, consistent, unsatisfiable } (the recursive ProofNodeWire wire projection) + a Python RdfClient.explain. Read-only; gated owl (same Pi-safe/pure-Rust gate as OwlReason). crates/eg-rdf/src/owl.rs, src/server/handlers/rdf.rs.
CONCEPT:EG-KG.ontology.justification-tracking Justification tracking in the EL⁺/RL completion Every DERIVED (non-reflexive) subsumption Classification::add_sub records inserts a Justification { rule, axioms, premises } keyed (sub, sup) — the completion rule that fired (CR-sub/CR-conj⁻/CR-some⁺/CR-some⁻/CR-chain/CR-subrole/CR-bot/CR-disjoint/CR-allValues), the human-readable axiom label(s) it cited, and the prior subsumption(s) (premises) it consumed. This is the DAG Classification::explain/explain_instance reconstruct (CONCEPT:EG-KG.ontology.owl-proof-tree-explanation) — the tracking was already present pre-Track-J (unit-tested at owl.rs el_derives_existential_restriction_subsumption/domain_range_derives_and_justifies), Track J adds the reconstruction + wire surface consuming it. crates/eg-rdf/src/owl.rs.
CONCEPT:EG-KG.query.r2rml-virtual-graph OBDA / R2RML virtual graphs (Track J) Ontology-Based Data Access: crates/eg-rdf/src/obda.rs exposes a foreign tabular source as a VIRTUAL RDF graph queried via SPARQL WITHOUT ever materializing the whole dataset — a TriplesMap (subject-IRI template + optional class + predicate→ObjectMap list, R2RML's rr:TriplesMap) over a named ObdaSource (an ObdaSourceRegistry-resolved TableSource/ClosureSource/engine adapter). parse_r2rml_turtle reads a STANDARD R2RML Turtle mapping document (rr:logicalTable/rr:subjectMap/rr:predicateObjectMap, typed/lang/IRI-template object maps, rr:class/rr:object shortcuts) into the SAME model; parse_mapping is the lightweight EG-101 textual form. Wired to the engine's OWN SQL user-table store (eg_query::TableStore, feature query) via a TableStoreSource ObdaSource adapter (src/server/handlers/rdf.rs) — the query rewrites to a projection-pushed table scan (CONCEPT:EG-KG.query.obda-query-rewrite) and evaluates the existing SPARQL engine over a TRANSIENT materialized view (never persisted). Surfaced as Method::SparqlVirtual { query, mapping, tables } + a Python RdfClient.sparql_virtual. Gated obda (= sparql + query). DEFERRED: a live EXTERNAL relational source (only the engine's own tables are wired today), rr:sqlQuery logical views, SPARQL→SQL predicate pushdown (today the pushdown stops at column projection — the WHOLE table's rows are scanned, then only the needed COLUMNS are kept), rr:joinCondition.
CONCEPT:EG-KG.query.obda-query-rewrite OBDA query-rewrite mechanics The 4-step rewrite run_outcome_virtual performs for every virtual-graph query: (1) walk the parsed SPARQL algebra's BGPs to learn the CONSTANT predicates it references (wanted_predicates/collect_predicates; a variable predicate or property path conservatively disables the narrowing, keeping completeness); (2) VirtualGraph::materialize computes the minimal needed-column set per TriplesMap (subject-template columns ∪ the WANTED predicate-object maps' columns) and scans ONLY those from the ObdaSource (projection pushdown); (3) apply the subject/predicate/object templates to each returned row, producing ONLY the query-relevant triples; (4) load them into a transient GraphView and run the UNCHANGED SPARQL evaluator over it, so BGP joins/FILTER/OPTIONAL/aggregates/projection all compose exactly as over a native graph. Proven by eg101_predicate_pushdown_narrows_columns_and_triples (a query naming one predicate scans/materializes only that predicate's triples) and the eg101_bgp_two_patterns_join_over_virtual_graph/eg305_r2rml_ref_template_joins cross-TriplesMap join tests. crates/eg-rdf/src/obda.rs.