A

Delivery structure

1 specs, 10 requirements, 1 with no code mapped
Knuckles-Team/webarchiverSpec delivery
Specs (1)
  • CONN-WEBARCHIVER-WEB-SOURCE10
Records not done
  • Requirementsall slices in all specs10
  • Requirements not donenot VERIFIED2
  • Specs not donewhole specs only1

Each spec shows its requirement count. Panel J lists every record.

B

Delivery states

count of total

Requirements (10)

SPECIFIED (specified)2 of 10 requirements
VERIFIED (verified)8 of 10 requirements

Specs (1)

SPECIFIED (specified)1 of 1 specs

Each bar shows one state against all records of the same kind.

Percentages count current rows. Done means VERIFIED.

C

State definitions

delivery_state
StateDefinition
SPECIFIEDThe requirement is written. No code for it is on the default branch.
LANDEDA commit that carries its ID in a Spec trailer is on the default branch.
VERIFIEDIt is LANDED and a test bound to its ID exists.

A spec state is the lowest state of its rows. Done means VERIFIED.

Invalid state: 0

D

Specifications

specs/*/status.json
Spec / titleReq.LandedDoneSpec stateCode references
CONN-WEBARCHIVER-WEB-SOURCECONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connector108/108SPECIFIEDspec.md documentation: file exists; coverage not verified
requirements.md documentation: file exists; coverage not verified
test-spec.md documentation: file exists; coverage not verified

Landed counts LANDED and VERIFIED requirements. Done counts VERIFIED requirements only.

The spec state comes from the spec status file only.

U

Spec architecture

repository / component packages

Packages group specs by repository and declared component. Unassigned means no component was declared. Code links do not prove coverage. Mermaid source.

Knuckles-Team/webarchiver / Unassigned (1)
Knuckles-Team/webarchiver / Unassigned (1)CONN-WEBARCHIVER-WEB-SOURCE

CONN-WEBARCHIVER-WEB-SOURCE: no plan diagram declared.

R

Spec relationships

declared dependencies and references

Depends on means a declared prerequisite. Extends and references do not imply prerequisite order. Shared code never creates dependencies. External targets are source declarations, not verified delivery.

Knuckles-Team/webarchiver / Unassigned
SpecRelationshipTargetEvidence
CONN-WEBARCHIVER-WEB-SOURCEextendsSDK-SOURCE-INGESTKnuckles-Team/agent-connector-sdk: external declaration; not verified; source: specs/web-source/spec.md
CONN-WEBARCHIVER-WEB-SOURCEreferencesARBORA-OPSKnuckles-Team/arbora: external declaration; not verified; source: specs/web-source/spec.md
G

Open issues and pull requests

open at capture

Issues

0 open at capture.

    Pull Requests

    0 open at capture.

      The build reads GitHub one time. This page makes no live API requests.

      J

      Record inventory

      11 records

      Each status file gives one spec row. Each requirement gives one more row. A record without a valid state shows INVALID. Expand a Definition to read the canonical requirement and verification text.

      All documented records (11 rows)
      ID / sourceKindParentStateCommit / test evidence
      CONN-WEBARCHIVER-WEB-SOURCE
      CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connector
      specNoneSPECIFIEDNo recorded commit/test evidence
      CONN-WEBARCHIVER-WEB-SOURCE-R001
      Pages answer through the web-source contract (rollup)
      Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source
      **Pages answer through the web-source contract (rollup).** The webarchiver connector must implement the web-source contract of `SDK-SOURCE-INGEST-R005.40`. It serves the per-operation tools `web_fetch`, `web_extract`, `web_render` and `web_screenshot`. It is the primary backend of `research.web` (`ARBORA-OPS-R067.3`). Children `.1` to `.8` and `.10` deliver it. | See each child.
      requirementCONN-WEBARCHIVER-WEB-SOURCESPECIFIEDNo recorded commit/test evidence
      CONN-WEBARCHIVER-WEB-SOURCE-R001.1
      Model
      Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source
      **Model.** `web_fetch` and `web_extract` are the SDK web-source contract's tools (`SDK-SOURCE-INGEST-R005.40.1`). `register_web_source_tools` serves them over `WebarchiverSource` in `webarchiver/api/web_source.py`. They take a `WebFetchRequest` and return a `WebPageResult` or a `WebFetchRefusal`, with `backend=webarchiver`. `web_render`, `web_screenshot`, `web_links` and `web_crawl` are the connector's own per-operation tools. Proving test: `tests/test_conformance.py::test_webarchiver_conforms_to_the_web_source_contract`. | The served tool schemas equal the contract's, and an unknown argument is refused.
      requirementCONN-WEBARCHIVER-WEB-SOURCEVERIFIED31e5998eddc5
      tests/test_conformance.py:20
      CONN-WEBARCHIVER-WEB-SOURCE-R001.2
      Behavior
      Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source
      **Behavior.** The connector must obey `robots.txt` for its user agent and cache each file for 3600 s. It must keep a per-domain rate limit of 1 request per second by default. It must stop reading at `max_bytes` and refuse an unlisted content type. `web_extract` must reduce the page to its main content as markdown. It must return the title, author, published date, canonical URL and outbound links. `content_digest` is the SHA-256 of the extracted text. Proving test: `tests/test_extract.py::test_main_content_and_metadata`. | Fixture pages yield the expected markdown and metadata, and a disallowed path is refused.
      requirementCONN-WEBARCHIVER-WEB-SOURCEVERIFIED8c38c25066e3
      tests/test_extract.py:103
      tests/test_extract.py:117
      tests/test_extract.py:133
      tests/test_extract.py:36
      tests/test_extract.py:64
      CONN-WEBARCHIVER-WEB-SOURCE-R001.3
      Wiring
      Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source
      **Wiring.** The browser capture in `Webarchiver.launch_browser` (`webarchiver/webarchiver.py:63`) serves `web_render` and `web_screenshot`. The server uses the SDK server factory and `register_tool_surface`. The production caller is `WebFetchSourceAdapter.extract` of `SDK-SOURCE-INGEST-R005.40.3`. Proving test: `tests/test_served.py::test_tools_call_reaches_extractor`. | A served `tools/call` reaches the extractor and the capture path.
      requirementCONN-WEBARCHIVER-WEB-SOURCEVERIFIED8c38c25066e3
      tests/test_served.py:34
      tests/test_served.py:52
      tests/test_served.py:69
      CONN-WEBARCHIVER-WEB-SOURCE-R001.4
      Refusal
      Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source
      **Refusal.** Each refusal must use a closed reason of the contract: `robots_disallowed`, `rate_limited`, `size_exceeded`, `unsupported_type`, `egress_denied`, `policy_refused`, `extract_empty` or `upstream_failed`. A private address must refuse through the SDK egress policy. The connector must never return an empty page as a success. Proving test: `tests/test_extract.py::test_refusals`. | Each case returns its reason and no page body.
      requirementCONN-WEBARCHIVER-WEB-SOURCEVERIFIED8c38c25066e3
      tests/test_extract.py:45
      CONN-WEBARCHIVER-WEB-SOURCE-R001.5
      Evidence
      Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source
      **Evidence.** Each result must name `backend=webarchiver`, the final URL, the HTTP status, the byte count, the content digest, the retrieval time and the `robots.txt` decision. Proving test: `tests/test_extract.py::test_result_provenance`. | Each named field is present and matches the fixture response.
      requirementCONN-WEBARCHIVER-WEB-SOURCEVERIFIED8c38c25066e3
      tests/test_extract.py:143
      CONN-WEBARCHIVER-WEB-SOURCE-R001.6
      Hot-path proof
      Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source
      **Hot-path proof.** The test serves the connector and runs the SDK conformance kit of `SDK-SOURCE-INGEST-R005.40.2` against it. It then runs `connector-sync --once` for one page into a running engine. Proving test: `tests/test_conformance_served.py::test_conformance_kit_and_sync`. | The kit reports no broken obligation, and the engine holds one Document with `backend=webarchiver`.
      requirementCONN-WEBARCHIVER-WEB-SOURCESPECIFIEDNo recorded commit/test evidence
      CONN-WEBARCHIVER-WEB-SOURCE-R001.7
      Bounded site crawl
      Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source
      **Bounded site crawl.** `web_crawl(seed_url, same_site, max_pages, max_depth, include, exclude)` in `webarchiver/api/crawl.py` must walk links breadth-first from one seed and return the page URLs it reached, in discovery order. It must fetch every page through the same policy as `web_fetch`: robots.txt, the per-domain rate limit, size and content-type caps and egress pinning. `same_site` defaults to true. `max_pages` ranges from 1 to 1000 and `max_depth` from 0 to 10. `include` and `exclude` are regular expressions. A robots-disallowed or failed page is reported as skipped with its closed reason. It returns URLs only; fetching content is the caller's next step. Proving test: `tests/test_crawl.py::test_bounded_crawl`. | A fixture site yields the expected URLs in order, the disallowed path is reported and never fetched, and out-of-range bounds refuse with `policy_refused`.
      requirementCONN-WEBARCHIVER-WEB-SOURCEVERIFIED4fde2c6fe305
      99c1281925fe
      tests/test_crawl.py:17
      tests/test_crawl.py:35
      tests/test_crawl.py:49
      CONN-WEBARCHIVER-WEB-SOURCE-R001.8
      Ingest mappings
      Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source
      **Ingest mappings.** `webarchiver/connector_manifest.yml` must declare `schema_mappings` for the `Document` and `Chunk` record types, mapped to the core `Document` and `Chunk` classes, so `SourceIngest` resolves `manifest:<connector>#schema_mappings/Document` and `.../Chunk` in the pack the fleet imports (`SDK-SOURCE-INGEST-R004.25`). Proving test: `tests/test_startup.py::test_manifest_maps_document_and_chunk_records`. | The served manifest maps both record types with their text, title, source and backend fields.
      requirementCONN-WEBARCHIVER-WEB-SOURCEVERIFIED482ceea819b0
      4d127d07b8ce
      tests/test_startup.py:51
      CONN-WEBARCHIVER-WEB-SOURCE-R001.10
      Conformance
      Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source
      **Conformance.** The connector passes the SDK conformance kit (`SDK-SOURCE-INGEST-R005.40.2`) against itself, served in process. The kit checks robots.txt, the per-domain rate limit, `max_bytes`, the content type, the digest and the tool schemas. The archivebox-api connector passes the same kit, so both produce the same tools and record shape (`CONN-ARCHIVEBOX-API-WEB-SOURCE-R001.8`). Proving test: `tests/test_conformance.py::test_webarchiver_conforms_to_the_web_source_contract`. | The kit reports no broken obligation.
      requirementCONN-WEBARCHIVER-WEB-SOURCEVERIFIED31e5998eddc5
      tests/test_conformance.py:21
      TitleSpec delivery: webarchiver
      RepositoryKnuckles-Team/webarchiver
      Commita784cc5a2fd7
      Captured (UTC)2026-10-18T14:27:05
      Snapshotsnapshot.json (v1)
      Sourcespecs/*/status.json
      Sheet1 of 1