- CONN-WEBARCHIVER-WEB-SOURCE10
- Requirementsall slices in all specs10
- Requirements not donenot VERIFIED2
- Specs not donewhole specs only1
Each spec shows its requirement count. Panel J lists every record.
Requirements (10)
Specs (1)
Each bar shows one state against all records of the same kind.
Percentages count current rows. Done means VERIFIED.
| State | Definition |
|---|---|
| SPECIFIED | The requirement is written. No code for it is on the default branch. |
| LANDED | A commit that carries its ID in a Spec trailer is on the default branch. |
| VERIFIED | It is LANDED and a test bound to its ID exists. |
A spec state is the lowest state of its rows. Done means VERIFIED.
Invalid state: 0
| Spec / title | Req. | Landed | Done | Spec state | Code references |
|---|---|---|---|---|---|
| CONN-WEBARCHIVER-WEB-SOURCECONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connector | 10 | 8/10 | 8 | SPECIFIED | spec.md documentation: file exists; coverage not verified requirements.md documentation: file exists; coverage not verified test-spec.md documentation: file exists; coverage not verified |
Landed counts LANDED and VERIFIED requirements. Done counts VERIFIED requirements only.
The spec state comes from the spec status file only.
Packages group specs by repository and declared component. Unassigned means no component was declared. Code links do not prove coverage. Mermaid source.
Knuckles-Team/webarchiver / Unassigned (1)
CONN-WEBARCHIVER-WEB-SOURCE: no plan diagram declared.
Depends on means a declared prerequisite. Extends and references do not imply prerequisite order. Shared code never creates dependencies. External targets are source declarations, not verified delivery.
Knuckles-Team/webarchiver / Unassigned
| Spec | Relationship | Target | Evidence |
|---|---|---|---|
| CONN-WEBARCHIVER-WEB-SOURCE | extends | SDK-SOURCE-INGEST | Knuckles-Team/agent-connector-sdk: external declaration; not verified; source: specs/web-source/spec.md |
| CONN-WEBARCHIVER-WEB-SOURCE | references | ARBORA-OPS | Knuckles-Team/arbora: external declaration; not verified; source: specs/web-source/spec.md |
Issues
0 open at capture.
Pull Requests
0 open at capture.
The build reads GitHub one time. This page makes no live API requests.
Each status file gives one spec row. Each requirement gives one more row. A record without a valid state shows INVALID. Expand a Definition to read the canonical requirement and verification text.
All documented records (11 rows)
| ID / source | Kind | Parent | State | Commit / test evidence |
|---|---|---|---|---|
| CONN-WEBARCHIVER-WEB-SOURCE CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connector | spec | None | SPECIFIED | No recorded commit/test evidence |
| CONN-WEBARCHIVER-WEB-SOURCE-R001 Pages answer through the web-source contract (rollup) Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source**Pages answer through the web-source contract (rollup).** The webarchiver connector must implement the web-source contract of `SDK-SOURCE-INGEST-R005.40`. It serves the per-operation tools `web_fetch`, `web_extract`, `web_render` and `web_screenshot`. It is the primary backend of `research.web` (`ARBORA-OPS-R067.3`). Children `.1` to `.8` and `.10` deliver it. | See each child. | requirement | CONN-WEBARCHIVER-WEB-SOURCE | SPECIFIED | No recorded commit/test evidence |
| CONN-WEBARCHIVER-WEB-SOURCE-R001.1 Model Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source**Model.** `web_fetch` and `web_extract` are the SDK web-source contract's tools (`SDK-SOURCE-INGEST-R005.40.1`). `register_web_source_tools` serves them over `WebarchiverSource` in `webarchiver/api/web_source.py`. They take a `WebFetchRequest` and return a `WebPageResult` or a `WebFetchRefusal`, with `backend=webarchiver`. `web_render`, `web_screenshot`, `web_links` and `web_crawl` are the connector's own per-operation tools. Proving test: `tests/test_conformance.py::test_webarchiver_conforms_to_the_web_source_contract`. | The served tool schemas equal the contract's, and an unknown argument is refused. | requirement | CONN-WEBARCHIVER-WEB-SOURCE | VERIFIED | 31e5998eddc5 tests/test_conformance.py:20 |
| CONN-WEBARCHIVER-WEB-SOURCE-R001.2 Behavior Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source**Behavior.** The connector must obey `robots.txt` for its user agent and cache each file for 3600 s. It must keep a per-domain rate limit of 1 request per second by default. It must stop reading at `max_bytes` and refuse an unlisted content type. `web_extract` must reduce the page to its main content as markdown. It must return the title, author, published date, canonical URL and outbound links. `content_digest` is the SHA-256 of the extracted text. Proving test: `tests/test_extract.py::test_main_content_and_metadata`. | Fixture pages yield the expected markdown and metadata, and a disallowed path is refused. | requirement | CONN-WEBARCHIVER-WEB-SOURCE | VERIFIED | 8c38c25066e3 tests/test_extract.py:103 tests/test_extract.py:117 tests/test_extract.py:133 tests/test_extract.py:36 tests/test_extract.py:64 |
| CONN-WEBARCHIVER-WEB-SOURCE-R001.3 Wiring Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source**Wiring.** The browser capture in `Webarchiver.launch_browser` (`webarchiver/webarchiver.py:63`) serves `web_render` and `web_screenshot`. The server uses the SDK server factory and `register_tool_surface`. The production caller is `WebFetchSourceAdapter.extract` of `SDK-SOURCE-INGEST-R005.40.3`. Proving test: `tests/test_served.py::test_tools_call_reaches_extractor`. | A served `tools/call` reaches the extractor and the capture path. | requirement | CONN-WEBARCHIVER-WEB-SOURCE | VERIFIED | 8c38c25066e3 tests/test_served.py:34 tests/test_served.py:52 tests/test_served.py:69 |
| CONN-WEBARCHIVER-WEB-SOURCE-R001.4 Refusal Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source**Refusal.** Each refusal must use a closed reason of the contract: `robots_disallowed`, `rate_limited`, `size_exceeded`, `unsupported_type`, `egress_denied`, `policy_refused`, `extract_empty` or `upstream_failed`. A private address must refuse through the SDK egress policy. The connector must never return an empty page as a success. Proving test: `tests/test_extract.py::test_refusals`. | Each case returns its reason and no page body. | requirement | CONN-WEBARCHIVER-WEB-SOURCE | VERIFIED | 8c38c25066e3 tests/test_extract.py:45 |
| CONN-WEBARCHIVER-WEB-SOURCE-R001.5 Evidence Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source**Evidence.** Each result must name `backend=webarchiver`, the final URL, the HTTP status, the byte count, the content digest, the retrieval time and the `robots.txt` decision. Proving test: `tests/test_extract.py::test_result_provenance`. | Each named field is present and matches the fixture response. | requirement | CONN-WEBARCHIVER-WEB-SOURCE | VERIFIED | 8c38c25066e3 tests/test_extract.py:143 |
| CONN-WEBARCHIVER-WEB-SOURCE-R001.6 Hot-path proof Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source**Hot-path proof.** The test serves the connector and runs the SDK conformance kit of `SDK-SOURCE-INGEST-R005.40.2` against it. It then runs `connector-sync --once` for one page into a running engine. Proving test: `tests/test_conformance_served.py::test_conformance_kit_and_sync`. | The kit reports no broken obligation, and the engine holds one Document with `backend=webarchiver`. | requirement | CONN-WEBARCHIVER-WEB-SOURCE | SPECIFIED | No recorded commit/test evidence |
| CONN-WEBARCHIVER-WEB-SOURCE-R001.7 Bounded site crawl Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source**Bounded site crawl.** `web_crawl(seed_url, same_site, max_pages, max_depth, include, exclude)` in `webarchiver/api/crawl.py` must walk links breadth-first from one seed and return the page URLs it reached, in discovery order. It must fetch every page through the same policy as `web_fetch`: robots.txt, the per-domain rate limit, size and content-type caps and egress pinning. `same_site` defaults to true. `max_pages` ranges from 1 to 1000 and `max_depth` from 0 to 10. `include` and `exclude` are regular expressions. A robots-disallowed or failed page is reported as skipped with its closed reason. It returns URLs only; fetching content is the caller's next step. Proving test: `tests/test_crawl.py::test_bounded_crawl`. | A fixture site yields the expected URLs in order, the disallowed path is reported and never fetched, and out-of-range bounds refuse with `policy_refused`. | requirement | CONN-WEBARCHIVER-WEB-SOURCE | VERIFIED | 4fde2c6fe305 99c1281925fe tests/test_crawl.py:17 tests/test_crawl.py:35 tests/test_crawl.py:49 |
| CONN-WEBARCHIVER-WEB-SOURCE-R001.8 Ingest mappings Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source**Ingest mappings.** `webarchiver/connector_manifest.yml` must declare `schema_mappings` for the `Document` and `Chunk` record types, mapped to the core `Document` and `Chunk` classes, so `SourceIngest` resolves `manifest:<connector>#schema_mappings/Document` and `.../Chunk` in the pack the fleet imports (`SDK-SOURCE-INGEST-R004.25`). Proving test: `tests/test_startup.py::test_manifest_maps_document_and_chunk_records`. | The served manifest maps both record types with their text, title, source and backend fields. | requirement | CONN-WEBARCHIVER-WEB-SOURCE | VERIFIED | 482ceea819b0 4d127d07b8ce tests/test_startup.py:51 |
| CONN-WEBARCHIVER-WEB-SOURCE-R001.10 Conformance Definition — CONN-WEBARCHIVER-WEB-SOURCE / CONN-WEBARCHIVER-WEB-SOURCE — Web page fetch, extraction, render and screenshot connectorCanonical requirement and verification source**Conformance.** The connector passes the SDK conformance kit (`SDK-SOURCE-INGEST-R005.40.2`) against itself, served in process. The kit checks robots.txt, the per-domain rate limit, `max_bytes`, the content type, the digest and the tool schemas. The archivebox-api connector passes the same kit, so both produce the same tools and record shape (`CONN-ARCHIVEBOX-API-WEB-SOURCE-R001.8`). Proving test: `tests/test_conformance.py::test_webarchiver_conforms_to_the_web_source_contract`. | The kit reports no broken obligation. | requirement | CONN-WEBARCHIVER-WEB-SOURCE | VERIFIED | 31e5998eddc5 tests/test_conformance.py:21 |