tx spec

Spec-to-test traceability with multi-language discovery and FCI phase scoring

Purpose

tx spec bridges docs and tests:

Sync invariants from tx docs (tx invariant sync behavior)
Discover test mappings across languages from source annotations and .tx/spec-tests.yml
Record test outcomes
Compute Feature Completion Index (FCI) and phase (BUILD -> HARDEN -> COMPLETE)

This is primitive infrastructure, not a test runner. You keep your existing framework and CI.

Prerequisites

To get useful output from tx spec, you need three things in place already:

tx doc: structured PRDs and design docs on disk so invariants have a canonical source
tx invariant: synced active invariants to score against
Existing tests with either source annotations or .tx/spec-tests.yml: discovery maps tests from tags, comments, or explicit manifest entries

Common follow-on primitives:

tx verify: make spec or FCI checks part of task completion
tx gate: require a human checkpoint before release movement
tx trace: inspect failing runs, transcripts, and stderr when outcomes regress

At A Glance

Spec Pipeline

Rendering diagram…

Specs become invariants, discovery builds mappings, and recorded outcomes drive FCI and phase.

FCI Gate

Rendering diagram…

BUILD is automatic while any invariant is failing. HARDEN starts at 100% FCI. COMPLETE requires human sign-off.

Use this as a release gate: automation can move from BUILD to HARDEN, but only a human can mark COMPLETE.

Canonical Test ID

Every mapped test uses:

{relative_file}::{test_name}

Example:

test/integration/core.test.ts::ready detection returns unblocked tasks

Multi-Language Discovery

Discovery is regex-based on raw text files. No AST parsing.

Supported annotation conventions:

Tag in test names: [INV-EARS-FL-001]
Underscore tags in function names: _INV_EARS_FL_001
Comment annotations: @spec INV-EARS-FL-001
Manifest mappings: .tx/spec-tests.yml

Default scan patterns include common JS/TS, Python, Go, Rust, Java, Ruby, and C/C++ conventions. Override via .tx/config.toml [spec].test_patterns.

CLI

# Discovery + mapping
 tx spec discover [--doc <name>] [--patterns <glob1,glob2,...>]
 tx spec link <inv-id> <file> [name] [--framework <name>]
 tx spec unlink <inv-id> <test-id>
 tx spec tests <inv-id>
 tx spec gaps [--doc <name>] [--sub <name>]
 tx spec matrix [--doc <name>] [--sub <name>]

# Scoring + lifecycle
 tx spec fci [--doc <name>] [--sub <name>]
 tx spec status [--doc <name>] [--sub <name>]
 tx spec complete [--doc <name> | --sub <name>] --by <human> [--notes <text>]

# Run ingestion
 tx spec run <test-id> --passed|--failed [--duration <ms>] [--details <text>]
 tx spec batch [--from generic|vitest|pytest|go|junit]   # reads stdin

Notes:

tx spec run requires exactly one of --passed or --failed.
tx spec complete requires --by and only succeeds from HARDEN.

Interfaces

tx spec discover --doc PRD-033-spec-test-traceability
tx spec fci --doc PRD-033-spec-test-traceability
tx spec status --doc PRD-033-spec-test-traceability
vitest run --reporter=json | tx spec batch --from vitest
tx spec complete --doc PRD-033-spec-test-traceability --by james

import { TxClient } from '@jamesaphoenix/tx-agent-sdk'

const tx = new TxClient({ dbPath: '.tx/tasks.db' })
const junitXml = '<testsuites>...</testsuites>'

const discover = await tx.spec.discover({
  doc: 'PRD-033-spec-test-traceability',
})

const status = await tx.spec.status({
  doc: 'PRD-033-spec-test-traceability',
})

const matrix = await tx.spec.matrix({
  doc: 'PRD-033-spec-test-traceability',
})

await tx.spec.batch({
  from: 'junit',
  raw: junitXml,
})

if (status.phase === 'HARDEN') {
  await tx.spec.complete({
    doc: 'PRD-033-spec-test-traceability',
    signedOffBy: 'james',
  })
}

console.log({
  discovered: discover.discoveredLinks,
  phase: status.phase,
  mappedInvariants: matrix.length,
})

tx_spec_discover
tx_spec_link
tx_spec_unlink
tx_spec_tests
tx_spec_invariants_for_test
tx_spec_gaps
tx_spec_fci
tx_spec_status
tx_spec_matrix
tx_spec_record_run
tx_spec_batch_run
tx_spec_complete

POST /api/spec/discover
GET  /api/spec/tests/:invariantId
GET  /api/spec/invariants?testId=<canonical-id>
GET  /api/spec/gaps
GET  /api/spec/fci
GET  /api/spec/status
GET  /api/spec/matrix
POST /api/spec/link
POST /api/spec/unlink
POST /api/spec/run
POST /api/spec/batch
POST /api/spec/complete

FCI and Phases

FCI = passing_invariants / total_active_invariants * 100

BUILD: FCI < 100
HARDEN: FCI = 100 (auto)
COMPLETE: human sign-off (tx spec complete)

Use this as a readiness primitive for moving from implementation to hardening and release review.

tx spec

On this page