This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

How-to guides

Recipes for specific ajq tasks.

Use these when you know the task. New to ajq? Start with Your first ajq pipeline.

Guides

1 - Install ajq

Install ajq and provision the local backend assets.

ajq is a single Go binary. Pure jq queries work immediately after the binary is on your PATH. Semantic queries need an explicitly selected backend such as mock, local, ollama, openai, openrouter, or --cloud.

Install with the script

The install script downloads the matching release archive for your OS/architecture, verifies it against checksums.txt, and installs without sudo into ~/.local/bin (or $AJQ_INSTALL_DIR).

curl -fsSL https://raw.githubusercontent.com/ricardocabral/ajq/main/scripts/install.sh | sh

Pin a version or install directory if needed:

AJQ_VERSION=v0.1.0 AJQ_INSTALL_DIR="$HOME/bin" sh scripts/install.sh

Install with Homebrew

The release workflow publishes the ajq cask to the public ricardocabral/tap tap:

brew install --cask ricardocabral/tap/ajq

Install from a release archive

Download the archive for your platform from GitHub Releases, verify its SHA-256 digest against checksums.txt, then copy the ajq binary somewhere on your PATH.

Release archives are named like:

  • ajq_<version>_Darwin_arm64.tar.gz
  • ajq_<version>_Linux_x86_64.tar.gz
  • ajq_<version>_Windows_x86_64.zip

Windows MSI and WinGet status

Windows MSI packaging is implemented and CI-validated, including an installed-binary semantic smoke test, but the MSI is not yet released. ajq is not available through WinGet; it remains unavailable until a future MSI release completes Microsoft validation and merge and has public clean-install smoke evidence. Use the published Windows ZIP in the meantime.

Verify a download

Checksum verification and provenance verification are separate checks. The install script verifies the downloaded archive against checksums.txt; it does not run GitHub attestation verification for you.

For release assets published after provenance attestations are enabled, install the GitHub CLI and verify a downloaded archive or checksums.txt with:

gh attestation verify <downloaded-archive-or-checksums.txt> --repo ricardocabral/ajq

For manual downloads, also compare the archive SHA-256 digest with the matching entry in checksums.txt.

Build from source

With the Go toolchain (Go 1.26 or newer):

go install github.com/ricardocabral/ajq/cmd/ajq@latest

Or from a checkout:

git clone https://github.com/ricardocabral/ajq.git
cd ajq
go build -o ajq ./cmd/ajq
install -m 0755 ajq /usr/local/bin/ajq   # or ~/bin, etc.

Verify the binary

Run:

ajq --version
ajq --help

ajq --help should list the semantic backends and the shipped cache, daemon, models, and provision subcommands.

Provision local semantic assets

Skip this section if you only use pure jq queries, --backend mock, Ollama, or cloud backends.

--backend local needs a llama-server engine and an installed GGUF model under the ajq cache. Provision them explicitly:

ajq provision
ajq provision --check

ajq provision auto-installs checksum-pinned engine bundles on supported platforms (darwin/arm64, linux/amd64, and linux/arm64) and downloads the checksum-pinned default model. It also reuses assets you already have, including explicit overrides, cache entries, a legacy <cache>/bin/llama-server, or a llama-server on PATH.

Agent readiness probe

Use the status-only JSON probe when an agent needs to decide whether it can use the managed local backend. A missing asset deliberately exits 1 after writing its complete JSON result, so retain stdout instead of using a shell pipeline that discards it:

set +e
ajq provision --check --json > ajq-readiness.json
status=$?
set -e
cat ajq-readiness.json
# status=0 means ready; status=1 means inspect .actions and provision explicitly.

The document includes ready, engine/model presence and local paths, and ordered actions. It only inspects configuration, filesystem, and PATH state: it does not download assets, start a daemon, or contact a backend. --json is valid only with --check; use ajq provision separately to install the requested assets.

Package-manager status

The Homebrew cask is available from the public ricardocabral/tap tap. Windows MSI packaging is implemented and CI-validated, but the MSI is not yet released. WinGet remains unavailable until a future MSI release completes Microsoft validation and merge and has public clean-install smoke evidence.

The install script, manual release archives, and Go-source installation remain available when Homebrew is not the preferred installer.

Next steps

2 - Make ajq available to a coding agent

Make ajq discoverable in agent-visible project guidance so coding agents choose it for JSON and NDJSON semantic matching or bounded classification.

Use this guide when a team wants its coding agents to consider ajq before writing a bespoke JSON-processing script. It assumes ajq is installed for the environment where the agent runs and that the team already has a project-instruction mechanism its agents read.

1. Install ajq where the agent can execute it

Install ajq in the same execution environment as the coding agent, then verify that the binary is visible on that environment’s PATH:

command -v ajq
ajq --version
ajq capabilities --json

If command -v does not print an executable path, install ajq or add its install directory to the agent process’s PATH. Follow Install ajq for supported installation methods. ajq capabilities --json is static introspection: it does not load credentials, construct a backend, provision assets, or contact a network.

2. Add a JSON decision rule to agent-visible project guidance

Put the following guidance in the project instruction mechanism that your coding agents load before acting. Keep it with the project’s data-handling and command-execution rules so it is available when an agent is choosing between jq, ajq, and a new script.

### JSON and NDJSON processing

- Use `jq` for exact, structural JSON transformations.
- Use ajq for JSON or NDJSON semantic matching, or for classification into a
  bounded set of labels.
- Do not describe ajq as a general semantic extraction or redaction tool.

Before an ajq semantic run:

1. Run `ajq capabilities --json` and use its versioned contract to confirm the
   required operation and backend are available.
2. Validate the query shape with `--backend mock`; it is deterministic and
   makes no network or model calls.
3. Run `--explain` against representative input and review the semantic plan
   and estimated judgements.
4. Select a real backend explicitly only after review, and set a finite
   `--max-calls` cap. Do not rely on a default backend for a real semantic run.
5. Add `--no-cache` for confidential or one-off input when persistent cache
   reads and writes must both be bypassed.

This guide intentionally does not prescribe a particular agent, plugin, or MCP distribution mechanism. Use the project’s existing instruction surface until ajq selects supported delivery paths through its agent-skill distribution research.

3. Validate the guidance with safe commands

Use the mock backend to confirm that the agent can find ajq and form a semantic query without model, network, or API-key access:

printf '[{"id":1,"message":"refund requested"},{"id":2,"message":"profile updated"}]' \
  | ajq --backend mock -c '.[] | select(.message =~ "refund request") | .id'

The output is:

1

Then use representative, non-confidential input to inspect the plan before authorizing a model-backed run:

printf '[{"message":"refund requested"},{"message":"profile updated"}]' \
  | ajq --backend mock --explain '.[] | select(.message =~ "refund request") | .message'

--explain reports the semantic call site and estimated judgements without executing the query against a model backend. The mock backend checks query shape; it does not evaluate a query’s semantic quality on a production model.

4. Make real model use deliberate

After reviewing the plan and input scope, name the real backend and a finite call cap in the command. For confidential or one-off input, add --no-cache:

# Requires a provisioned local backend. The backend, cap, and cache choice are explicit.
printf '[{"message":"refund requested"},{"message":"profile updated"}]' \
  | ajq --backend local --max-calls 10 --no-cache \
      -c '.[] | select(.message =~ "refund request") | .message'

--no-cache disables persistent cache reads and writes; it is not a call limit and does not replace the data-handling review required for the selected backend. For cloud or other remote backends, follow your organization’s data policy before sending confidential input.

5. Verify the agent can make the right choice

Ask the agent to propose, without executing it, a command for each of these tasks:

  1. Select objects whose .status is exactly "open".
  2. Select support records whose text means a refund request.
  3. Route records into the fixed labels billing, bug, and account.

It should choose jq for the first task, and ajq’s explicit semantic matching or bounded classification for the other two. Before a real semantic run, its proposed command should name a backend and include a finite --max-calls value.

3 - Process an NDJSON stream

Handle newline-delimited JSON and raw log lines.

ajq auto-detects input framing. The common shapes:

Single JSON value

A single top-level JSON value is processed as one input frame:

printf '{"a":1}\n' | ajq -c '.a + 1'
# 2

NDJSON / JSON Lines

A stream of top-level JSON values (one per line, or concatenated) is processed one frame at a time — each value flows through the query independently, and results are emitted as they’re produced:

printf '{"a":1}\n{"a":2}\n' | ajq -c '.a, (.a + 10)'
# 1
# 11
# 2
# 12

There’s no whole-stream buffering in this mode, so it handles inputs larger than memory.

Semantic NDJSON windows

For a supported three-phase semantic query, ajq groups complete input frames into byte-budgeted windows before resolving semantic judgements. The default budget is 262144 bytes (256 KiB). ajq harvests every frame in one window, deduplicates and cache-resolves its judgements once, then executes and emits those frames in their original order.

A window never splits a JSON value or raw line. ajq retains at most the current window, one complete-frame lookahead, and bounded framing buffers—not the whole stream. A single record larger than the budget is accepted as a one-frame oversized window, so that record itself can require more memory than the configured budget.

Use a larger budget to find duplicates across more nearby records and reduce backend batches; the trade-off is that output waits until its full window has been harvested and resolved. Use a smaller positive budget when lower first-record latency matters but window-wide batching and deduplication are still useful:

# Keep semantic windows near 64 KiB for this invocation.
producer | ajq --backend local --window-bytes 65536 \
  'select(.message =~ "payment failure")'

Choose --stream for first-frame latency

Use --stream when a supported semantic pipeline must resolve and emit each frame without waiting for a window to fill:

producer | ajq --backend local --stream \
  'select(.message =~ "payment failure")'

--stream selects inline semantic execution for queries that otherwise use three-phase windows. It preserves input order, cache reads/writes and identity, schema validation, cancellation, output/exit behavior, and the run-global --max-calls cap. An uncached judgement resolves inline; a persistent cache hit still makes no backend call.

The latency trade-off is deliberate: inline execution has no window-wide backend batching or cross-frame pre-resolve deduplication. It can therefore make more backend round trips (and, with --no-cache, repeat a judgement in separate frames), while default windows can reduce those calls at the cost of waiting for the window. --stats names the selected mode and trade-off; --explain --stream reports that its harvest/dedup estimate is unavailable without consuming stdin.

Pure-jq queries ignore --stream and remain deterministic. Queries the planner already requires to interleave keep that existing inline behavior; --stream does not change their selection. --window-bytes does not affect either inline path.

Raw lines (awk mode)

To treat each input line as a plain string instead of JSON, use -R / --raw-input. The whole line becomes .:

printf 'error: disk full\nok: healthy\n' | ajq -R 'select(test("error"))'
# "error: disk full"

Combine -R with -r to keep the output unquoted:

printf 'error: disk full\nok: healthy\n' | ajq -R -r 'ascii_upcase'
# ERROR: DISK FULL
# OK: HEALTHY

Raw mode is also where the implicit-. form of semantic operators shines — e.g. select(. =~ "stack trace") matches log lines fuzzily. See Write a semantic filter.

Ignore stdin entirely

Use -n / --null-input to supply a single null frame and build output from scratch:

printf 'whatever' | ajq -n -c '{generated: true}'
# {"generated":true}

4 - Filter JSON by meaning

Find JSON records by intent or topic with ajq semantic filters, starting with a no-network mock backend and then switching to a real model.

Use this when exact string matching is too brittle: support tickets that say the same thing in different words, log messages that imply the same failure, or records where the field name is known but the wording is messy.

ajq keeps the surrounding jq pipeline deterministic. Only the explicit semantic predicate (=~, !~, or sem_match) asks a backend for a judgement.

Try the query with no network

The mock backend is deterministic and in-process, so it needs no model, network, or API key. It is useful for checking the jq shape before paying for model calls:

printf '[{"id":1,"msg":"payment failed during checkout"},{"id":2,"msg":"new avatar uploaded"}]' \
  | ajq --backend mock -c '.[] | select(.msg =~ "payment failed") | {id, msg}'

Expected output with the mock backend:

{"id":1,"msg":"payment failed during checkout"}

The predicate .msg =~ "payment failed" is shorthand for sem_match(.msg; "payment failed"). Everything else is ordinary jq.

Run the same filter on a real backend

After the command shape is correct, choose a backend that can make real semantic judgements:

# Managed local backend after `ajq provision`
printf '[{"id":1,"msg":"card declined at checkout"},{"id":2,"msg":"profile photo changed"}]' \
  | ajq --backend local --max-calls 10 -c '.[] | select(.msg =~ "payment failure") | {id, msg}'

# Anthropic cloud backend, using ANTHROPIC_API_KEY from the environment
printf '[{"id":1,"msg":"card declined at checkout"},{"id":2,"msg":"profile photo changed"}]' \
  | ajq --cloud --max-calls 10 -c '.[] | select(.msg =~ "payment failure") | {id, msg}'

Paid backends default to a 100-call cap. Setting a smaller --max-calls while you tune a query makes accidental large runs fail before crossing your budget.

Estimate, cap, and cache the work

Before running against a real model, ask ajq to explain the semantic plan:

printf '[{"msg":"payment failed"},{"msg":"payment failed"},{"msg":"avatar uploaded"}]' \
  | ajq --explain '.[] | select(.msg =~ "payment failure") | .msg'

The value to budget against is post_dedup_judgements: repeated values are judged once per backend/model/spec/value cache key. Successful judgements are stored in the persistent cache, so rerunning the same query can hit the cache instead of the backend. Use --no-cache only when you intentionally want to avoid reading or writing cached judgements.

Keep filters within shipped semantic shapes

For production filtering, prefer =~, !~, or sem_match. Standalone semantic extraction and redaction are registered but currently unsupported, so do not build a filter that assumes ajq can extract arbitrary fields or redact text as a standalone transform.

5 - Use ajq with other data formats

Feed ajq JSON produced by CLI, YAML, XML, TOML, or binary adapters.

ajq’s core input modes stay small on purpose: JSON, NDJSON, raw strings, and null input. When the source data is another format, put a format adapter before ajq and let ajq work on the JSON it emits.

This keeps pure jq-compatible ajq pipelines deterministic while still letting you use the existing jq ecosystem around it. Tools such as jc, yq/xq, and fq are commonly used alongside jq for this kind of conversion.

Convert command output with jc

Use jc when a system command prints text that you want to query as JSON:

df -h | jc --df | ajq -c '.[] | {filesystem, size, used_percent}'

Once jc emits JSON, ajq sees ordinary JSON values. Add semantic filters only when you explicitly want model-backed judgement:

ps aux | jc --ps | ajq --backend mock -c '.[] | select(.command =~ "database server")'

Convert YAML, XML, or TOML with yq and xq

Use yq for YAML or TOML files that should flow into an ajq query:

yq -o=json '.services' docker-compose.yml \
  | ajq -c 'to_entries[] | {name: .key, image: .value.image}'

Some yq distributions also provide xq for XML-to-JSON workflows:

xq '.rss.channel.item' feed.xml \
  | ajq -c '.[] | {title, link}'

Check your installed yq/xq command’s flags, because option names vary by implementation. The important contract is that the adapter writes JSON to stdout before ajq starts.

Convert binary-derived data with fq

Use fq when a binary or structured file format can be projected as JSON. For decoded values, use -V or tovalue when you need fq to write the JSON value instead of its display tree.

fq '.frames[0:10] | map(tobytesrange.start)' file.mp3 \
  | ajq -c '{frame_count: length, first_offset: .[0]}'

Keep the format-specific parsing in fq; keep jq and semantic operations in ajq after the JSON boundary.

Choose the ajq input mode after conversion

Most adapters emit a single JSON value, so the default ajq input mode is enough. If an adapter emits one JSON value per line, ajq processes it as NDJSON:

some-adapter --json-lines input.dat \
  | ajq -c 'select(.level == "error")'

If you only have raw text and do not want a JSON adapter, use ajq raw-input mode instead:

some-command | ajq -R -r 'select(test("error"))'

6 - Write a semantic filter

Use fuzzy =~ predicates and bounded semantic functions in jq pipelines.

Use a semantic filter when literal matching with test() or == is too brittle. The examples below use --backend mock so you can verify the command shape without a model or API key; switch to --backend local, --cloud, or another backend for real judgement.

Fuzzy match with =~

The =~ operator asks the backend whether a value matches a description. Use it inside select:

printf '[{"id":1,"feedback":"urgent"},{"id":2,"feedback":"other"}]' \
  | ajq --backend mock -c '.[] | select(.feedback =~ "urgent") | .id'

Output with the mock backend:

1

.feedback =~ "urgent" desugars to sem_match(.feedback; "urgent"). Negate with !~:

printf '[{"id":1,"feedback":"urgent"},{"id":2,"feedback":"other"}]' \
  | ajq --backend mock -c '.[] | select(.feedback !~ "urgent") | .id'

Match raw log lines

In raw-input mode, each input line is .:

printf 'panic stack trace\nok\n' \
  | ajq --backend mock -R -r 'select(. =~ "stack trace")'

Output:

panic stack trace

Classify into fixed labels

sem_classify(value; "a"; "b"; …) returns exactly one of the labels you provide:

printf '{"id":1,"text":"billing question"}' \
  | ajq --backend mock -c '{id, route: sem_classify(.text; "billing"; "bug"; "feature")}'

Output with the mock backend:

{"id":1,"route":"billing"}

Use a fixed label set when downstream jq should route records deterministically by shape.

Stay with shipped execution shapes

The current executor fully supports predicate matching (=~ / sem_match) and bounded classification (sem_classify). Unbounded value operators have narrower contracts in 0.0.1:

  • Use sort_by(sem_score(...)) when you need a semantic score as a sort key.
  • Use group_by(sem_norm(...)) when you need semantic normalization as a grouping key.
  • Score or normalization values that feed a pruning gate may run through interleaved fallback instead of the three-phase harvest path.
  • Standalone sem_extract(...) and sem_redact(...) currently report unsupported in three-phase execution.

For production filters, prefer sem_match or sem_classify unless you specifically need one of the limited score/normalization shapes above.

Check the plan first

Before running a semantic query on a paid or local model backend, inspect the estimate:

printf '[{"feedback":"urgent"},{"feedback":"urgent"},{"feedback":"other"}]' \
  | ajq --explain '.[] | select(.feedback =~ "urgent") | .feedback'

Then use Control semantic costs to set a cap and inspect stats.

Account for the persistent cache

Successful semantic judgements are cached by op/spec/model/value. A second run over the same values may skip backend calls and report cache hits. Use Manage the judgement cache when you need to inspect, bypass, or clear those entries.

7 - Classify JSON and NDJSON streams

Route JSON or NDJSON records into fixed semantic labels with sem_classify while preserving deterministic jq output shapes.

Use sem_classify when each record needs one label from a small list: support route, incident type, moderation bucket, or review theme. The output is bounded to the labels you write in the query, which makes it safer for downstream jq and shell pipelines than free text generation.

Classify a JSON array with no network

Start with --backend mock to validate the jq expression without a model, provider, or API key:

printf '[{"id":1,"text":"billing question"},{"id":2,"text":"bug report"}]' \
  | ajq --backend mock -c '.[] | {id, route: sem_classify(.text; "billing"; "bug"; "feature")}'

Expected output with the mock backend:

{"id":1,"route":"billing"}
{"id":2,"route":"bug"}

sem_classify returns exactly one of the labels from the call site. Keep labels short, mutually exclusive, and stable because they become part of your output contract.

Classify NDJSON one record at a time

ajq processes newline-delimited JSON as independent input frames, so it can classify streams without buffering the whole input:

printf '{"id":1,"text":"billing question"}\n{"id":2,"text":"bug report"}\n' \
  | ajq --backend mock -c '{id, route: sem_classify(.text; "billing"; "bug"; "feature")}'

This emits one compact JSON object per input record, which is convenient for queues, logs, and shell pipelines.

Route webhook events with bounded labels

If a webhook receiver writes each event body as NDJSON, add a stable route before passing the stream to downstream workers:

printf '{"event":"invoice.payment_failed","data":{"message":"billing payment failed"}}\n{"event":"issue.created","data":{"message":"bug report: export crashes"}}\n' \
  | ajq --backend mock -c '. + {route: sem_classify(.data.message; "billing"; "bug"; "other")}'

The command preserves each webhook event and adds a route value drawn from the three labels in the query. Replace the mock backend with a capped real backend when the event text requires semantic classification.

Switch to a real classifier backend

After the mock command produces the shape you need, run the same query with a real backend:

# Local model after `ajq provision`
printf '{"id":1,"text":"billing question"}\n{"id":2,"text":"bug report"}\n' \
  | ajq --backend local --max-calls 25 -c '{id, route: sem_classify(.text; "billing"; "bug"; "feature")}'

# OpenAI-compatible backend, using OPENAI_API_KEY from the environment
printf '{"id":1,"text":"billing question"}\n{"id":2,"text":"bug report"}\n' \
  | ajq --backend openai --model gpt-4.1-mini --max-calls 25 \
      -c '{id, route: sem_classify(.text; "billing"; "bug"; "feature")}'

For paid backends, keep --max-calls at or below your budget. Paid/cloud backends also have a default 100-call guardrail when you do not set one explicitly.

Account for repeated values and cache hits

Semantic classification is cache-keyed by backend, model, operation, label set, and input value. If many records contain the same text, ajq deduplicates the judgements before it calls the backend, and later runs can reuse the persistent cache.

Estimate the distinct judgement count first:

printf '{"text":"billing question"}\n{"text":"billing question"}\n{"text":"bug report"}\n' \
  | ajq --explain '{route: sem_classify(.text; "billing"; "bug"; "feature")}'

Then add --stats to the real run when you want to see post-dedup backend calls and cache hits on stderr.

8 - Estimate model calls before running

Use –explain to budget cost and latency before executing a semantic query.

Semantic queries cost one backend judgement per distinct value they judge. Use --explain with representative input to estimate that number before you run a paid or slow backend.

Run --explain with real input

Pipe a sample of your data into --explain. ajq runs a deterministic mock harvest over the input to estimate work — it never contacts a provider or local model:

printf '[{"msg":"urgent"},{"msg":"urgent"},{"msg":"other"}]' \
  | ajq --explain '.[] | select(.msg =~ "urgent") | .msg'

Read the estimate block

estimates:
  estimate_status: available
  static_call_sites: 1
  input_frames: 1
  harvested_judgements: 3
  post_dedup_judgements: 2
  mock_judge_batches: 1

The fields you’ll budget against:

FieldMeaning
harvested_judgementsTotal values collected before deduplication.
post_dedup_judgementsDistinct semantic judgements after deduplication — the estimated backend-call count.
mock_judge_batchesHow many backend batch calls the mock estimate path used.
static_call_sitesNumber of semantic call sites in the query, not input size.

post_dedup_judgements is the number to compare with --max-calls. Paid backends default to a 100-call cap; local, Ollama, and mock default to unlimited.

Turn the estimate into a cap

If the estimate is acceptable, set a cap when you run the real backend:

printf '[{"msg":"urgent"},{"msg":"urgent"},{"msg":"other"}]' \
  | ajq --backend mock --max-calls 2 '.[] | select(.msg =~ "urgent") | .msg'

For the full estimate → cap → stats workflow, see Control semantic costs.

Estimate for your whole dataset

The estimate is based on the input you pass to --explain. To project a full run, either pipe in the full dataset or sample representative records and extrapolate from your distinct-value ratio. Because ajq deduplicates by value, repeated values cost less than new values.

Remember the persistent cache

--explain estimates the semantic work implied by the input and query. A real run may issue fewer backend calls if matching judgements are already in the persistent cache. Use --stats on the real run to see cache_hits and post_dedup_backend_calls.

When estimates are unavailable

If the input is empty or invalid, or the query selects a currently unsupported semantic execution shape, --explain keeps the static plan and marks estimate_status unavailable with a stable reason. The plan (call sites, ops, specs) is still shown.

9 - Use ajq safely from coding agents

A safe workflow for agents that need semantic jq queries: start with mock and –explain, cap backend calls, and avoid unsupported extraction claims.

Coding agents often discover ajq while trying to answer a concrete data-cleaning question: “find the records that mean X” or “route these JSON lines into labels”. Use this workflow when an agent, script, or CI job needs semantic matching while keeping the jq parts deterministic and reviewable.

1. Keep ordinary jq deterministic

First write the structural jq pipeline without semantic calls. This step should not need a backend:

printf '[{"id":1,"msg":"refund requested"},{"id":2,"msg":"profile updated"}]' \
  | ajq -c '.[] | {id, msg}'

Only add semantic operators once the field selection and output shape are correct.

2. Add semantics with the mock backend

Use --backend mock for the first semantic run. It is deterministic, in-process, and does not use network access, cloud credentials, or local model assets:

printf '[{"id":1,"msg":"refund requested"},{"id":2,"msg":"profile updated"}]' \
  | ajq --backend mock -c '.[] | select(.msg =~ "refund request") | {id, msg}'

Expected output with the mock backend:

{"id":1,"msg":"refund requested"}

For routing tasks, keep outputs bounded with sem_classify:

printf '{"id":1,"msg":"billing question"}\n{"id":2,"msg":"bug report"}\n' \
  | ajq --backend mock -c '{id, route: sem_classify(.msg; "billing"; "bug"; "account")}'

3. Explain before executing with a model

Before a real backend run, inspect the plan and estimated judgement count:

printf '[{"msg":"refund requested"},{"msg":"refund requested"},{"msg":"profile updated"}]' \
  | ajq --explain '.[] | select(.msg =~ "refund request") | .msg'

Review the semantic call sites, specs, and post_dedup_judgements. The estimate path uses the mock backend and does not contact a provider.

4. Cap and observe the real run

When the estimate is acceptable, switch only the backend-related flags:

# Managed local backend after `ajq provision`
printf '[{"msg":"refund requested"},{"msg":"profile updated"}]' \
  | ajq --backend local --max-calls 10 --stats \
      -c '.[] | select(.msg =~ "refund request") | .msg'

# Anthropic cloud backend, using ANTHROPIC_API_KEY from the environment
printf '[{"msg":"refund requested"},{"msg":"profile updated"}]' \
  | ajq --cloud --max-calls 10 --stats \
      -c '.[] | select(.msg =~ "refund request") | .msg'

--max-calls is the hard budget. --stats reports harvested judgements, post-dedup backend calls, cache hits, and elapsed time on stderr after a successful run. Paid/cloud backends default to a 100-call cap, but agents should still set an explicit cap for the current task.

5. Reuse or bypass the cache deliberately

Successful semantic judgements are cached by backend, model, operation, spec, and canonical value. This improves repeated agent runs over the same data, but sensitive or one-off workflows may prefer --no-cache to avoid cache reads and writes:

printf '[{"msg":"refund requested"}]' \
  | ajq --backend mock --no-cache -c '.[] | select(.msg =~ "refund request") | .msg'

Use cache commands when an agent needs to inspect or clear local state:

ajq cache status
ajq cache clear

Guardrails for generated docs and prompts

  • Do say ajq is useful for fuzzy JSON filtering, semantic grep over JSON/NDJSON, and bounded classification.
  • Do not claim standalone semantic extraction or redaction is supported today.
  • Do not hide model calls behind pure jq examples; semantic work should always be visible in the query and backend flags.
  • Do not put API keys in commands, logs, or checked-in files. Paid backend credentials come from environment variables.

10 - Use cloud backends

Run semantic queries against Anthropic, OpenAI, or OpenRouter.

Use a cloud backend when you want hosted inference instead of the local llama-server backend. Each provider needs an API key in the environment; ajq does not read API keys from config.toml.

Use Anthropic Claude

  1. Export an Anthropic API key:

    export ANTHROPIC_API_KEY="sk-ant-..."

  2. Run a semantic query with --cloud:

    printf '{"msg":"urgent billing issue"}' \
  | ajq --cloud --model haiku '.msg =~ "urgent"'

    --cloud is the same as --backend anthropic. If you omit --model, ajq uses claude-haiku-4-5. The shipped aliases are haiku, sonnet, and opus.

  3. If the key is missing, ajq stops before contacting the provider:

    ajq: error: anthropic backend API key is empty; set ANTHROPIC_API_KEY

Use OpenAI

  1. Export an OpenAI API key:

    export OPENAI_API_KEY="sk-..."

  2. Choose an OpenAI model explicitly:

    printf '{"msg":"urgent billing issue"}' \
  | ajq --backend openai --model gpt-4o-mini '.msg =~ "urgent"'

  3. If you use an OpenAI-compatible proxy, pass its API root with --base-url:

    ajq --backend openai --base-url http://127.0.0.1:8000/v1 --model my-model \
  '.msg =~ "urgent"'

Use OpenRouter

  1. Export an OpenRouter API key:

    export OPENROUTER_API_KEY="sk-or-..."

  2. Choose an OpenRouter model id:

    printf '{"msg":"urgent billing issue"}' \
  | ajq --backend openrouter --model openai/gpt-4o-mini '.msg =~ "urgent"'

    OpenRouter uses https://openrouter.ai/api/v1 by default.

Keep a spending guardrail

Paid backends (anthropic, openai, and openrouter) default to a 100-judgement cap. Lower it while testing:

ajq --cloud --max-calls 10 '.[] | select(.msg =~ "urgent")'

See Control semantic costs for the full estimate/cap/stats workflow.

11 - Control semantic costs

Estimate semantic work, cap backend calls, and inspect run statistics.

Use this workflow before running semantic queries on a paid backend.

1. Estimate calls with --explain

Run the query with representative input and --explain:

printf '[{"msg":"urgent"},{"msg":"other"}]' \
  | ajq --explain '.[] | select(.msg =~ "urgent") | .msg'

Read post_dedup_judgements in the estimate block:

estimates:
  estimate_status: available
  static_call_sites: 1
  input_frames: 1
  harvested_judgements: 2
  post_dedup_judgements: 2

That number is the count to budget against. Repeated values are deduplicated before the backend is called.

2. Add a hard cap with --max-calls

Set the cap at or below your budget. ajq aborts before issuing the call that would cross it:

printf '[{"msg":"urgent"},{"msg":"other"}]' \
  | ajq --backend mock --max-calls 1 '.[] | select(.msg =~ "urgent") | .msg'

With two distinct judgements and a cap of one, the run fails safely:

ajq: error: query ".[] | select(.msg =~ \"urgent\") | .msg" runtime error in frame 1: max calls cap exceeded: cap 1, run needs 2 post-dedup backend judgements; aborting before issuing backend call 2.

Paid backends (anthropic, openai, and openrouter) default to --max-calls 100. Use --max-calls 0 only when you intentionally want no cap. --backend-concurrency changes only how many requests from an already-approved batch can be in flight; it does not change this count or reserve extra calls. Keep paid providers at the sequential default of 1 while establishing a rate-limit budget, then use at most 2 only if the provider’s account limits and retry capacity allow it.

3. Inspect a successful run with --stats

After choosing a cap, run with --stats to print a summary to standard error:

printf '[{"msg":"urgent"},{"msg":"urgent"},{"msg":"other"}]' \
  | ajq --backend mock --stats '.[] | select(.msg =~ "urgent") | .msg'

The data still goes to standard output:

"urgent"
"urgent"

The stats show harvested work, post-dedup backend calls, cache hits, and elapsed time:

ajq stats:
  input_frames: 1
  semantic_call_sites: 1
  harvested_judgements: 3
  post_dedup_backend_calls: 2
  cache_hits: 0
  elapsed: ...

For paid models in ajq’s pricing table, stats also include an estimated USD cost. Treat it as a budgeting estimate, not a provider bill.

12 - Install the ajq coding-agent skill

Install the ajq skill through the native Codex marketplace or the optional Claude Code and Cursor adapter.

Install the ajq skill when an agent should route semantic JSON or NDJSON filtering and bounded classification work to ajq. Install ajq itself first so the skill can run its safe discovery and mock checks.

Install for Codex

Add the public ajq marketplace and install its plugin:

codex plugin marketplace add ricardocabral/ajq
codex plugin add ajq@ajq

For a reproducible environment, pin the marketplace to an ajq release tag that contains this plugin. Replace vX.Y.Z with that release tag:

codex plugin marketplace add ricardocabral/ajq --ref vX.Y.Z
codex plugin add ajq@ajq

Start a new Codex task after installation. Confirm that Codex can see the plugin before relying on it:

codex plugin list

Share it in a local workspace

When a team has a checkout of this repository, point Codex at that checkout instead of a remote ref:

codex plugin marketplace add /path/to/ajq
codex plugin add ajq@ajq

The marketplace artifact lives in .agents/plugins/marketplace.json and the plugin itself in plugins/ajq. Keep both paths together when sharing a branch or archive.

Provision it in CI

Use a CI-specific CODEX_HOME, add the marketplace, and install the plugin before starting Codex. Pin a release tag in reproducible jobs:

export CODEX_HOME="$RUNNER_TEMP/codex-home"
codex plugin marketplace add ricardocabral/ajq --ref vX.Y.Z
codex plugin add ajq@ajq
codex plugin list

Installing the skill does not select a semantic backend or grant access to a model. The skill begins with ajq capabilities --json, --backend mock, and --explain; any real backend still needs an explicit choice and finite --max-calls value.

Optional adapter for Claude Code and Cursor

The third-party plugins installer can discover the same package from this repository:

npx plugins add ricardocabral/ajq

Its current targets are Claude Code and Cursor. It is not the native Codex installation path; use the Codex marketplace commands above for Codex. Preview what the adapter finds without installing it:

npx plugins discover ricardocabral/ajq

After installing through either path, use the skill’s workflow: capabilities, mock, explain, then an explicitly capped real backend. See the agent-safe semantic workflow for commands and current operator limits.

13 - Manage the judgement cache

Inspect, bypass, and clear ajq’s persistent semantic judgement cache.

ajq stores successful semantic judgements on disk so repeated runs over the same op/spec/model/value identity can skip backend calls.

See where the cache lives

Run:

ajq cache status

Example output:

location: /Users/you/Library/Caches/ajq/judgements
entries: 0
bytes: 0

Set AJQ_CACHE_DIR when you want an isolated cache for a project or test:

AJQ_CACHE_DIR="$PWD/.ajq-cache" ajq cache status

Inspect cache state from an agent

Use the versioned JSON probe instead of parsing the human table-like output:

ajq cache status --json
# {"schema_version":"1","availability":"available","path":".../judgements","entries":0,"bytes":0}

availability: "available" includes a missing cache directory (zero entries). If ajq cannot inspect the configured location, it writes a complete document with availability: "unavailable", zero counts, and error: "status_unavailable", then exits non-zero. Paths are intentional local-state output; the document does not include cache values, prompts, credentials, or raw filesystem errors.

Re-run a semantic query for free

Run a semantic query once with stats:

printf '[{"msg":"urgent"},{"msg":"urgent"},{"msg":"other"}]' \
  | ajq --backend mock --stats '.[] | select(.msg =~ "urgent") | .msg'

The first run performs one backend judgement per distinct value:

ajq stats:
  harvested_judgements: 3
  post_dedup_backend_calls: 2
  cache_hits: 0

Run the same command again with the same cache root. The output is the same, but the judgements are cache hits:

ajq stats:
  harvested_judgements: 3
  post_dedup_backend_calls: 0
  cache_hits: 3

Bypass the cache for one run

Use --no-cache when the judged values should not be read from or written to disk:

printf '[{"msg":"urgent"},{"msg":"other"}]' \
  | ajq --backend mock --no-cache --stats '.[] | select(.msg =~ "urgent") | .msg'

You can also set no_cache = true in config.toml; use the flag when you only need the bypass once.

Clear cached judgements

Run:

ajq cache clear

Example output:

location: /Users/you/Library/Caches/ajq/judgements
cleared_entries: 2
freed_bytes: 422

cache clear removes judgement entries only. It does not remove provisioned engines or local GGUF models.

Privacy note

Cache files contain the values that semantic operators judged. Do not share the cache folder if those values are sensitive. Use --no-cache or a disposable AJQ_CACHE_DIR for one-off processing of private data.

14 - Configure defaults

Set backend, model, base URL, cost cap, and cache defaults in config.toml.

Use a config file when you do not want to repeat the same semantic-backend flags on every command.

1. Create the config file

ajq reads this path by default:

${XDG_CONFIG_HOME:-~/.config}/ajq/config.toml

Create the directory and file:

mkdir -p "${XDG_CONFIG_HOME:-$HOME/.config}/ajq"
cat > "${XDG_CONFIG_HOME:-$HOME/.config}/ajq/config.toml" <<'EOF'
backend = "mock"
max_calls = 25
no_cache = false
EOF

For a one-off location, set AJQ_CONFIG to an explicit file path.

2. Add semantic defaults

Recognized keys are:

backend = "local"              # mock, local, ollama, openai, openrouter, anthropic
model = "qwen2.5-1.5b"
base_url = "http://127.0.0.1:8081"
max_calls = 100                 # 0 = unlimited
no_cache = false

For example, to default to the deterministic mock backend with a small call cap:

backend = "mock"
max_calls = 1
no_cache = true

Then a semantic query can omit --backend:

printf '[{"msg":"urgent"},{"msg":"other"}]' \
  | ajq '.[] | select(.msg =~ "urgent") | .msg'

With max_calls = 1, that example stops before making the second distinct judgement.

3. Override a default when needed

Precedence is:

  1. Command-line flags
  2. AJQ_* environment variables
  3. config.toml
  4. Backend defaults

A flag wins for one command:

printf '[{"msg":"urgent"},{"msg":"other"}]' \
  | ajq --max-calls 0 '.[] | select(.msg =~ "urgent") | .msg'

Environment variables are useful for a shell session or CI job:

export AJQ_BACKEND=mock
export AJQ_MAX_CALLS=0

The config file still supplies any setting that the flag or environment did not override.

4. Keep API keys out of config

API keys are environment-only:

export ANTHROPIC_API_KEY="sk-ant-..."
export OPENAI_API_KEY="sk-..."
export OPENROUTER_API_KEY="sk-or-..."

If a config file contains credential-looking keys such as api_key, apikey, or token, ajq rejects it:

ajq: error: config key "api_key" looks like a credential; API keys are env-only

15 - Use a larger local model

List, download, and select a larger pinned GGUF model for the local backend.

The local backend defaults to qwen2.5-1.5b. Use ajq models when you want one of the larger shipped catalog models.

1. List catalog models

Run:

ajq models list

Example output before any downloads:

NAME          ACTIVE  INSTALLED  SIZE    RAM
qwen2.5-1.5b  *       no         1.2GiB  ~4 GiB RAM
qwen2.5-3b            no         2.0GiB  ~6 GiB RAM
qwen3-4b              no         2.3GiB  ~8 GiB RAM

ACTIVE shows the model selected by config/env/defaults. INSTALLED shows whether the GGUF file exists in the ajq cache.

2. Download the model you want

Choose a catalog name and pull it:

ajq models pull qwen2.5-3b

The pull uses checksum-pinned public URLs and stores the model under the ajq cache. For a project-local cache, set AJQ_CACHE_DIR before pulling:

AJQ_CACHE_DIR="$PWD/.ajq-cache" ajq models pull qwen2.5-3b

3. Select the installed model

Persist the selected local model:

ajq models use qwen2.5-3b

If the model is not installed yet, ajq tells you what to run:

ajq: error: model qwen2.5-3b is not installed; run `ajq models pull qwen2.5-3b` first

After the model is installed, models use writes the model name to config.toml:

active model set to qwen2.5-3b in /Users/you/.config/ajq/config.toml

The resulting config entry is:

model = "qwen2.5-3b"

models use preserves unrelated config keys, but it does not preserve comments or the original formatting.

4. Run with the local backend

Use the selected model with the local backend:

printf '{"msg":"urgent billing issue"}' \
  | ajq --backend local '.msg =~ "urgent"'

You can override the configured model for one command:

ajq --backend local --model qwen3-4b '.msg =~ "urgent"'