Skip to content

Contribute to Bearing

The code is open, but so are the questions behind the code. Bearing benefits from contributors who test our assumptions, correct evidence, improve experience or suggest better data structures. You should not have to request access to a maintained database to participate.

For commands, PR steps and licence guidance, read CONTRIBUTING.md.

Pick the contribution that fits

Contribution Example How to start
Evidence A benchmark refers to a reasoning variant, not the base model Model or evidence issue
Methodology A capability gate excludes a plausible candidate Challenge the methodology
Data model Deployment route and evaluated model variant need separate identities Propose a schema change
Code / tests Add a regression case to the deterministic scoring engine Fork and make a PR
Design / documentation Explain a decision or simplify a confusing screen Issue or small PR

A useful contribution may be a careful question backed by an example, not a finished answer.

No Neon access? That's normal.

The repository includes scoring functions, tests, evaluation tasks, migration SQL and a generated model registry. Much of this can be inspected and checked without a database.

  • No environment setup: participate in issues and proposals, or edit Markdown.
  • Local code checks: npm ci, npm test, npm run eval:ranking, npm run lint, npx tsc --noEmit, npm run build.
  • Migration checks: npm run db:test:migrations runs the real migration files on an isolated, disposable PostgreSQL container using Docker, with no Neon credentials. This is not a production ledger check.
  • Full app or persistence work: requires independently supplied database/provider configuration. A fixture-backed offline workbench remains under development in PR #122, rather than being part of the production app.

The committed registry is a generated snapshot, and production maintains canonical model records in Neon. Do not assume a pull request directly editing registry JSON will survive regeneration. Use an evidence proposal with source, variant and date.

Evidence should be inspectable

A proposed change is easier to review when it records:

  • the problem and who it affects;
  • the current behaviour, expected outcome and a reproducible example;
  • what the evidence actually measures, its source, date and variant;
  • uncertainty, competing explanations and possible harm;
  • a proportionate way to evaluate whether the change helped.

Where the evidence is inconclusive, documenting the disagreement is a valid outcome. Not every challenge needs to become a score adjustment.

Decisions, not just suggestions

For changes with lasting architectural or methodological effects, propose an architecture decision record in the PR once an issue has established the question. Distinguish a proposal from an accepted decision. Maintainers decide production changes, with rationale and outstanding uncertainty recorded in public.

What we won't ask for

We do not need your production credentials, data exports, user messages, raw prompts, private outcomes or personally identifying data to reproduce an issue. Use synthetic examples, minimal samples and public evidence, observing licences attached to third-party benchmarks.

See also How Bearing works, How we rate models and the public data model.