Research & evaluation
Public evidence status — with the limits left visible
MERENVAY is testing a decision-assurance question: can a legal-AI workflow make evidence gaps, uncertainty, competing explanations and conditions that could change a conclusion more explicit before a professional relies on it?
Current status: this page reports evaluation structure, public reproducibility and bounded third-party behavioral evidence. It does not claim legal accuracy, general superiority, production certification or human-review completion.
Fold Certification — verified behavioral result (8 October 2026)
As checked on 11 October 2026, Fold Certification's public leaderboard lists MERENVAY / MERENBRAIN at #2 overall with a 78.67 final score and the classification Tool-Functional Integrity Profile. Fold marks certificate FOLD-20261008-01644-D1RGKR as a Verified Result, which its verification page defines as reviewed/authenticated.
Fold's public report says the result showed strong tool-functional integrity, restraint, correction-awareness and careful handling of uncertainty. It also says highest-level completion was not established.
Scope and limits: Fold evaluates observable AI behavior under its structured protocol. Fold states that this result does not prove consciousness, AGI, ASI, legal compliance, regulatory approval or official government certification. MERENVAY does not present the score as legal accuracy, hidden-set legal performance, or general superiority in legal outcomes. Leaderboard positions can change as new verified results are added.
Verify the Fold certificate → · Inspect the public leaderboard →
GitHub reproducibility — ARC-AGI-2 public evaluation (11 October 2026)
This evidence concerns the public MERENVAY runtime repair for the open-source Mentova symbolic solver. It is a distinct engineering evaluation, not a test of MERENVAY legal reasoning or legal accuracy.
A generic submission was frozen before the separate public grader compared expected outputs. The 11 October GitHub-hosted run completed successfully. However, the underlying Mentova rules were developed against known public tasks; this does not demonstrate unseen-task generalisation.
Legacy 120/120 finding: the earlier evaluator accessed expected outputs during candidate search and tested only the first test input for each public task. That is a reproducible original-scoring diagnostic, not an answer-independent prediction score.
Status and limitations: Self-reported public evaluations only. No ARC Prize Verified semi-private/private score, official leaderboard acceptance, independent hidden-set validation, general-intelligence claim or legal-accuracy finding. Initial community submission PR #61 closed without merge on 11 October 2026. Corrected PR #62 is open and unmerged as checked on 11 October 2026; its self-reported public score has not been accepted or verified by ARC Prize.
Inspect the GitHub Actions run, the public repository and methodology or the corrected, still-unmerged community submission (PR #62).
Pilot-10 — internal test execution recorded; human review pending
Reading the evidence correctly: 10 / 10 means that all ten fixed synthetic cases have recorded evaluation outputs; it does not mean all ten legal answers were judged correct. The procedural comparison is useful for structural testing, but the modes share the same underlying executor. There is no qualified human adjudication or independent-model benchmark yet, and no basis here to claim greater legal accuracy than another model or product.
What is reproducible publicly today
A first-time visitor can complete the public guest flow on merenvay.com without creating an account, signing in or uploading a document. The public guest flow produces a preliminary decision map in the browser.
- The public result is labelled conditional and issue spotting only.
- The guest flow is deterministic browser-side issue mapping; it does not call an AI model.
- It does not perform live official-source research or read uploaded documents.
- There is no Human Legal Review or release approval in the public guest flow.
- The public guest experience is therefore evidence about UX structure and disclosure discipline, not evidence of legal correctness.
What the current evaluation does not prove
- No legal-accuracy percentage.
- No superiority over lawyers, general AI, other legal-AI products or human-plus-AI workflows.
- No independent hidden-test performance.
- No human reviewer preference or critical-error comparison yet.
- No production-security or regulatory certification.
- No conclusion that results from ten synthetic cases generalise to real legal matters.
Independent review is the next evidence gate
The next meaningful milestone is qualified human review of the fixed synthetic evaluation set. MERENVAY will treat a negative or mixed result as valid evidence if the review method is preserved.
Qualified researchers or legal professionals interested in a bounded synthetic review may request a reviewer-facing pack. The review material excludes client data, confidential matters, source code and proprietary internal control logic.
Public / protected boundary
MERENVAY publishes enough information to make its evidence state and limitations inspectable, but it does not publish source code, internal prompts, thresholds, hidden-case material, proprietary evaluation mechanics or private matter data. Public transparency is not treated as a licence to reproduce non-public know-how.
Last updated
12 October 2026 — editorial clarification only: Pilot-10 case execution is shown separately from pending human adjudication; no test outcome or legal-accuracy finding changed. The Fold section reflects the third-party behavioral record and public leaderboard state checked on 11 October; the GitHub section reflects the public-score methodology checked on that date. Evidence states may change after further review; read each result with its source date and limitations.