Spaces:
Sleeping
Sleeping
Delete README-1.md
Browse files- README-1.md +0 -168
README-1.md
DELETED
|
@@ -1,168 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
title: Truth Verification Pipeline
|
| 3 |
-
emoji: 🔎
|
| 4 |
-
colorFrom: blue
|
| 5 |
-
colorTo: indigo
|
| 6 |
-
sdk: gradio
|
| 7 |
-
sdk_version: 6.20.0
|
| 8 |
-
app_file: app.py
|
| 9 |
-
pinned: false
|
| 10 |
-
license: mit
|
| 11 |
-
short_description: Grounded 14-stage fact-verification API with live search
|
| 12 |
-
---
|
| 13 |
-
|
| 14 |
-
# Truth Verification Pipeline
|
| 15 |
-
|
| 16 |
-
A multi-stage fact-verification API. Given a claim or question, it extracts
|
| 17 |
-
the underlying factual assertions, searches the live web for evidence,
|
| 18 |
-
scores and cross-checks that evidence, and returns a verdict with a
|
| 19 |
-
calibrated confidence score and the exact sources used.
|
| 20 |
-
|
| 21 |
-
It's built to be called programmatically (see **API usage** below) as much
|
| 22 |
-
as used through the demo UI above.
|
| 23 |
-
|
| 24 |
-
## Setup
|
| 25 |
-
|
| 26 |
-
This Space needs two secrets set under **Settings → Repository secrets**:
|
| 27 |
-
|
| 28 |
-
| Secret | Required | Purpose |
|
| 29 |
-
|---|---|---|
|
| 30 |
-
| `GROQ_API_KEY` | Yes (primary) | Runs the pipeline's reasoning/extraction stages via Groq (`openai/gpt-oss-120b`) |
|
| 31 |
-
| `GEMINI_API_KEY` | Recommended (fallback) | Used only if Groq is unavailable, via `gemini-3.1-flash-lite` |
|
| 32 |
-
|
| 33 |
-
If neither key is set, the API responds with a clear error instead of
|
| 34 |
-
crashing. If only one is set, that provider handles every stage.
|
| 35 |
-
|
| 36 |
-
No search API key is needed — evidence retrieval uses `ddgs`, a free
|
| 37 |
-
multi-engine metasearch library (no signup required).
|
| 38 |
-
|
| 39 |
-
## API usage
|
| 40 |
-
|
| 41 |
-
The UI above exposes a single endpoint, `verify`, via the Gradio client:
|
| 42 |
-
|
| 43 |
-
```python
|
| 44 |
-
from gradio_client import Client
|
| 45 |
-
|
| 46 |
-
client = Client("your-username/your-space-name")
|
| 47 |
-
result = client.predict(
|
| 48 |
-
"The Great Wall of China is visible from space with the naked eye.",
|
| 49 |
-
api_name="/verify"
|
| 50 |
-
)
|
| 51 |
-
print(result)
|
| 52 |
-
```
|
| 53 |
-
|
| 54 |
-
Or over plain HTTP:
|
| 55 |
-
|
| 56 |
-
```bash
|
| 57 |
-
curl -X POST https://your-username-your-space-name.hf.space/call/verify \
|
| 58 |
-
-H "Content-Type: application/json" \
|
| 59 |
-
-d '{"data": ["Is the Great Wall of China visible from space?"]}'
|
| 60 |
-
```
|
| 61 |
-
|
| 62 |
-
### Response shape
|
| 63 |
-
|
| 64 |
-
```json
|
| 65 |
-
{
|
| 66 |
-
"answer": "string — the final, evidence-grounded answer",
|
| 67 |
-
"confidence": 0.0,
|
| 68 |
-
"reasoningSummary": "string — why this verdict was reached",
|
| 69 |
-
"supportingEvidence": [{"fact": "string", "source": "url"}],
|
| 70 |
-
"sources": ["url", "..."],
|
| 71 |
-
|
| 72 |
-
"verdict": "Verified | Debunked | Uncertain | Disputed | Not Applicable — Opinion/Subjective | Rate Limited | No Input | Error",
|
| 73 |
-
"claimsAnalyzed": ["the atomic claims extracted from your input"],
|
| 74 |
-
"domains": ["subject-matter tags, e.g. Science, Health, Politics"],
|
| 75 |
-
"caveats": ["warnings, e.g. high-stakes domain or safety adjustment"],
|
| 76 |
-
"sourceCount": 0,
|
| 77 |
-
"knowledgeGraph": [{"subject": "...", "predicate": "...", "object": "..."}],
|
| 78 |
-
"pipelineWarnings": ["transparency notes about anything that degraded, e.g. a stripped fabricated source"],
|
| 79 |
-
"verifiedAt": "ISO 8601 timestamp"
|
| 80 |
-
}
|
| 81 |
-
```
|
| 82 |
-
|
| 83 |
-
`answer`, `confidence`, `reasoningSummary`, `supportingEvidence`, and
|
| 84 |
-
`sources` are the original fields and are always present with these exact
|
| 85 |
-
names/types. Everything else is additive.
|
| 86 |
-
|
| 87 |
-
Rate limit: 5 requests per client IP per 60 seconds.
|
| 88 |
-
|
| 89 |
-
## How the pipeline works
|
| 90 |
-
|
| 91 |
-
Every input goes through the same 14 stages. Stages 1–12 are LLM calls with
|
| 92 |
-
a specific, narrow job; stages 13–14 are plain Python — no model involved —
|
| 93 |
-
which is deliberate: the parts of the pipeline that most directly police
|
| 94 |
-
accuracy don't rely on an LLM grading its own homework.
|
| 95 |
-
|
| 96 |
-
1. **Query Analyzer** — classifies the input as a verifiable factual claim
|
| 97 |
-
vs. an opinion/preference. Opinions short-circuit immediately with a
|
| 98 |
-
`Not Applicable — Opinion/Subjective` verdict instead of being forced
|
| 99 |
-
through a Verified/Debunked judgment they were never suited for.
|
| 100 |
-
2. **Claim Extractor** — splits the input into distinct, atomic,
|
| 101 |
-
independently-checkable claims (a compound sentence can assert several
|
| 102 |
-
things at once; each is checked on its own).
|
| 103 |
-
3. **Domain Classifier** — tags the subject domain(s) (Science, Health,
|
| 104 |
-
Politics, Finance, etc.). Claims in high-stakes domains (health, legal,
|
| 105 |
-
financial, elections) get an explicit caveat appended to the final
|
| 106 |
-
answer.
|
| 107 |
-
4. **Search Query Generator** — writes real search-engine queries for each
|
| 108 |
-
claim. Deliberately generates queries aimed at *confirming* the claim
|
| 109 |
-
**and** queries aimed at *refuting* it, to avoid one-sided,
|
| 110 |
-
confirmation-biased research.
|
| 111 |
-
5. **Retrieval Engine** — runs those queries against `ddgs`, which itself
|
| 112 |
-
fans out across multiple engines (Bing, Brave, Google, DuckDuckGo,
|
| 113 |
-
Yahoo, Yandex, Wikipedia) and aggregates/de-duplicates results. Retried
|
| 114 |
-
with backoff on transient failures.
|
| 115 |
-
6. **Evidence Extractor** — pulls factual snippets out of the raw search
|
| 116 |
-
results. The model is only shown the URLs actually returned by search
|
| 117 |
-
and is explicitly told never to invent one; anything it cites that
|
| 118 |
-
doesn't match a real retrieved URL is discarded right here.
|
| 119 |
-
7. **Evidence Scorer** — scores each snippet for relevance and reliability.
|
| 120 |
-
Reliability is a *blend* of the model's judgment and a rule-based prior
|
| 121 |
-
(`.gov`/`.edu`/major wire services score higher, personal blogs score
|
| 122 |
-
lower) �� so one model's guess about source quality isn't the only
|
| 123 |
-
signal.
|
| 124 |
-
8. **Verification Engine** — reaches a first-pass verdict (Verified /
|
| 125 |
-
Debunked / Uncertain) strictly from the scored evidence, with explicit
|
| 126 |
-
instructions not to guess confidently when evidence is thin or mixed.
|
| 127 |
-
9. **Conflict Resolver** — an adversarial second pass whose only job is to
|
| 128 |
-
find problems with the verdict from stage 8: underweighted
|
| 129 |
-
contradictions, sources disagreeing with each other, low-reliability
|
| 130 |
-
evidence being treated as decisive. It can downgrade or change the
|
| 131 |
-
verdict, or mark it "Disputed" if credible sources genuinely disagree.
|
| 132 |
-
10. **Truth & Safety Policy** — reviews the resolved verdict for
|
| 133 |
-
responsible-communication concerns (e.g. medical/legal framing,
|
| 134 |
-
defamation risk) and can attach a phrasing adjustment that the final
|
| 135 |
-
answer is required to apply.
|
| 136 |
-
11. **Knowledge Graph Builder** — structures the verified facts into
|
| 137 |
-
subject–predicate–object triplets, surfaced in `knowledgeGraph`.
|
| 138 |
-
12. **Response Generator** — writes the final answer, reasoning summary,
|
| 139 |
-
and cites its supporting evidence, using only the exact source URLs
|
| 140 |
-
already validated in earlier stages.
|
| 141 |
-
13. **Grounding Audit** *(code, not a model call)* — a hard check that
|
| 142 |
-
strips any source in the final answer that isn't in the set of URLs
|
| 143 |
-
actually retrieved in stage 5. This is the last line of defense against
|
| 144 |
-
a fabricated citation slipping through.
|
| 145 |
-
14. **Confidence Calibration** *(code, not a model call)* — recomputes the
|
| 146 |
-
final confidence score from concrete signals rather than trusting the
|
| 147 |
-
model's self-reported number: it's capped hard if search returned
|
| 148 |
-
nothing, capped further if fewer than 2 independent sources back a
|
| 149 |
-
"Verified"/"Debunked" verdict, and capped again if the grounding audit
|
| 150 |
-
had to remove anything.
|
| 151 |
-
|
| 152 |
-
Throughout, every LLM call is wrapped in JSON-repair logic (strip markdown
|
| 153 |
-
fences → parse → extract the first balanced JSON block → one corrective
|
| 154 |
-
retry) with a typed fallback default, and the whole pipeline runs inside a
|
| 155 |
-
top-level try/except — so a single stage failing degrades that one part of
|
| 156 |
-
the result (and is logged in `pipelineWarnings`) instead of crashing the
|
| 157 |
-
request.
|
| 158 |
-
|
| 159 |
-
## Known limitations
|
| 160 |
-
|
| 161 |
-
- Web search quality depends on what the underlying engines surface; a
|
| 162 |
-
claim about something very recent or very obscure may return little or
|
| 163 |
-
no evidence, in which case the pipeline correctly reports low confidence
|
| 164 |
-
rather than guessing.
|
| 165 |
-
- The rule-based source-credibility list is a small, illustrative set of
|
| 166 |
-
high/low-credibility domain hints, not an exhaustive authority ranking.
|
| 167 |
-
- This is a research/demo tool, not a substitute for professional
|
| 168 |
-
judgment — see the caveats field for claims in high-stakes domains.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|