Adedoyinjames commited on
Commit
ef33363
·
verified ·
1 Parent(s): 5f1d8f2

Delete README-1.md

Browse files
Files changed (1) hide show
  1. README-1.md +0 -168
README-1.md DELETED
@@ -1,168 +0,0 @@
1
- ---
2
- title: Truth Verification Pipeline
3
- emoji: 🔎
4
- colorFrom: blue
5
- colorTo: indigo
6
- sdk: gradio
7
- sdk_version: 6.20.0
8
- app_file: app.py
9
- pinned: false
10
- license: mit
11
- short_description: Grounded 14-stage fact-verification API with live search
12
- ---
13
-
14
- # Truth Verification Pipeline
15
-
16
- A multi-stage fact-verification API. Given a claim or question, it extracts
17
- the underlying factual assertions, searches the live web for evidence,
18
- scores and cross-checks that evidence, and returns a verdict with a
19
- calibrated confidence score and the exact sources used.
20
-
21
- It's built to be called programmatically (see **API usage** below) as much
22
- as used through the demo UI above.
23
-
24
- ## Setup
25
-
26
- This Space needs two secrets set under **Settings → Repository secrets**:
27
-
28
- | Secret | Required | Purpose |
29
- |---|---|---|
30
- | `GROQ_API_KEY` | Yes (primary) | Runs the pipeline's reasoning/extraction stages via Groq (`openai/gpt-oss-120b`) |
31
- | `GEMINI_API_KEY` | Recommended (fallback) | Used only if Groq is unavailable, via `gemini-3.1-flash-lite` |
32
-
33
- If neither key is set, the API responds with a clear error instead of
34
- crashing. If only one is set, that provider handles every stage.
35
-
36
- No search API key is needed — evidence retrieval uses `ddgs`, a free
37
- multi-engine metasearch library (no signup required).
38
-
39
- ## API usage
40
-
41
- The UI above exposes a single endpoint, `verify`, via the Gradio client:
42
-
43
- ```python
44
- from gradio_client import Client
45
-
46
- client = Client("your-username/your-space-name")
47
- result = client.predict(
48
- "The Great Wall of China is visible from space with the naked eye.",
49
- api_name="/verify"
50
- )
51
- print(result)
52
- ```
53
-
54
- Or over plain HTTP:
55
-
56
- ```bash
57
- curl -X POST https://your-username-your-space-name.hf.space/call/verify \
58
- -H "Content-Type: application/json" \
59
- -d '{"data": ["Is the Great Wall of China visible from space?"]}'
60
- ```
61
-
62
- ### Response shape
63
-
64
- ```json
65
- {
66
- "answer": "string — the final, evidence-grounded answer",
67
- "confidence": 0.0,
68
- "reasoningSummary": "string — why this verdict was reached",
69
- "supportingEvidence": [{"fact": "string", "source": "url"}],
70
- "sources": ["url", "..."],
71
-
72
- "verdict": "Verified | Debunked | Uncertain | Disputed | Not Applicable — Opinion/Subjective | Rate Limited | No Input | Error",
73
- "claimsAnalyzed": ["the atomic claims extracted from your input"],
74
- "domains": ["subject-matter tags, e.g. Science, Health, Politics"],
75
- "caveats": ["warnings, e.g. high-stakes domain or safety adjustment"],
76
- "sourceCount": 0,
77
- "knowledgeGraph": [{"subject": "...", "predicate": "...", "object": "..."}],
78
- "pipelineWarnings": ["transparency notes about anything that degraded, e.g. a stripped fabricated source"],
79
- "verifiedAt": "ISO 8601 timestamp"
80
- }
81
- ```
82
-
83
- `answer`, `confidence`, `reasoningSummary`, `supportingEvidence`, and
84
- `sources` are the original fields and are always present with these exact
85
- names/types. Everything else is additive.
86
-
87
- Rate limit: 5 requests per client IP per 60 seconds.
88
-
89
- ## How the pipeline works
90
-
91
- Every input goes through the same 14 stages. Stages 1–12 are LLM calls with
92
- a specific, narrow job; stages 13–14 are plain Python — no model involved —
93
- which is deliberate: the parts of the pipeline that most directly police
94
- accuracy don't rely on an LLM grading its own homework.
95
-
96
- 1. **Query Analyzer** — classifies the input as a verifiable factual claim
97
- vs. an opinion/preference. Opinions short-circuit immediately with a
98
- `Not Applicable — Opinion/Subjective` verdict instead of being forced
99
- through a Verified/Debunked judgment they were never suited for.
100
- 2. **Claim Extractor** — splits the input into distinct, atomic,
101
- independently-checkable claims (a compound sentence can assert several
102
- things at once; each is checked on its own).
103
- 3. **Domain Classifier** — tags the subject domain(s) (Science, Health,
104
- Politics, Finance, etc.). Claims in high-stakes domains (health, legal,
105
- financial, elections) get an explicit caveat appended to the final
106
- answer.
107
- 4. **Search Query Generator** — writes real search-engine queries for each
108
- claim. Deliberately generates queries aimed at *confirming* the claim
109
- **and** queries aimed at *refuting* it, to avoid one-sided,
110
- confirmation-biased research.
111
- 5. **Retrieval Engine** — runs those queries against `ddgs`, which itself
112
- fans out across multiple engines (Bing, Brave, Google, DuckDuckGo,
113
- Yahoo, Yandex, Wikipedia) and aggregates/de-duplicates results. Retried
114
- with backoff on transient failures.
115
- 6. **Evidence Extractor** — pulls factual snippets out of the raw search
116
- results. The model is only shown the URLs actually returned by search
117
- and is explicitly told never to invent one; anything it cites that
118
- doesn't match a real retrieved URL is discarded right here.
119
- 7. **Evidence Scorer** — scores each snippet for relevance and reliability.
120
- Reliability is a *blend* of the model's judgment and a rule-based prior
121
- (`.gov`/`.edu`/major wire services score higher, personal blogs score
122
- lower) �� so one model's guess about source quality isn't the only
123
- signal.
124
- 8. **Verification Engine** — reaches a first-pass verdict (Verified /
125
- Debunked / Uncertain) strictly from the scored evidence, with explicit
126
- instructions not to guess confidently when evidence is thin or mixed.
127
- 9. **Conflict Resolver** — an adversarial second pass whose only job is to
128
- find problems with the verdict from stage 8: underweighted
129
- contradictions, sources disagreeing with each other, low-reliability
130
- evidence being treated as decisive. It can downgrade or change the
131
- verdict, or mark it "Disputed" if credible sources genuinely disagree.
132
- 10. **Truth & Safety Policy** — reviews the resolved verdict for
133
- responsible-communication concerns (e.g. medical/legal framing,
134
- defamation risk) and can attach a phrasing adjustment that the final
135
- answer is required to apply.
136
- 11. **Knowledge Graph Builder** — structures the verified facts into
137
- subject–predicate–object triplets, surfaced in `knowledgeGraph`.
138
- 12. **Response Generator** — writes the final answer, reasoning summary,
139
- and cites its supporting evidence, using only the exact source URLs
140
- already validated in earlier stages.
141
- 13. **Grounding Audit** *(code, not a model call)* — a hard check that
142
- strips any source in the final answer that isn't in the set of URLs
143
- actually retrieved in stage 5. This is the last line of defense against
144
- a fabricated citation slipping through.
145
- 14. **Confidence Calibration** *(code, not a model call)* — recomputes the
146
- final confidence score from concrete signals rather than trusting the
147
- model's self-reported number: it's capped hard if search returned
148
- nothing, capped further if fewer than 2 independent sources back a
149
- "Verified"/"Debunked" verdict, and capped again if the grounding audit
150
- had to remove anything.
151
-
152
- Throughout, every LLM call is wrapped in JSON-repair logic (strip markdown
153
- fences → parse → extract the first balanced JSON block → one corrective
154
- retry) with a typed fallback default, and the whole pipeline runs inside a
155
- top-level try/except — so a single stage failing degrades that one part of
156
- the result (and is logged in `pipelineWarnings`) instead of crashing the
157
- request.
158
-
159
- ## Known limitations
160
-
161
- - Web search quality depends on what the underlying engines surface; a
162
- claim about something very recent or very obscure may return little or
163
- no evidence, in which case the pipeline correctly reports low confidence
164
- rather than guessing.
165
- - The rule-based source-credibility list is a small, illustrative set of
166
- high/low-credibility domain hints, not an exhaustive authority ranking.
167
- - This is a research/demo tool, not a substitute for professional
168
- judgment — see the caveats field for claims in high-stakes domains.