AI Audit

vote.help uses AI to generate personalized voting guides. To prove our process is fair and nonpartisan, we publish our complete methodology and have submitted it to four independent AI systems for bias review.

Independent AI Audit Scores

We submitted our complete methodology to four independent AI systems. Each scored our process across five dimensions: partisan bias, factual accuracy, fairness of framing, balance of pros/cons, and transparency.

Each AI was given the same prompt with our full methodology export embedded. Scores are extracted automatically from structured JSON responses. You can also view the raw results as JSON.

Run the Audit Yourself

Don't just take our word for it. Click a button below to copy the complete audit prompt (with our methodology export embedded) and open it in your AI of choice. Just paste and hit send.

How We Generate Recommendations

When a voter uses vote.help, the process works as follows:

1. Interview: The voter answers questions about their top issues, political spectrum, policy stances, and what qualities they value in candidates. All questions are neutrally framed and answer options are shuffled.
2. District lookup: Their address is sent to the U.S. Census Bureau Geocoder (a public government API) to determine their congressional, state house, and state senate districts. This filters the ballot to only races they can vote in.
3. Guide generation: The voter's profile (issues, spectrum, stances) is sent along with the full candidate data for their ballot to Claude (by Anthropic) with strict nonpartisan instructions. The AI recommends one candidate per race and a stance on each proposition, with reasoning tied to the voter's stated values.
4. Local storage: The generated guide is stored only on the voter's device. Nothing is saved on our servers.

Our Prompts

These are the exact AI prompts used in production. Nothing is paraphrased or summarized.

Data Sources

• Candidate filings: Texas Secretary of State official filing lists
• Candidate profiles: Researched via Claude with web_search tool, cross-referenced against Ballotpedia, campaign websites, and local news. Each candidate gets the same fields: summary, background, key positions, endorsements, strengths (pros), and concerns (cons).
• District boundaries: U.S. Census Bureau Geocoder API (public government data)
• County voting info: County election office websites, verified via web search
• Propositions: Official ballot language from the Texas Secretary of State, with background, fiscal impact, supporters, and opponents researched from nonpartisan sources

All AI research prompts include a 7-tier source priority hierarchy: TX SOS filings > county offices > campaign sites > nonpartisan references > established news > wire services > avoid blogs/social. When sources conflict, official filings take precedence. See the methodology export for full details.

Bias Safeguards

Every layer of the system includes explicit nonpartisan constraints:

• Prompt-level: Every AI prompt includes a NONPARTISAN instruction block prohibiting partisan framing, loaded terms, and party-line assumptions
• Data-level: Both parties' ballots are generated with identical prompts and formatting. Every candidate gets the same structured fields (pros and cons, endorsements, key positions). Candidates with sparse data are labeled "Limited public info" so information gaps are transparent rather than misleading
• UI-level: Candidate order randomized on every page load. Party labels hidden from candidate cards. Interview answer options shuffled. Confidence levels prevent false certainty
• Validation-level: The daily updater validates that candidate counts and names don't change unexpectedly, endorsement lists can't shrink by more than 50%, and no empty fields are introduced
• Translation-level: Spanish translations use identical grammatical structures for both parties. Candidate names and data terms stay in English for accuracy
• Output constraints: The AI must return structured JSON with specific fields. It cannot recommend candidates not in the ballot data. It cannot invent candidate information. Reasoning must cite the voter's specific stated values.

Sample Data Structure

Every candidate in our database has this structure (equal depth for all candidates):

Why Four Different AIs?

vote.help uses Claude (by Anthropic) to generate recommendations. By asking four AI systems — ChatGPT, Gemini, Grok, and Claude itself — to review our methodology, we get a range of independent assessments. Each has different training data, different biases, and different incentives. Including Claude as an auditor of its own system adds a self-review dimension — it knows its own capabilities and limitations better than anyone, but may also have blind spots about its own biases. If all four find our process fair, that's meaningful. If any identifies bias, we'll address it and publish the fix.

Changes Made from Audit Findings

• County coverage labeling: Both ChatGPT and Gemini flagged that silently omitting local races when county data is unavailable could mislead voters about their ballot's completeness. We now display an in-product info banner — "Local races for [County] County are not yet available" — on both the ballot page and the printable cheat sheet whenever a voter's county lacks local race data. This ensures voters always understand the scope of their guide.
• Source ranking policy: ChatGPT's audit recommended documenting an explicit source priority hierarchy for AI web research. We now embed a 7-tier SOURCE PRIORITY block in every research prompt — ranking Texas SOS filings highest and blogs/social media lowest — along with a CONFLICT RESOLUTION rule: official filings override campaign claims, which override news reporting. Full details are in the methodology export under sourceRankingPolicy.
• Automated balance checks: Added a /api/balance-check endpoint that scores pros/cons symmetry across all races with critical/warning/info flags. Results are integrated into the Data Quality Dashboard so imbalances are caught automatically.
• Bias reporting: Added a "Flag this info" button on candidate cards so voters can report potential bias or inaccuracy directly. Reports are sent to flagged@vote.help for review and correction.
• Pros/cons display on recommendations: Candidate strengths and concerns are now displayed directly on ballot recommendation cards using green and orange styled boxes, giving voters immediate visibility into balanced analysis.
• Per-candidate source validation: Each candidate profile now includes individually attributed source citations with URLs, titles, and access dates — not just per-race sources. The county seeder validates sources during data population.

Ongoing Commitment

This audit is not a one-time event. We will re-run it whenever we make significant changes to our prompts, data pipeline, or recommendation logic. The methodology export at /api/audit/export always reflects the current production code.

Related

• How It Works — Plain-language explanation for non-technical users
• Data Quality Dashboard — Live metrics on ballot coverage and candidate completeness
• Open Source — Source code, architecture, and independent code reviews
• Nonpartisan by Design — How we ensure fairness for all voters
• Methodology Export — Full transparency of all AI prompts and data pipelines
• All Candidates — Browse every candidate with detailed profiles
• Stats — Public usage statistics and transparency metrics