AI Projects
MarTech Vendor Evaluation Agent
AI research tool evaluating MarTech vendors against a pre-determined scorecard.
Agentic Design
Researcher-Judge Workflow
Tech Stack
Python · LangGraph · Anthropic API (researcher) · OpenAI API (judge) · Gradio
How it works
STEP 1
Gather evidence
- You give the agent a vendor name, your use case, and any documents you already hold, such as analyst reports or an RFP response. NOTE: Only vendor name is mandatory
- The agent researches the vendor across public sources including product pages, analyst coverage, customer reviews, and trust portals
- Every fact is mapped to the scorecard question it answers and graded by source strength, from vendor-stated to independently verifiable
STEP 2
Generate scores
- The agent scores the vendor 1–5 (1=low, 5=high) on 28 weighted criteria across 8 categories
- Categories include functional fit, integration, AI capability, security, cost, vendor viability, lock-in risk, and implementation support
- Claims backed only by vendor marketing are capped at 3. Any missing evidence stays blank and is not guessed
STEP 3
Cross-examine & report
- A second AI from a different provider plays judge and reviews the evidence gathered by the first agent
- It challenges scores that are weakly supported, flags evidence gaps, and recommends adjustments
- The final output is a downloadable Excel report with scores, sources, and judge dissent