System Overview

This system is designed as a modular, multi-tenant evaluation pipeline for sales conversations. Rather than treating LLMs as free-form generators, it treats them as structured evaluators operating inside a controlled, observable workflow.

At a high level, the system transforms raw conversation data into validated, analyzable evaluation artifacts through a sequence of well-defined stages.

High-Level Flow

The end-to-end flow follows five primary steps:

Transcript ingestion
Prompted LLM scoring
Structured JSON validation
Storage and analytics
Evaluation

Each step is independently observable, retryable, and auditable, allowing the system to scale while maintaining evaluation integrity.

Transcript Ingestion

Transcripts currently enter the system exclusively through manual file uploads. Integration with external call platforms (e.g. sales calls, demos) is planned for future releases.

During ingestion:

Raw transcripts are queued for processing
A lightweight pre-processing stage normalizes the data into a consistent internal format: Speaker turns, timestamps, and metadata (rep, org, call context) are extracted or standardized

Both the raw transcript and the normalized representation are persisted. This ensures traceability, reproducibility, and the ability to re-run scoring logic as prompts evolve.

Prompted LLM Scoring

Normalized transcripts are processed by a dedicated scoring service.

For each transcript:

The system assembles a prompt using:
- A stable scoring rubric (SPIN dimensions)
- Organization-specific prompt templates or criteria
ewable, and aligned with the SPIN framework.
The LLM is invoked synchronously or asynchronously, depending on workload and configuration.
Latency, token usage, model identity, and prompt version are captured alongside the raw model output.

Structured JSON Validation

LLM output is treated as untrusted input until validated

The system expects all scoring responses to conform to a predefined JSON schema, typically containing:

Numeric scores per SPIN dimension
Structured coaching feedback
Optional rationales or evidence tags

Validation occurs in two layers:

Schema validation, ensuring the response is structurally correct.
Business rule validation, enforcing constraints such as score ranges, required fields, and completeness

If validation fails:

The run is marked invalid
Error context is recorded for analysis
The system may optionally retry with a corrected prompt or route the request to a fallback model

This strict validation layer is what allows the system to safely operationalize LLM-based evaluation.

Storage and Analytics

PostgreSQL serves as the system’s source of truth.

Stored entities include:

Transcripts and normalized representations
Scoring runs and validated results
Prompt versions and model metadata
Evaluation metrics and audit logs

From this core data, the system derives analytical views used by dashboards and reports, such as:

Performance trends over time
Per-representative or per-company summaries
Prompt effectiveness and drift indicators

Sales & Revenue Teams

Teams that want scalable, consistent, and methodology-aligned coaching without relying exclusively on manual call reviews. The system provides structured scores and coaching signals that can support training, performance tracking, and continuous improvement.

AI Engineers

Engineers exploring LLM-as-a-Judge patterns who need a production-grade reference for prompt-based evaluation systems. The project demonstrates how to combine strict contracts, evaluation loops, and observability into a coherent architecture.

Researchers & Practitioners

Anyone investigating how LLMs can be used to assess human behavior, reasoning, or performance—while maintaining rigor, transparency, and accountability. The system is designed to make evaluation assumptions explicit and measurable.

Everything

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius.

Code block

This is a block which you can put code in, that people can copy with one click! The code to make this work is in the sitewide body and head.

try {
function factorial(n) {
  if (n === 0 || n === 1) return 1;
  return n * factorial(n - 1);
}

// Array of numbers
const numbers = [1, 2, 3, 4, 5];

// Using map to create a new array with factorials
const factorials = numbers.map(factorial);

// Object with person details
const person = { name: "Alice", age: 30, city: "New York" };

// Destructuring object properties
const { name, age } = person;

console.log(`${name} is ${age} years old.`);
console.log(`Factorials: ${factorials.join(", ")}`);
}

Callouts

Use these callouts to share important information - you can use grey to show neutral important information, yellow to share warnings, red to share limitations, and green to emphasise positive information.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

Accordions

Accordions are an excellent way to share information which only some people will need to read - that way, you aren't distracting people who don't need it. This is controlled by a Webflow interaction

Some niche topic

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

try {
function factorial(n) {
  if (n === 0 || n === 1) return 1;
  return n * factorial(n - 1);
}

Text formatting

You may need to indicate that a piece of text is to be used in code or an attribute. You can just add a styled span to indicate code.

Video block

I've not got much to say here - videos are great, and you should use them as much as possible to educate users on your product!

Image

A picture is worth a thousand words! Use images as much as possible to show your users what you're trying to explain!

Simple Setup

Configuration settings

Introduction

This is a UI Kit for building out API/developer documentation in Webflow - with a similar UI to GitBook! Below you will find various sections which you can use for building out whatever type of documentation you need. To get started, here's an intro grid block which you can use to link to different sections/pages.

Something

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius.

Something else

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius.

Nothing

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius.

Everything

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius.

Code block

This is a block which you can put code in, that people can copy with one click! The code to make this work is in the sitewide body and head.

try {
function factorial(n) {
  if (n === 0 || n === 1) return 1;
  return n * factorial(n - 1);
}

// Array of numbers
const numbers = [1, 2, 3, 4, 5];

// Using map to create a new array with factorials
const factorials = numbers.map(factorial);

// Object with person details
const person = { name: "Alice", age: 30, city: "New York" };

// Destructuring object properties
const { name, age } = person;

console.log(`${name} is ${age} years old.`);
console.log(`Factorials: ${factorials.join(", ")}`);
}

Callouts

Accordions

Some niche topic

try {
function factorial(n) {
  if (n === 0 || n === 1) return 1;
  return n * factorial(n - 1);
}

Text formatting

You may need to indicate that a piece of text is to be used in code or an attribute. You can just add a styled span to indicate code.

Video block

I've not got much to say here - videos are great, and you should use them as much as possible to educate users on your product!

Image

A picture is worth a thousand words! Use images as much as possible to show your users what you're trying to explain!

Simple Setup

Configuration settings

Step 5: Text formatting

Step 6: Video

Step 7: Image