rageval - v0.1.1
    Preparing search index...

    Interface Metric

    A Metric is a named evaluation function that scores a single RAG sample.

    All five built-in metrics implement this interface. You can also implement it yourself to create custom domain-specific metrics -- any string name is accepted, not just the five built-in names:

    import type { Metric, MetricInput, MetricOutput } from 'rageval'
    import { jsonInstruction, parseLlmScore } from 'rageval'

    // Custom metric with a domain-specific name — any string is valid
    const sourceAttribution: Metric = {
      name: 'sourceAttribution',
      description: 'Checks whether the answer cites sources from the context.',
      async score(input, provider, includeReasoning = false) {
        const prompt = `Evaluate whether the answer cites sources.

    CONTEXT: ${input.contexts.join('\n')}
    ANSWER: ${input.answer}

    ${jsonInstruction(includeReasoning)}`
        const response = await provider.complete(prompt)
        const { score, reasoning } = parseLlmScore(response)
        return { score, ...(includeReasoning && { reasoning }) }
      },
    }
    interface Metric {
        name: string;
        description: string;
        score(
            input: MetricInput,
            provider: LlmProvider,
            includeReasoning?: boolean,
        ): Promise<MetricOutput>;
    }
    Index

    Properties

    name description

    Methods

    score

    Properties

    name: string

    Unique camelCase name identifying this metric. Used as keys in score results — must be unique within a single evaluate() call. Any string is valid; built-in metrics use faithfulness, contextRelevance, etc.

    description: string

    Human-readable description of what this metric measures.

    Methods

    • Compute the metric score for one RAG sample.

      Parameters

      • input: MetricInput

        The sample's question, answer, contexts, and optional groundTruth.

      • provider: LlmProvider

        The LLM provider used as the judge for this evaluation.

      • OptionalincludeReasoning: boolean

        When true, include the LLM's reasoning in the output.

      Returns Promise<MetricOutput>

      Promise resolving to a MetricOutput with score and optional reasoning. May return skipped: true when the metric cannot be computed (e.g. no groundTruth).

    Settings

    Member Visibility

    On This Page

    Properties
    Methods