Appearance
Scoring
Score function runs and steps to track how well your workflows perform over time.
Callout: Scoring is in beta. Install the latest SDK with npm install inngest@latest.
inngest.score(options)
Score a run or step. Can be called inside or outside a function.
name(string): A label for the score. Use consistent names across runs for aggregation. Examples: "accuracy", "user-feedback", "guardrail-pass".
value(number | boolean): The score value. A finite number or a boolean. Use 0-1 for percentages, integers for counts, booleans for pass/fail guardrails, or any range that fits your use case.
runId(string): The run to score. Required when scoring from outside a function. Omit when scoring the current run from inside a function.
stepId(string): The step to score. Optional. When provided, the score attaches to that specific step in the trace view.
ts
// Score the current run (inside a function)
await inngest.score({ name: "accuracy", value: 0.9 });
// Score a specific run (from anywhere)
await inngest.score({ name: "accuracy", value: 0.9, runId: "01ABC..." });
// Score a specific step
await inngest.score({
name: "accuracy",
value: 0.9,
runId: "01ABC...",
stepId: "generate-summary",
});inngest.score.experiment(options)
Attribute a score to the experiment variant that produced a result. group.experiment() returns an experimentRef, which identifies the experiment and the variant that was served. Pass it here to credit the score to that variant.
The signal you want to score often arrives much later, from somewhere else entirely: a click, a rating, an LLM-judge verdict. Because you pass the experimentRef yourself, you can credit any of these back to the variant that produced the output, even hours later from a separate run.
name(string): Score label.
value(number | boolean): Score value. A finite number or a boolean.
experiment(ExperimentRef): The experimentRef returned by group.experiment(). Identifies the experiment and the served variant. Shape: { experimentName: string, variant: string }.
runId(string): The run to attach the score to. Required when scoring from outside the run that produced the result; omit to use the current run.
Callout: Scoring an experiment from a different run than the one that served the variant? Pass that run's runId so the score shows up in the experiment view — otherwise it attaches to the run you're scoring from and won't appear under the experiment.
ts
// Inside the function that runs the experiment
const { result, experimentRef } = await group.experiment("summary-strategy", {
variants: { /* ... */ },
select: experiment.weighted({ questionLed: 50, factLed: 50 }),
});
// Persist experimentRef + runId (e.g. on your record) so later runs can score it
// From a separate, later run (a click, rating, or deferred judge)
await inngest.score.experiment({
name: "clickthrough",
value: 1,
experiment: experimentRef, // restored from your store
runId: summarizeRunId,
});step.score(id, options)
Score the current run from within a step context, as a durable, memoized step. runId is automatically set to the current run. Requires scoreMiddleware() (from inngest/experimental) on the client.
id(string): A durable step ID used to memoize this score write, so a replay or retry can't write it twice. Must be unique within the run.
options(object): Score label.Score value. A finite number or a boolean.
ts
await step.score("score-guardrail-pass", { name: "guardrail-pass", value: true });createScorer(client, config, handler)
Create a reusable deferred scorer: a separate function, triggered via defer(), that evaluates a result without blocking or slowing the run that produced it. Use it when scoring is expensive or slow, such as an LLM-as-a-judge call or waiting on a signal that arrives later.
Imported from inngest/experimental.
client(Inngest): Your Inngest client instance.
config(object): Unique identifier for the scorer function.A Standard Schema (e.g. Zod) describing the input data the scorer expects. Optional; when set, the data passed to defer() is validated against it and the handler's event.data is typed from it.
handler(function): Async function that receives event, step, and parents. Return { name, value } and the score is written for you, attributed to the parent run, and to the served variant when the scorer was deferred with an experiment ref. Return null or undefined to write nothing.
The handler's event.data is the payload sent via defer(), typed from schema. The simplest scorer returns one score and lets the SDK attribute it:
ts
import { createScorer } from "inngest/experimental";
import { z } from "zod";
export const verbosityScorer = createScorer(
inngest,
{ id: "verbosity-scorer", schema: z.object({ text: z.string() }) },
async ({ event }) => {
const wordCount = event.data.text.split(" ").length;
return { name: "verbosity", value: wordCount };
}
);To emit several scores, or to attribute explicitly, write them yourself with inngest.score.experiment(). The parent run and its served variant are on ctx.parents[0], so there's no need to persist them:
ts
export const judgeScorer = createScorer(
inngest,
{ id: "judge-summary", schema: z.object({ summary: z.string() }) },
async ({ event, step, parents }) => {
const judged = await step.run("judge", () => judge(event.data.summary));
const { experiment, runId } = parents[0];
await inngest.score.experiment({ name: "faithfulness", value: judged.faithfulness, experiment, runId });
await inngest.score.experiment({ name: "spoiler", value: judged.spoiler, experiment, runId });
return null; // scores already written above
}
);defer(id, options)
Trigger a deferred scorer from inside a function. Available as a handler argument.
id(string): A deterministic identifier for this deferred call.
options(object): The scorer function to trigger.Input data matching the scorer's schema.The experimentRef from group.experiment(). Pass it to attribute the scorer's result to the served variant; surfaced on the scorer's ctx.parents[0].experiment.
defer() is fire-and-forget: it returns void, so there's nothing to await.
ts
defer("score-feedback", {
function: feedbackScorer,
data: { ticketId: "tk_123" },
experiment: experimentRef, // optional; credits the score to the served variant
});