Skip to content

ref(aci): BaseDetectorHandler owns the detector evaluation lifecycle - #127133

Draft
saponifi3d wants to merge 1 commit into
masterfrom
jcallender/detector-handler-2-lifecycle
Draft

saponifi3d wants to merge 1 commit into
masterfrom
jcallender/detector-handler-2-lifecycle

Conversation

@saponifi3d

@saponifi3d saponifi3d commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

BaseDetectorHandler._evaluate now runs every detector through one lifecycle. evaluate decides which groups trigger. For each triggered evaluation that has no result yet, the platform calls create_occurrence and builds the IssueOccurrence, evidence, fingerprint, and event data. Before this, each handler did that work itself.

flowchart LR
    P[DataPacket] --> X[extract_value]
    X -->|empty DetectorGroupValues| Skip[skip]
    X --> E["evaluate(data_packet, values)"]
    E -->|triggered, result=None| C[create_occurrence]
    C --> O[platform: IssueOccurrence + evidence + fingerprint + event data]
    E -->|already has a result, e.g. StatusChangeMessage| Out[results]
    O --> Out
Loading

A new detector that decides with the detector's data conditions now needs just two methods:

class ExampleDetectorHandler(DetectorHandler[ExamplePacket, float]):
    def extract_value(self, data_packet: DataPacket[ExamplePacket]) -> float:
        return data_packet.packet.value

    def create_occurrence(
        self, evaluation: DetectorEvaluation, data_packet: DataPacket[ExamplePacket]
    ) -> tuple[DetectorOccurrence, EventData]:
        return DetectorOccurrence(issue_title=..., subtitle=..., type=ExampleGroupType, level="error", culprit=""), {}

Thresholds and priorities come from the DataConditions on the detector's trigger group. To change ids or fingerprints, override get_event_id, get_occurrence_id, or get_issue_fingerprint. A handler that doesn't use a condition group subclasses BaseDetectorHandler and implements evaluate.

  • create_occurrence(evaluation, data_packet): priority, group key, and trigger group evaluation now come from the DetectorEvaluation.
  • StatefulDetectorHandler.evaluate adds dedupe, threshold counters, and state on top of super().evaluate() (this resolves its TODO). Its event/occurrence id behavior is now expressed through the get_event_id and get_occurrence_id hooks.
  • Uptime overrides evaluate instead of _evaluate, so no handler overrides _evaluate anymore.
  • GroupedDetectorEvaluationResult is renamed to DetectorEvaluations. trigger_group_evaluation is optional for handlers that don't use a condition group.

Issue platform payloads are unchanged. I recorded every _evaluate result (IssueOccurrence, StatusChangeMessage, event_data, and the evaluation artifact logged to EAP) from the product detector test suites on master and on this branch. After masking per-run values (DB ids, UUIDs, timestamps, random fixture names), both runs produced the same output:

Detector suite Tests Evaluations Identical
Metric issues (incidents/) 34 49 34
Uptime (uptime/) 52 11 52
Preprod size analysis (preprod/size_analysis/) 31 23 31
Processing errors (processing_errors/) 9 8 9
Data packets (workflow_engine/processors/) 1 1 1

Five ProcessUpdateComparisonAlertTest cases were left out because they also give different results between runs on master. Product test files are untouched.

The one contract change: a triggered evaluation returned without a result used to pass through with no result. Now the platform builds an occurrence for it. Only the mock handler in processors/test_detector.py returned that shape, so the two tests that relied on it now assert the occurrence.

@saponifi3d
saponifi3d added this pull request to stack #127136 October 10, 2026 01:03
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 10, 2026
Decides the outcome for each group's extracted value. Ungrouped detectors receive a single value keyed by `None`.

Return `DetectorGroupValues` to evaluate each group independently.
Return a triggered `DetectorEvaluation` with `result=None` for each group that should create an issue;

@RudraPatel2003 RudraPatel2003 Oct 21, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think from a type standpoint it is super clear that result=None means that this method will handle create_occurrence for you

I think it would be nice if we could have a more explicit typing to make it super clear from the perspective of building a detector "In THIS case I want to create issues, in THIS OTHER case I do not"

Base automatically changed from jcallender/detector-handler-1-prep to master October 21, 2026 23:35
_evaluate now composes extract_value, evaluate, and create_occurrence for every handler: evaluate decides which groups trigger, and the platform builds the IssueOccurrence, evidence, fingerprint, and event data for each triggered evaluation without a result. A new detector built on DetectorHandler only implements extract_value and create_occurrence; detection logic lives in the detector's data conditions.

StatefulDetectorHandler layers dedupe, thresholds, and state over super().evaluate() instead of copying the condition evaluation and occurrence decoration. Uptime overrides evaluate instead of _evaluate. Issue platform payloads are unchanged.
@saponifi3d
saponifi3d force-pushed the jcallender/detector-handler-2-lifecycle branch from c6a205d to 187388f Compare October 21, 2026 23:50

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants