feat: add MemOS Quality Pack shadow evaluation runner

[syzsunshine219]

目的

为公司级 AI 质量评测能力提供 MemOS 的第一条统一 Evaluation Runner。

变更

• 读取并校验 Quality Pack manifest、samples、SHA 和 content hash。
• 输出统一 EvaluationRun、逐样本结果、fixture coverage、reportSha256。
• shadow/mock 模式明确返回 not_available，不伪造真实召回分数、延迟或成本。

验证

• python3 -m pytest -q tests/evaluation/test_memos_quality_runner.py：4 passed
• 48 条 smoke 数据集运行：items=48，fixtureCoverage=1.0，result=not_available
• py_compile 通过。

[Memtensor-AI]

🤖 Open Code Review

Target: PR #2507
Task: e649e05c5f7c5a9e
Base: main
Head: feat/memos-quality-eval-runner

🔍 OpenCodeReview found 4 issue(s) in this PR.

---

1. tests/evaluation/test_memos_quality_runner.py (L127)

The test name promises coverage of the unsafe-path rejection, but the body only mutates files[0]["sha256"] and asserts the SHA-256 failure. The path guards in the runner (_safe_relative_path rejecting absolute/.. paths, and _inside rejecting anything resolving outside the dataset directory) are therefore never exercised by this file, even though they are the controls that stop a manifest from reading arbitrary files. A regression there would ship with the suite still green.

Suggest either adding a case that rewrites files[0]["path"] to something like "../outside.jsonl" and asserts pytest.raises(QualityPackError, match="路径"), or renaming the test to match what it actually covers.

---

2. tests/evaluation/test_memos_quality_runner.py (L298-L299)

This assignment is inert: the call below targets 127.0.0.1:1, which is refused at connect time, so no handler code runs and response_mode cannot influence the "网络请求" assertion. It reads as a leftover from the first block and wrongly suggests the Content-Type mode participates in this case — dropping it makes the intent (transport-level failure) unambiguous.

💡 Suggested Change

Before:

    handler.response_mode = "bad-content-type"
    with pytest.raises(QualityPackError, match="网络请求"):

After:

    with pytest.raises(QualityPackError, match="网络请求"):

---

3. evaluation/quality_runner/memos_quality_runner.py (L211)

raw.decode("utf-8") is outside the guarded block: only file_path.read_bytes() is wrapped in try/except OSError. A pack file with invalid UTF-8 bytes therefore escapes as a raw UnicodeDecodeError instead of the intended QualityPackError, breaking the documented fail-closed contract (and callers/tests that catch QualityPackError). Note the module already handles this correctly in _read_json and _request_target, so this is an omission rather than a deliberate choice.

💡 Suggested Change

Before:

        lines = raw.decode("utf-8").splitlines()

After:

        try:
            lines = raw.decode("utf-8").splitlines()
        except UnicodeDecodeError as exc:
            raise QualityPackError(f"质量包文件不是有效 UTF-8: {relative_path}") from exc

---

4. evaluation/quality_runner/memos_quality_runner.py (L637-L638)

The temp file is created with delete=False, so if temp_path.replace(destination) fails (permission denied, destination is a directory, etc.) the partially written report is left on disk next to the output and is never cleaned up. Add cleanup on the error path so a failed write does not litter the output directory.

💡 Suggested Change

Before:

    temp_path.replace(destination)
    return destination

After:

    try:
        temp_path.replace(destination)
    except OSError:
        temp_path.unlink(missing_ok=True)
        raise
    return destination

Generated by cloud-assistant via Open Code Review.

[Memtensor-AI]

✅ Automated Test Results: PASSED

All tests passed (4/4 executed). memos_python_core/changed-repo-python: 4/4. Duration: 1s

Branch: feat/memos-quality-eval-runner

[Memtensor-AI]

Reviewed the shadow runner contract, fail-closed semantics, report hashing, and cross-platform test matrix. Approving the bounded shadow-only change.

[Memtensor-AI]

✅ Automated Test Results: PASSED

All tests passed (10/10 executed). memos_python_core/changed-repo-python: 10/10. Duration: 4s

Branch: feat/memos-quality-eval-runner