Defining the quality standard of the Agent era. 我们正在定义 Agent 时代的 质量标准。

From Agent evaluation infrastructure to AI workflow automation — we build the full stack that makes AI products reliable and deployable at scale. 从 Agent 评测基础设施到 AI 工作流落地， 我们构建让 AI 产品真正可靠、可规模化交付的全套能力。

Explore Products探索产品 Let's talk聊聊合作

§ 01 · THE PROBLEM § 01 · 问题

Agents fail
silently. Agent 正在
无声地失效。

Agents don't crash — they smoothly do the wrong thing. APIs return 200, tools fire at the wrong moment, context drifts across turns, the answer reads fluent but is business-useless. Today's tools tell you whether the system is up. They can't tell you whether it's any good. Agent 不会崩溃——它们顺滑地做错事。接口返回 200，工具在错误的时机被调用，上下文在多轮中悄悄漂移，回答看似通顺却毫无业务价值。现有工具只能告诉你系统"挂没挂"，无法告诉你它"好不好"。

0% Already in production已部署生产

Of enterprises have already deployed AI Agents into production environments. 企业已将 AI Agent 部署至真实生产环境。

0%+ Predicted to fail · Gartner预计失败 · Gartner

Of Agentic AI projects will be cancelled by 2027 — quality monitoring is the single biggest cause. 的 Agentic AI 项目将于 2027 年前因质量监控缺失被叫停。

0 Industry standard行业标准

There is still no shared answer to the question: what does Agent quality even mean? 至今没有共同的答案来回答：什么叫 Agent 质量合格？

GAP · 01

Smooth failures, invisible to logs 顺滑的失败，日志看不见

Tracing dashboards show latency and token cost. They don't show why the user gave up in turn 3, or why the demo worked but production doesn't. trace 看板能看延迟和 token 成本，但看不见用户为什么在第 3 轮放弃，也看不见 demo 能跑但生产不行的真正原因。

GAP · 02

Multi-turn collapse, single-turn benchmarks 多轮崩塌，单轮评测

Real Agent quality lives in trajectories — context fidelity, tool timing, recovery, session-level goal completion. None of which a single-turn benchmark can capture. 真正的质量发生在轨迹上——上下文保真、工具时机、错误恢复、会话级目标。这些是单轮 benchmark 永远抓不到的。

GAP · 03

No data flywheel from production 线上数据没有飞轮

Real failure logs accumulate by the millions. None of them automatically become the regression sets, failure taxonomies, or eval candidates that should drive the next iteration. 百万级失败日志在堆积，却没有一条自动变成下一版的回归集、失败分类或评测样本。

SEE WHO FEELS THIS MOST 看看哪些行业最痛

§ 02 · WHAT WE BUILD § 02 · 我们在构建

Four layers. One closed loop. 四层能力，一个闭环。

From production sessions to product decisions — every layer turns information into action that the next iteration can use. 从真实生产会话到产品决策——每一层都把信息转化为下一次迭代可使用的行动。

01 Collect采集

Sessions, structured 会话结构化

Trace ingestion, tool-call normalization, user/agent turn alignment, drop & retry & escalation events — all on OpenTelemetry, no lock-in. trace 摄取、工具调用归一化、用户/Agent 轮次对齐，drop / retry / escalation 事件——基于 OpenTelemetry，无锁定。

02 Discover发现

Failure patterns, ranked 失败模式排序

Auto-clustering of recurring failures, drop-off detection, tool misuse, plan rupture, memory drift — surfacing the worst, not the random. 自动聚类反复出现的失败、流失点识别、工具误用、计划断裂、记忆漂移——浮现最值得修的，而不是随机抽样。

03 Reflow回流

Production → eval, automatic 线上自动转评测

Production traces become eval candidates, regression sets, and version-comparable benchmarks. Every fix is verifiable. Every release improves. 生产 trace 自动生成评测样本、回归集、版本可对比基准。每次修复都可验证，每次发版都在变好。

04 Decide决策

A shared quality language 共同的质量语言

Translate technical failures into "what broke, why, who's affected, fix priority." PM, Eng, and Ops finally argue about the same thing. 把技术失败翻译为"哪里坏、为什么、影响谁、先修哪个"。PM、工程、运营终于在讨论同一件事。

§ 03 · WHY THIS COMPOUNDS § 03 · 为什么会复利

PRODUCTION SESSIONS线上会话
FAILURE DIAGNOSIS失败诊断
REGRESSION ASSETS回归资产
VERIFIED FIX验证修复
0×
customer客户

Our 100th customer is more valuable than our 1st. 我们的第 100 个客户，比第 1 个更有价值。

Every customer integration deposits real ground truth into our system — what "good" looks like for that scenario, what failure modes recur, how user behavior translates to satisfaction. The taxonomy thickens. The benchmarks sharpen. The standard becomes inevitable. 每一次客户接入，都向系统沉淀真实的 ground truth——这个场景下"好"的样子、反复出现的失败模式、用户行为如何映射到满意度。分类持续加厚，基准持续锐化，标准走向不可逆。

Functions can be copied. A 12-month head start of real-world failure data cannot. 功能可以被复制。但 12 个月的真实失败数据飞轮，无法被复制。

MEET THE TEAM BUILDING THIS 见见正在做这件事的团队
01 Products产品 → 02 Industry行业 → 03 Services服务 → 04 Resources资源 → 05 Company公司 →

Join us · let's set the standard together 一起把标准做出来

We're looking for
the next sharp mind. 我们正在寻找
锐利的思考者

Co-founder level · equity design · core decision-maker from day one.
We need people who can turn ambiguity into systems. 联合创始人身份 · 参与股权设计 · 从第一天起进入核心决策。
我们需要能把模糊问题做成系统的人。

CO-FOUNDER · CTO

CTO · Chief Technology Officer CTO · 首席技术官

Agent evaluation engine · trace intelligence · data flywheel Agent 评测引擎 · Trace 智能分析 · 数据飞轮

You will own你会负责

  • Architecture and implementation for trace ingestion, analysis pipelines, LLM-as-judge and regression set management. 评测引擎的系统架构与工程实现：trace 摄取、分析 pipeline、LLM-as-judge 框架和回归测试集管理。
  • SDK onboarding that lets builders integrate within 30 minutes while keeping data local. 设计并推进 SDK 接入体验：让 builder 能在 30 分钟内完成接入，数据不出本地。
  • Product definition: turning "evaluation" from an abstract idea into concrete modules teams can ship with. 参与产品定义：把"评测"这件抽象的事情拆成可工程化、可交付的模块。

You probably are你大概是

  • Someone who has shipped LLM apps or Agent systems and understands trace, context and tool calls in production. 做过 LLM 应用 / Agent 系统完整工程链路，知道 trace、context、tool call 在生产环境里怎么流转。
  • Experienced with observability, evaluation or data pipelines. LangSmith, Datadog or OpenTelemetry experience is a plus. 对可观测性、评测、数据 pipeline 有真实工程经验；做过 LangSmith、Datadog、OpenTelemetry 相关接入是加分项。
  • AI native, independent, and more excited by valuable uncertainty than by a clean title. AI Native，能独立提出技术方案并动手做出来；在乎"有价值的事"多于 title，愿意在早期不确定性里换更大的空间。
Co-founder title · equity design · real problem · validated early market · team already running. 联合创始人身份 · 参与公司股权设计 · 真实存在且市场刚被验证的问题 · 一支已经在跑的团队。
Apply for CTO 申请 CTO 岗位
OPEN CONVERSATION

You · if we haven't named the role yet 你 · 如果还没有一个岗位名能框住你

Research · engineering · founding customers · anything load-bearing 研究 · 工程 · 首批共生客户 · 任何承重的位置

  • You've done something irreducible — can't be replaced by the average做过别人替代不了的事 —— 独特到不能被平均
  • You think about compounding — not the next sprint, the next decade你的时间尺度是复利，不是冲刺
  • You build things instead of explaining why they can't be built你把想法落地，而不是解释为什么做不了
  • You want co-founder stakes, not a salary bracket你要的是合伙人位置，不是薪资区间
Write to us directly 直接写信给我们

Reach out to联系我们

Roger Yang · Founder

Write us a letter 写一封信给我们