Say what to do with a tool result, not just what not to do - #30
Say what to do with a tool result, not just what not to do#30CNSeniorious000 wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
您好——我发现了 1 个问题
AI Agent 提示词
请处理本次代码审查中的评论:
## 个别评论
### 评论 1
<location path="lib/index.js" line_range="61" />
<code_context>
+- ONLY the cell's output and its final expression come back to you; a tool result consumed inside the cell costs you nothing until you surface it. Every tool answers in FULL — every field it declares, for every row — and you almost always want a few of them. Bind the result and end on what you actually asked: \`r = await search(q)\` then \`[(x["url"], x["score"]) for x in r["results"]]\`. Ending a cell on the raw result spends your context on fields you cannot use and will read again next turn.
</code_context>
<issue_to_address>
**问题:** 此指令告诉模型,每个桥接工具的结果都包含所有已声明的字段和所有行,但现有的 `dsh-spill-policy` 可以用溢出存储引用替代超大结果,而不是返回完整载荷。根据此指导复制的代码随后会在该引用上索引诸如 `r["results"]` 之类的字段,因而无法检索实际行。
**触发条件:** 启用 `dsh-spill-policy` 且工具结果超过其配置的内联大小限制时。
**建议修复:** 修改该表述,说明只有在结果未溢出时结果才是完整的,并解释模型应如何跟随或读取溢出引用。
```suggestion
- ONLY the cell's output and its final expression come back to you; a tool result consumed inside the cell costs you nothing until you surface it. Every tool answers in FULL — every field it declares, for every row — when the result is not spilled. If \'dsh-spill-policy\' returns a spill reference instead, follow or read that reference to retrieve the full result before indexing its fields. Bind the result and end on what you actually asked: \`r = await search(q)\` then \`[(x["url"], x["score"]) for x in r["results"]]\`. Ending a cell on the raw result spends your context on fields you cannot use and will read again next turn.
```
</issue_to_address>Sourcery 评估
等待批准。 请先处理 1 个发现的问题。
阻塞性发现:lib/index.js:61
帮助我变得更有用!请在每条评论上点击 👍 或 👎,我会利用反馈来改进审查结果。
Original comment in English
Hey - I've found 1 issue
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="lib/index.js" line_range="61" />
<code_context>
+- ONLY the cell's output and its final expression come back to you; a tool result consumed inside the cell costs you nothing until you surface it. Every tool answers in FULL — every field it declares, for every row — and you almost always want a few of them. Bind the result and end on what you actually asked: \`r = await search(q)\` then \`[(x["url"], x["score"]) for x in r["results"]]\`. Ending a cell on the raw result spends your context on fields you cannot use and will read again next turn.
</code_context>
<issue_to_address>
**issue:** This instruction tells the model that every bridged tool result contains all declared fields and all rows, but the existing `dsh-spill-policy` can replace an oversized result with a spill-store reference instead of returning the full payload. Code copied from this guidance then indexes fields such as `r["results"]` on the reference and fails to retrieve the actual rows.
**Triggers:** When `dsh-spill-policy` is enabled and a tool result exceeds its configured inline-size limit.
**Suggested fix:** Qualify the statement to say that results are full only when they are not spilled, and explain how the model should follow or read a spill reference.
```suggestion
- ONLY the cell's output and its final expression come back to you; a tool result consumed inside the cell costs you nothing until you surface it. Every tool answers in FULL — every field it declares, for every row — when the result is not spilled. If \'dsh-spill-policy\' returns a spill reference instead, follow or read that reference to retrieve the full result before indexing its fields. Bind the result and end on what you actually asked: \`r = await search(q)\` then \`[(x["url"], x["score"]) for x in r["results"]]\`. Ending a cell on the raw result spends your context on fields you cannot use and will read again next turn.
```
</issue_to_address>Sourcery assessment
Approval pending. 1 finding to address first.
Blocking findings: lib/index.js:61
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
Merged The The surfacing bullet is |
filter and aggregate in code and surface just the conclusionis advice about a category, not something the model can act on at the moment it is writing the last line of a cell. It also states the mechanism —tool results consumed inside the cell never enter the conversation— as a fact about plumbing rather than as the reason extracting pays.(The merge with
mainkeeps #36's shape clause in the same bullet — the resolved line carries both.)What it is written against
gpt-5.6-terraon VLB, whose median cell consuming 2–20K of tool output surfaces 98.5% of it:Measured, and the effect is model-dependent
Shipped with #27 as one arm. Band 2K–20K, dump rate = share of cells surfacing ≥ 90% of what they consumed, median ratio in brackets.
gpt-5.6-terra, 4/4 tasks — a wash:private_lending_33dhsa_medical_bill_liquidity_guard_30dfood_safety_dispute_33dbroadcast_exam_posture_breathing_32dmacaron-v1-venti, 2/4 tasks so far — mildly positive:private_lending_33dfood_safety_dispute_33dImportant
The regressions are terra-specific and the mechanism is legible in its cells. Told to surface something derived rather than the raw value, terra writes
print(x['result'][:12000])— obeying this rule and the neighbouring one that offers "show a slice of something large" as a reshape. It stopped echoing and started slicing, at bounds larger than the payload. venti was already slicing at a median bound of 120 and did not change habit. #34 has the histogram.With #35 and #36 alongside, terra reaches 15.1% pooled against the baseline's 47.3% and improves on 4/4 tasks — including both that regressed here. On venti the same arm is also mildly better than baseline (
private_lending: 9.5% → 7.7%, median 0.075 → 0.013), so it costs the models that were already fine nothing.Split out of #27, which deletes the sentence that contradicted this one.