07 · Watching and steering a run
Show your work
The run takes forty seconds and the screen shows nothing until it is over, so you put print() inside the loop. Then a shell script wants only the answer, and another program wants JSON.
This lesson builds on lesson 06. New here? Start at lesson 01, or carry on: every lab is self-contained.
The canned pytest in the lab prints 3,000 lines, one of which is the failing assertion. The cell runs that output through both of the truncators you wrote last lesson, on the default budget. Where does the failing assertion end up?
truncate_head keeps the first lines, and the verdict of a test run is at the bottom
bash takes the tail and read takes the head. Same budget, opposite ends, chosen by what the text is.bash result your tool adds a line saying what was left out — the result is smaller, not spoiled.1,283 lines each way, because 50,000 bytes ran out before 2,000 lines did, and only the tail holds assert total([1, 2, 3]) == 6. Choosing the end is the whole of it. Today nothing gets cut: the problem is that a run this size shows the user nothing at all while it happens.
import harness, lab
shell = lab.Shell(lab.Workspace())
output, code = shell.run("pytest")
lines = output.splitlines()
print("pytest printed", len(lines), "lines and exited", code)
worth_having = next(line for line in lines if "assert" in line)
print("the one line worth having:", worth_having)
print()
head, kept_head = harness.truncate_head(output)
tail, kept_tail = harness.truncate_tail(output)
print("truncate_head keeps the first", kept_head, "lines.",
"Is that line in it?", "assert" in head)
print("truncate_tail keeps the last ", kept_tail, "lines.",
"Is that line in it?", "assert" in tail)pytest printed 3000 lines and exited 1 the one line worth having: > assert total([1, 2, 3]) == 6 truncate_head keeps the first 1283 lines. Is that line in it? False truncate_tail keeps the last 1283 lines. Is that line in it? True
A teammate's loop has one line yours does not: if the reply asks for a tool that is not in the list, it skips the call and moves on. Nothing raises, nothing is appended. The model asks for raed, and a minute later the user types one more line. What does that next request carry?
raed is the one that typed it. Raising leaves the same hole in the record, and quietly.400 invalid_request: messages[1]: toolCall c1 has no toolResult. Your run_tool answers an unknown name with Tool raed not found and is_error, so the pair is closed and the model gets to try again. Skipping is the cheapest-looking way to break a transcript, and you will meet it again in lesson 10.
Where this refusal comes from: [general] hosted providers reject a request in which a call has no result, and Tau keeps a function whose whole job is to guarantee that "every tool call has exactly one adjacent result" before a provider sees the list (src/tau_agent/tool_history.py:40-47).
import harness, lab
# The record a loop that skips unknown tools leaves behind. The model asked for `raed`,
# the loop found no such tool, skipped the call, and appended nothing.
asked = lab.call("raed", {"path": "config.py"}, id="c1")
messages = [lab.user("Read config.py."), lab.reply(lab.text("Let me look."), asked)]
print("the record:", lab.shape(messages))
# A minute later the user types one more line, and it goes out on that list.
messages.append(harness.user_message("Well? What is in it?"))
model = lab.ScriptedModel([lab.say("PORT = 9090.")])
reply = model.complete(harness.SYSTEM, harness.context_for_model(messages), [])
print("what the request carried:", lab.shape(harness.context_for_model(messages)))
print("what came back:")
print(lab.show([reply], stop_reason=True))the record: U A[c1] what the request carried: U A[c1] U what came back: assistant -> (nothing) [stop_reason=error] [error: 400 invalid_request: messages[1]: toolCall c1 has no toolResult]
Warm-up: generators in three cells
Today's change to run_agent turns it into a
A function with yield in it. What does calling it do?
# A generator is an ordinary function with `yield` in it. This one is six lines.
def countdown(n):
print(" inside: starting, n =", n)
while n > 0:
yield n
n -= 1
print(" inside: falling off the end")
print("before the call")
counting = countdown(3)
print("after the call, counting is a", type(counting).__name__)
print("still nothing above this line came from inside the function.")
print()
print("next(counting) ->", next(counting))before the call after the call, counting is a generator still nothing above this line came from inside the function. inside: starting, n = 3 next(counting) -> 3
The same function with a line before and a line after the yield. In what order do the two sides print?
# The same generator, with a line before and a line after the `yield`.
def countdown(n):
while n > 0:
print(f" inside: about to yield {n}")
yield n
print(f" inside: woke up again, n is still {n}")
n -= 1
for value in countdown(2):
print(f"consumer: got {value}. Drawing it now, slowly.")
print( "consumer: still drawing.")
print("consumer: the for loop is over.")inside: about to yield 2 consumer: got 2. Drawing it now, slowly. consumer: still drawing. inside: woke up again, n is still 2 inside: about to yield 1 consumer: got 1. Drawing it now, slowly. consumer: still drawing. inside: woke up again, n is still 1 consumer: the for loop is over.
The same function again, with a finally around its loop, and a consumer who walks away after two values.
# The same generator again, with a `finally` around its loop.
def countdown(n):
try:
while n > 0:
yield n
n -= 1
finally:
print(" inside: finally, closing down")
counting = countdown(3)
print("next ->", next(counting))
print("next ->", next(counting))
print("the consumer has had enough and walks away:")
counting.close()
print("closed. One value never came out, and the generator knew it was over.")next -> 3 next -> 2 the consumer has had enough and walks away: inside: finally, closing down closed. One value never came out, and the generator knew it was over.
Three facts to carry out of that: calling a generator function runs none of it; the body stops dead at each yield and only moves when somebody asks for the next value; and close() is how a consumer says it has had enough, which the generator finds out about through finally.
Forty seconds of nothing
Your loop works. It reads files, it survives a dead provider, it keeps a firehose inside a budget. Here it is on a three-file job, and the only thing wrong with it is what a person sees while it goes on.
Your lesson 6 loop, unchanged, on the bug from lesson 3. Watch the first two lines and the gap between them; everything after those is put together from the record once the run is over.
import harness, lab
ws = lab.Workspace({"main.py": "from util import scale\n",
"util.py": "from const import FACTOR\n",
"const.py": "FACTOR = 0\n"})
model = lab.ScriptedModel([
lab.reply(lab.text("Let me look."), lab.call("read", {"path": "main.py"})),
lab.reply(lab.text("It comes from util."), lab.call("read", {"path": "util.py"})),
lab.reply(lab.text("And from const."), lab.call("read", {"path": "const.py"})),
lab.say("FACTOR is 0 in const.py. That is your bug."),
])
messages = []
print("[you press Enter]")
final = harness.run_agent(model, harness.SYSTEM, messages,
[harness.make_read_tool(ws)],
"scale(21) prints 0. Find the bug.")
print("[forty seconds later]", harness.text_of(final))
print()
print("Four model calls and three files, and between those two lines "
"the screen showed nothing.")
print("Here is the whole story, put together from the record once it was all over:")
print(lab.show(messages))[you press Enter]
[forty seconds later] FACTOR is 0 in const.py. That is your bug.
Four model calls and three files, and between those two lines the screen showed nothing.
Here is the whole story, put together from the record once it was all over:
user -> "scale(21) prints 0. Find the bug."
assistant -> "Let me look." + toolCall c1 read({"path": "main.py"})
toolResult c1 -> "from util import scale\n"
assistant -> "It comes from util." + toolCall c2 read({"path": "util.py"})
toolResult c2 -> "from const import FACTOR\n"
assistant -> "And from const." + toolCall c3 read({"path": "const.py"})
toolResult c3 -> "FACTOR = 0\n"
assistant -> "FACTOR is 0 in const.py. That is your bug."
From the outside, a job that takes forty seconds is indistinguishable from a job that has hung, and the transcript printed at the end is no comfort to somebody who spent that minute wondering. So you do the obvious thing, and it is the right thing: put print() inside the loop. Six of them, one for each thing the loop has to say.
print(f"user: {prompt}")
print(f"--- turn {turn} ---")
print(f"stopped: {reply['error_message']}")
print(f"assistant: {text_of(reply)}")
print(f"tool: {call['name']} {json.dumps(call['arguments'])}")
print(f"result: {len(result['content'])} characters" + ", is_error" * result["is_error"])Those six are already in this lesson's starter, marked # given in lesson 7, and you will see them in the lab's diff. The run narrates itself, the user relaxes, and for about a day this is the best code on the team.
Then the day ends. A colleague wants to call your agent from a shell script — agent "fix the test" > answer.txt — and wants the file to hold the answer and nothing else. A second colleague is writing a web dashboard and wants one JSON object per thing that happens, so their code can read it.
The straight way to give them what they want is a flag. Thread a second mode through each of the six print sites with an if, and then a third mode for the dashboard. How many display branches are sitting inside run_agent once all three are supported?
Six sites, three modes: eighteen. The cell below has the first two of the three, so you can see twelve of them. Nothing about the eighteen is hard — that is what makes it worth arguing about. Count what else they do to the file: the loop is now the only place a display can be changed, every new screen needs a commit in the file that also decides when to stop paying a provider, and the next bug in the loop itself has to be found among eighteen branches that have nothing to do with the bug.
Two modes, threaded through the six sites, run on the same small job. It works. If you want the feel of it, add the dashboard's third mode to one site and then to all six.
import json
import harness, lab
def run_agent(model, system, messages, tools, prompt, *,
max_turns=None, json_mode=False):
"""Your loop, with the six progress lines of your starter, and a second mode
threaded through each one of them."""
turn = 0
messages.append(harness.user_message(prompt))
if json_mode: print(json.dumps({"kind": "prompt", "text": prompt}))
else: print(f"user: {prompt}")
while True:
turn += 1
if json_mode: print(json.dumps({"kind": "turn", "n": turn}))
else: print(f"--- turn {turn} ---")
if max_turns is not None and turn > max_turns:
reply = harness.error_message(f"Agent stopped after max_turns={max_turns}")
else:
reply = model.complete(system, harness.context_for_model(messages),
harness.tool_specs(tools))
messages.append(reply)
if reply["stop_reason"] == "error":
if json_mode: print(json.dumps({"kind": "stopped",
"why": reply["error_message"]}))
else: print(f"stopped: {reply['error_message']}")
return reply
if harness.text_of(reply):
if json_mode: print(json.dumps({"kind": "text",
"text": harness.text_of(reply)}))
else: print(f"assistant: {harness.text_of(reply)}")
calls = harness.tool_calls(reply)
if not calls:
return reply
for call in calls:
if json_mode: print(json.dumps({"kind": "tool", "name": call["name"],
"arguments": call["arguments"]}))
else: print(f"tool: {call['name']} {json.dumps(call['arguments'])}")
result = harness.run_tool(tools, call)
messages.append(result)
if json_mode: print(json.dumps({"kind": "result",
"chars": len(result["content"]),
"is_error": result["is_error"]}))
else: print(f"result: {len(result['content'])} characters"
+ ", is_error" * result["is_error"])
def job():
ws = lab.Workspace({"config.py": "PORT = 9090\n"})
model = lab.ScriptedModel([lab.reply(lab.text("Let me look."),
lab.call("read", {"path": "config.py"})),
lab.say("The port is 9090.")])
return model, [], [harness.make_read_tool(ws)]
model, messages, tools = job()
print("### the terminal wants this")
run_agent(model, harness.SYSTEM, messages, tools, "Find the port.")
print()
model, messages, tools = job()
print("### the other program wants this")
run_agent(model, harness.SYSTEM, messages, tools, "Find the port.", json_mode=True)### the terminal wants this
user: Find the port.
--- turn 1 ---
assistant: Let me look.
tool: read {"path": "config.py"}
result: 12 characters
--- turn 2 ---
assistant: The port is 9090.
### the other program wants this
{"kind": "prompt", "text": "Find the port."}
{"kind": "turn", "n": 1}
{"kind": "text", "text": "Let me look."}
{"kind": "tool", "name": "read", "arguments": {"path": "config.py"}}
{"kind": "result", "chars": 12, "is_error": false}
{"kind": "turn", "n": 2}
{"kind": "text", "text": "The port is 9090."}
What the loop actually knows
Look at the JSON half of that output. Every line is one thing that happened, with its pieces separate: a name, a path, a number of characters. The text half threw all of that away and left a sentence. So the loop already has, at each of those six points, exactly what any of the three screens needs — and it is spending its own lines deciding how to spell it.
The next change is small and strange. Every print becomes a yield of the dict it was formatting, and the return at the end becomes a yield too.
Your caller has not changed since lesson 3: final = run_agent(model, SYSTEM, messages, tools, "Find the port."), and then it uses final. What is final now, and where does the run's last message come from?
final catches
final is bound the moment the call returns, and at that moment no model has been spoken to.final is a generator and not one line of the loop has run. Iterate it, and the last thing it hands over carries the messages this run added, with the final one last
final is a generator, but the work is already done: Python ran the body on the call and stored the twelve things for the caller to walk through
model.calls: it is 0 after the call and 2 after somebody iterates.Zero model calls and an empty record until somebody iterates; then two model calls, four messages, and twelve things handed over one at a time. Each one is an type saying what just happened and nothing about how to draw it. The last event, agent_end, carries this run's messages, so the final message is the last of those — the return value has not disappeared, it has become one field of the last thing the loop says.
import harness, lab
ws = lab.Workspace({"config.py": "PORT = 9090\n"})
model = lab.ScriptedModel([lab.reply(lab.text("Let me look."),
lab.call("read", {"path": "config.py"})),
lab.say("The port is 9090.")])
messages = []
# The caller you have written since lesson 3, not changed by one character.
final = harness.run_agent(model, harness.SYSTEM, messages,
[harness.make_read_tool(ws)], "Find the port.")
print("final is a", type(final).__name__)
print("model calls so far:", len(model.calls))
print("messages on the record:", len(messages))
print()
print("now somebody iterates it:")
events = [event for event in final]
print(" events:", len(events), "-", " ".join(event["type"] for event in events))
print(" model calls now:", len(model.calls))
print(" messages on the record now:", len(messages))
print()
last = events[-1]
print("the last event is", last["type"], "and it carries",
len(last["messages"]), "messages:")
print(lab.show(last["messages"]))final is a generator
model calls so far: 0
messages on the record: 0
now somebody iterates it:
events: 12 - agent_start turn_start message_end message_end tool_execution_start message_end tool_execution_end turn_end turn_start message_end turn_end agent_end
model calls now: 2
messages on the record now: 4
the last event is agent_end and it carries 4 messages:
user -> "Find the port."
assistant -> "Let me look." + toolCall c1 read({"path": "config.py"})
toolResult c1 -> "PORT = 9090\n"
assistant -> "The port is 9090."
Which raises a fair question about who is in charge. The dashboard's code takes a couple of hundred milliseconds to draw each line it is given. That is a long time to keep a loop waiting.
The code doing the drawing takes 200 ms per event. What is run_agent doing during those 200 ms? The cell stands in for the drawing with a busy loop, and counts the model calls and the messages on the record on both sides of it.
yield that handed that event over, and it will not move again until somebody asks for the next one
tool_execution_start the tool is running; on message_end there is nothing to do, so it waits
Every row has the same numbers on both sides of the arrow. Work happens between the consumer's turns, never during them, which is why the [the read tool is running, right now] line sits on a line of its own. Whoever iterates the run sets its pace; call them the
import harness, lab
ws = lab.Workspace({"config.py": "PORT = 9090\n"})
read = harness.make_read_tool(ws)
inner = read["execute"]
def noisy(arguments):
print(" [the read tool is running, right now]")
return inner(arguments)
read = {**read, "execute": noisy}
model = lab.ScriptedModel([lab.reply(lab.call("read", {"path": "config.py"})),
lab.say("The port is 9090.")])
messages = []
print("event model calls / messages, "
"measured before and after drawing it")
for event in harness.run_agent(model, harness.SYSTEM, messages, [read],
"Find the port."):
before = f"{len(model.calls)}/{len(messages)}"
# the drawing: real work, taking real time, in place of the 200 ms
for _ in range(200_000):
pass
after = f"{len(model.calls)}/{len(messages)}"
print(f"{event['type']:<22} {before} -> {after}")event model calls / messages, measured before and after drawing it
agent_start 0/0 -> 0/0
turn_start 0/0 -> 0/0
message_end 0/1 -> 0/1
message_end 1/2 -> 1/2
tool_execution_start 1/2 -> 1/2
[the read tool is running, right now]
message_end 1/3 -> 1/3
tool_execution_end 1/3 -> 1/3
turn_end 1/3 -> 1/3
turn_start 1/3 -> 1/3
message_end 2/4 -> 2/4
turn_end 2/4 -> 2/4
agent_end 2/4 -> 2/4
Figure 7.1 One event per next(). The lower half is the same run as two lanes: while the consumer draws, the loop's lane is empty, and while the loop works, the consumer's is.
Every way out
A dashboard that opens a spinner when a turn starts has to close it when the turn ends. Lesson 5 gave your loop more than one way to leave.
Here are four things that can happen inside a turn. Tap each one that must still hand over a turn_end and an agent_end on its way out.
- The reply asks for no tools: the job is done
max_turnsis reached and the model is never asked again- The provider answers 500 and the reply comes back failed
- A tool raises
FileNotFoundErrorhalf way through the turn
Three exits and one impostor. The tool that raises is not an exit at all: since lesson 4 it becomes a result with is_error set, the model is told, and the turn carries on — so it does not need its own ending because it never left. The other three do, and they are the three the lab's tests pin down: the ordinary answer, the limit, and the failed reply.
An exit that skips turn_end costs you a spinner that never stops. An exit that skips agent_end costs you the run's messages, since nothing else carries them. Lesson 5's answer pays off here: the limit and the failed reply already leave your loop by the same two lines, so there are two doors to close and not three.
Where this comes from: Tau's loop writes the same closing pair out three times, once per early return, rather than letting any exit skip it (src/tau_agent/loop.py:83-91, src/tau_agent/loop.py:112-120, src/tau_agent/loop.py:141-151).
Which one is the answer?
Back to the shell script. agent "find the port" > answer.txt, and the file is meant to hold one thing.
A tool-using run produces three assistant messages: "Let me look.", "One more file." and "The port is 9090." Which of the three may be in answer.txt when the run ends, and how would the code writing that file know?
In the second run the message with no tool calls is the max_turns stop: empty, stop_reason error, and not an answer to anything. A rule that reads each message as it arrives cannot tell the two runs apart, because at the moment "Still looking." arrives it looks exactly like "Let me look.". So whatever writes answer.txt needs two moments, not one: something for each event, and something once, at the end, which also knows whether the run went well enough to exit zero.
import harness, lab
ws = lab.Workspace({"config.py": "PORT = 9090\n", "backup.py": "PORT = 8080\n"})
def watch(title, model, **options):
"""Run the job and write down every assistant message as it arrives, the
way a frontend that prints on the spot would."""
print(title)
messages, printed = [], []
for event in harness.run_agent(model, harness.SYSTEM, messages,
[harness.make_read_tool(ws)],
"Find the port.", **options):
if event["type"] == "message_end" and event["message"]["role"] == "assistant":
reply = event["message"]
printed.append(harness.text_of(reply) or "(nothing)")
print(f" assistant {len(printed)}: {printed[-1]!r:<22} "
f"calls: {len(harness.tool_calls(reply))} "
f"stop_reason: {reply['stop_reason']}")
print(" answer.txt, from a frontend that prints each one as it arrives:")
for line in printed:
print(" " + line)
print()
watch("### the model finds it",
lab.ScriptedModel([lab.reply(lab.text("Let me look."),
lab.call("read", {"path": "config.py"})),
lab.reply(lab.text("One more file."),
lab.call("read", {"path": "backup.py"})),
lab.say("The port is 9090.")]))
watch("### the same job, and the model never stops asking (max_turns=2)",
lab.ScriptedModel([lab.forever(
lab.reply(lab.text("Still looking."),
lab.call("read", {"path": "config.py"})))]),
max_turns=2)### the model finds it
assistant 1: 'Let me look.' calls: 1 stop_reason: toolUse
assistant 2: 'One more file.' calls: 1 stop_reason: toolUse
assistant 3: 'The port is 9090.' calls: 0 stop_reason: stop
answer.txt, from a frontend that prints each one as it arrives:
Let me look.
One more file.
The port is 9090.
### the same job, and the model never stops asking (max_turns=2)
assistant 1: 'Still looking.' calls: 1 stop_reason: toolUse
assistant 2: 'Still looking.' calls: 1 stop_reason: toolUse
assistant 3: '(nothing)' calls: 0 stop_reason: error
answer.txt, from a frontend that prints each one as it arrives:
Still looking.
Still looking.
(nothing)
Append, or announce?
One question left, and it is the one that decides the shape of the code. A tool has just returned. Two things have to happen: the result goes on the record, and the result is announced. They cannot both be first.
Put one tool call in the order you want it to happen, from the moment the loop reaches the call.
tool_execution_startgoes out, carrying the call- the tool runs and returns
PORT = 9090 - the result is appended to
messages, the caller's list - a
message_endgoes out, carrying that result tool_execution_endgoes out, carrying the call and the result
The two that matter are the middle pair. Announce first and append second, and there is a moment — one yield wide — at which a consumer has been told the result is on the record while the record does not have it. Since lesson 3 the caller's list is the product of a run, so at that moment the consumer's picture and the truth disagree, and a consumer that walks away right there keeps the wrong one.
Worse, put any yield between the tool returning and the result being appended, and a consumer can end a run in which a file was really written, a command was really executed, and the record says nothing about it. The model will never hear of that work. So:
Fifteen runs of the same two-tool job. Run k listens to k events and then closes the run. Two columns to read: the record the consumer walked away from, and whether every tool that ran left a result behind.
import harness, lab
FILES = {"config.py": "PORT = 9090\n", "backup.py": "PORT = 8080\n"}
def stop_after(k):
"""Listen to k events of one run, then close it, and see what everyone believes."""
ran = []
ws = lab.Workspace(FILES)
read = harness.make_read_tool(ws)
inner = read["execute"]
read = {**read, "execute": lambda arguments: (ran.append(arguments["path"]),
inner(arguments))[1]}
model = lab.ScriptedModel([lab.reply(lab.call("read", {"path": "config.py"}),
lab.call("read", {"path": "backup.py"})),
lab.say("They differ.")])
messages, heard = [], []
run = harness.run_agent(model, harness.SYSTEM, messages, [read],
"Compare the two ports.")
for event in run:
heard.append(event)
if len(heard) == k:
run.close()
break
recorded = [m for m in messages if m["role"] == "toolResult"]
return heard[-1]["type"], lab.shape(messages) or "(empty)", len(ran), len(recorded)
print(" k the consumer's last word the record it walked away from "
"tools run results recorded")
for k in range(1, 16):
kind, shape, ran, recorded = stop_after(k)
print(f"{k:>2} {kind:<24} {shape:<29} {ran:^9} {recorded:^17}".rstrip())k the consumer's last word the record it walked away from tools run results recorded 1 agent_start (empty) 0 0 2 turn_start (empty) 0 0 3 message_end U 0 0 4 message_end U A[c1,c2] 0 0 5 tool_execution_start U A[c1,c2] 0 0 6 message_end U A[c1,c2] R(c1) 1 1 7 tool_execution_end U A[c1,c2] R(c1) 1 1 8 tool_execution_start U A[c1,c2] R(c1) 1 1 9 message_end U A[c1,c2] R(c1) R(c2) 2 2 10 tool_execution_end U A[c1,c2] R(c1) R(c2) 2 2 11 turn_end U A[c1,c2] R(c1) R(c2) 2 2 12 turn_start U A[c1,c2] R(c1) R(c2) 2 2 13 message_end U A[c1,c2] R(c1) R(c2) A 2 2 14 turn_end U A[c1,c2] R(c1) R(c2) A 2 2 15 agent_end U A[c1,c2] R(c1) R(c2) A 2 2
Rows 6 and 9 are the ones to look at: the moment a result is announced, it is already on the record, and the tool and result columns never disagree at any k. That is not luck, and it is not something the tests can give you — it is what the order you just chose buys, at every point in the run.
Build: say what happened
Here is the whole design, in one sentence. The loop reports what happened as plain-data events, in balanced pairs on every way out, always after the record has changed — and a render(event) for each one, then finish(), which says whether the run went well.
Seven kinds of event, and they nest: the run holds turns, a turn holds messages, a tool call holds its result.
Figure 7.2 One tool-using run, event by event, beside the list it is describing. Every event is message-granular: the smallest thing it reports is a whole message.
agent_startandagent_endwrap the run.agent_endcarriesmessages— the ones this run added, not the caller's whole list.- Inside, one
turn_startandturn_endper model call. Two model calls in this run, so two turns, the second opening as soon as the first has closed. - One
message_endfor every message appended, each tied to its slip in the list: the prompt (inside the first turn), the reply, the tool result, the second reply. By the time an event goes out, its message is already there. tool_execution_startandtool_execution_endwrap one call, and the result's ownmessage_endsits between them: recorded, then announced, then the execution is over.
What these events do not have is anything smaller than a message. [general] A real reply arrives a fragment at a time, and a screen that wanted to show words appearing as the model writes them would need those fragments. They are not here, and the page will not pretend otherwise: these events are message-granular, the whole reply arrives at once, and the fragments are lesson 16's, where they come from — a provider's stream.
Two gaps. The first is run_agent's body, which today is lesson 6's loop with the six given print calls in it. Rewrite it as a generator: every print becomes a yield, and so does the return. About twenty-five lines, most of them moved rather than written.
- Seven event types, each a plain dict:
agent_start;turn_start;message_endwithmessage;tool_execution_startwithcall;tool_execution_endwithcallandresult;turn_endwithmessage, the turn's reply;agent_endwithmessages. The starter lists them again, above your gap. - One
message_endfor every message appended — the prompt, each reply, a failed reply, themax_turnsstop, every tool result. The prompt is appended inside the first turn, after itsturn_start. - Mutate, then announce, with nothing in between: a result's
message_endcomes aftertool_execution_startand beforetool_execution_end. - Balanced on every exit. The three ways a run can end — the answer, the limit, the failed reply — all close their turn and then the run.
agent_end'smessagesis what this run appended. Run twice on one list and the secondagent_endholds only the second run's four messages.- Plain data: dicts, lists, strings, numbers, no tool dicts and no functions, because one frontend writes every event as a line of JSON.
run_agentitself prints nothing. Not one line.
The second gap is FinalTextRenderer, the frontend for > answer.txt. The class line and its docstring are given; write __init__, render and finish. Nothing is printed while the run goes on. finish() prints the text of the last assistant message and returns True; if that message failed, it prints Error: and the reason instead and returns False, so the shell script can exit non-zero. JsonRenderer is given above it as the worked example: read it first, and note where its one piece of state lives.
Twelve hidden tests. They compare your events against the exact sequence for a text-only run, a one-tool run, two calls in one reply, a max_turns=2 stop and a provider failure; they close a run after every possible number of events and check the record at each one; they run twice on one list and read what the second agent_end carries; they feed every event of two runs through JsonRenderer; they listen for anything the loop prints; they check that answer.txt ends up holding the last answer, and that a failed run says why instead; and they run two renderers over two different runs at once.
- Go through the six
printcalls one at a time and ask, for each: which line of the loop appends the thing this print is talking about, and does the print sit above it or below it? Every one of them is already below its append, so the order you chose is the order you have. What the six are not is seven events: one of them describes something that is never appended at all, two of them describe the same message, and one fires only when the reply has words in it — while amessage_endis owed for every message appended. - The shape does not change much. One
while True;turn_startis the first thing in each pass; on the first pass only, the prompt is recorded; then themax_turnscheck and the model call exactly as they are now; then the reply; then the calls, if the reply did not fail; thenturn_end; then the one way out. Two things will bite. A nested helper cannotyieldon behalf of the function that called it, so if you want one place that appends a message and makes its announcement, have that helper return the event and let the loop writeyield helper(...). Andagent_endneeds this run's messages alone, which the caller's list does not hand you, because it may already have had messages in it when you were called: you need either a second list of your own or the length it had when you started. - In outline. Keep a second list,
new, empty at the start; a helper that appends its message to bothmessagesandnewand returns themessage_enddict for it. Yieldagent_start. Then loop: count the turn, yieldturn_start; on turn one, yield the helper's event for the prompt; work out the reply exactly as lesson 5 does; yield the helper's event for it; the calls are none if the reply failed, otherwisetool_calls(reply); for each call, yield the start event, run the tool, yield the helper's event for the result, yield the end event; yieldturn_endcarrying the reply; and if there were no calls, leave the loop. After the loop, yieldagent_endcarryingnew. ForFinalTextRenderer: one attribute, set in__init__, holding the newest assistant message seen —renderonly ever looks atmessage_endevents whose message is an assistant one — andfinishis the only method that prints.
The loop no longer knows what a screen is. Look at what the tests just proved for you: every exit closed its turn and then the run, the record was already true at each of the fifteen points a consumer could have walked away from a two-tool run, and your loop printed nothing at all while proving it.
What changed since lesson 06
The line-by-line diff needs JavaScript. The whole file this exercise starts from is printed at the end of it.
- A run with no tools yields agent_start, turn_start, a message_end for the prompt, one for the answer, turn_end, agent_end.
- One tool call: its result's message_end comes after tool_execution_start and before tool_execution_end, inside the first turn.
- Two calls in one reply: start, result, end for the first call, then start, result, end for the second, in one turn.
- max_turns=2 against a stuck model: the stop is a third turn with its own turn_start, message_end and turn_end, then agent_end.
- The provider answers 500: the failed reply gets its message_end, then turn_end and agent_end still come.
- Stop the run after any number of events: the message just announced is already the last of the caller's list, and every tool that ran has its result there.
- A second run on the same list: its agent_end holds the messages that run appended, and no earlier ones.
- Every event of a tool run and of a failed run can be written as a line of JSON by the given JsonRenderer.
- run_agent itself prints nothing, on a tool run, a max_turns stop or a provider failure: what a run looks like is a frontend's business.
- FinalTextRenderer prints nothing during the run; finish() prints the last assistant text, once, and returns True.
- After a provider failure or a max_turns stop, finish() returns False, says why, and prints no earlier text as if it were the answer.
- Two FinalTextRenderer objects fed two different runs each print their own run's answer.
One run of a three-turn job, and three frontends over it. The first is five statements written here in the cell and drawn while the run happens, as a terminal would; the second is the given JsonRenderer; the third is yours. No tests: read the three shapes of the same twenty-one events. Once the lab has passed, this runs against your code.
import harness, lab
class Terminal:
"""A whole frontend in five statements: a line for each turn, a line for
each tool that came back, and nothing else. It draws while the run happens;
the answer is frontend 3's."""
def __init__(self):
self.turn = 0
def render(self, event):
if event["type"] == "turn_start":
self.turn += 1
print(f" turn {self.turn}")
elif event["type"] == "tool_execution_end":
print(f" {event['call']['name']}({event['call']['arguments']['path']}) "
f"-> {len(event['result']['content'])} characters")
def finish(self):
return True
def job():
ws = lab.Workspace({"config.py": "PORT = 9090\n", "backup.py": "PORT = 8080\n",
"README.md": "The service listens on the port in config.py.\n"})
model = lab.ScriptedModel([
lab.reply(lab.text("Two files to compare."),
lab.call("read", {"path": "config.py"}),
lab.call("read", {"path": "backup.py"})),
lab.reply(lab.text("And the README, to see which one counts."),
lab.call("read", {"path": "README.md"})),
lab.say("config.py is the live one: port 9090. backup.py still says 8080."),
])
return model, [], [harness.make_read_tool(ws)]
print("### frontend 1: a terminal display, drawn while the run happens")
model, messages, tools = job()
terminal, events = Terminal(), []
for event in harness.run_agent(model, harness.SYSTEM, messages, tools,
"Which port is live?"):
events.append(event)
terminal.render(event)
print(" ok:", terminal.finish())
print()
print("### frontend 2: the same", len(events),
"events, as JSON lines for another program")
json_lines = harness.JsonRenderer()
for event in events:
json_lines.render(event)
print(" ok:", json_lines.finish())
print()
print("### frontend 3: the same events again, as `agent > answer.txt`")
answer = harness.FinalTextRenderer()
for event in events:
answer.render(event)
print(" ok:", answer.finish())### frontend 1: a terminal display, drawn while the run happens
turn 1
read(config.py) -> 12 characters
read(backup.py) -> 12 characters
turn 2
read(README.md) -> 46 characters
turn 3
ok: True
### frontend 2: the same 21 events, as JSON lines for another program
{"type": "agent_start"}
{"type": "turn_start"}
{"type": "message_end", "message": {"role": "user", "content": "Which port is live?"}}
{"type": "message_end", "message": {"role": "assistant", "content": [{"type": "text", "text": "Two files to compare."}, {"type": "toolCall", "id": "c1", "name": "read", "arguments": {"path": "config.py"}}, {"type": "toolCall", "id": "c2", "name": "read", "arguments": {"path": "backup.py"}}], "stop_reason": "toolUse", "usage": {"input": 134, "cache_read": 0, "output": 56}}}
{"type": "tool_execution_start", "call": {"type": "toolCall", "id": "c1", "name": "read", "arguments": {"path": "config.py"}}}
{"type": "message_end", "message": {"role": "toolResult", "tool_call_id": "c1", "tool_name": "read", "content": "PORT = 9090\n", "is_error": false}}
{"type": "tool_execution_end", "call": {"type": "toolCall", "id": "c1", "name": "read", "arguments": {"path": "config.py"}}, "result": {"role": "toolResult", "tool_call_id": "c1", "tool_name": "read", "content": "PORT = 9090\n", "is_error": false}}
{"type": "tool_execution_start", "call": {"type": "toolCall", "id": "c2", "name": "read", "arguments": {"path": "backup.py"}}}
{"type": "message_end", "message": {"role": "toolResult", "tool_call_id": "c2", "tool_name": "read", "content": "PORT = 8080\n", "is_error": false}}
{"type": "tool_execution_end", "call": {"type": "toolCall", "id": "c2", "name": "read", "arguments": {"path": "backup.py"}}, "result": {"role": "toolResult", "tool_call_id": "c2", "tool_name": "read", "content": "PORT = 8080\n", "is_error": false}}
{"type": "turn_end", "message": {"role": "assistant", "content": [{"type": "text", "text": "Two files to compare."}, {"type": "toolCall", "id": "c1", "name": "read", "arguments": {"path": "config.py"}}, {"type": "toolCall", "id": "c2", "name": "read", "arguments": {"path": "backup.py"}}], "stop_reason": "toolUse", "usage": {"input": 134, "cache_read": 0, "output": 56}}}
{"type": "turn_start"}
{"type": "message_end", "message": {"role": "assistant", "content": [{"type": "text", "text": "And the README, to see which one counts."}, {"type": "toolCall", "id": "c3", "name": "read", "arguments": {"path": "README.md"}}], "stop_reason": "toolUse", "usage": {"input": 255, "cache_read": 134, "output": 39}}}
{"type": "tool_execution_start", "call": {"type": "toolCall", "id": "c3", "name": "read", "arguments": {"path": "README.md"}}}
{"type": "message_end", "message": {"role": "toolResult", "tool_call_id": "c3", "tool_name": "read", "content": "The service listens on the port in config.py.\n", "is_error": false}}
{"type": "tool_execution_end", "call": {"type": "toolCall", "id": "c3", "name": "read", "arguments": {"path": "README.md"}}, "result": {"role": "toolResult", "tool_call_id": "c3", "tool_name": "read", "content": "The service listens on the port in config.py.\n", "is_error": false}}
{"type": "turn_end", "message": {"role": "assistant", "content": [{"type": "text", "text": "And the README, to see which one counts."}, {"type": "toolCall", "id": "c3", "name": "read", "arguments": {"path": "README.md"}}], "stop_reason": "toolUse", "usage": {"input": 255, "cache_read": 134, "output": 39}}}
{"type": "turn_start"}
{"type": "message_end", "message": {"role": "assistant", "content": [{"type": "text", "text": "config.py is the live one: port 9090. backup.py still says 8080."}], "stop_reason": "stop", "usage": {"input": 339, "cache_read": 255, "output": 24}}}
{"type": "turn_end", "message": {"role": "assistant", "content": [{"type": "text", "text": "config.py is the live one: port 9090. backup.py still says 8080."}], "stop_reason": "stop", "usage": {"input": 339, "cache_read": 255, "output": 24}}}
{"type": "agent_end", "messages": [{"role": "user", "content": "Which port is live?"}, {"role": "assistant", "content": [{"type": "text", "text": "Two files to compare."}, {"type": "toolCall", "id": "c1", "name": "read", "arguments": {"path": "config.py"}}, {"type": "toolCall", "id": "c2", "name": "read", "arguments": {"path": "backup.py"}}], "stop_reason": "toolUse", "usage": {"input": 134, "cache_read": 0, "output": 56}}, {"role": "toolResult", "tool_call_id": "c1", "tool_name": "read", "content": "PORT = 9090\n", "is_error": false}, {"role": "toolResult", "tool_call_id": "c2", "tool_name": "read", "content": "PORT = 8080\n", "is_error": false}, {"role": "assistant", "content": [{"type": "text", "text": "And the README, to see which one counts."}, {"type": "toolCall", "id": "c3", "name": "read", "arguments": {"path": "README.md"}}], "stop_reason": "toolUse", "usage": {"input": 255, "cache_read": 134, "output": 39}}, {"role": "toolResult", "tool_call_id": "c3", "tool_name": "read", "content": "The service listens on the port in config.py.\n", "is_error": false}, {"role": "assistant", "content": [{"type": "text", "text": "config.py is the live one: port 9090. backup.py still says 8080."}], "stop_reason": "stop", "usage": {"input": 339, "cache_read": 255, "output": 24}}]}
ok: True
### frontend 3: the same events again, as `agent > answer.txt`
config.py is the live one: port 9090. backup.py still says 8080.
ok: True
One loop, three screens, and the loop was not asked about any of them. The third colleague can write the fourth this afternoon without opening harness.py, which is the whole return on today's thirty-five lines.
- Hold a conversation: it knows what was said, and who said it.
- Run a tool the model asks for and show it the result.
- Keep going until the model stops asking.
- Tell the model when a tool fails, and carry on.
- Stop a runaway, and survive a provider failure.
- Keep every tool result within a budget.
- Report what it is doing, as events, to any frontend.
Say it in your own words
Your loop used to print. Now it does not, and three programs show more than it ever did. In a sentence or two: what did you actually move, and what did you refuse to decide?
Compare one of the six print lines with the event that replaced it. What is in both, and what is in only one of them?
The facts stayed in the loop, because only the loop has them. The wording left, because the loop was the last place that could sensibly choose it: it cannot know whether it is talking to a terminal, a pipe, a browser or a test. So it stopped choosing, and the choice went to whoever is watching — who can, and who is allowed to be wrong about it without touching the code that spends money.
Common answers, and what each one misses
- "I moved the printing out of the loop." Half of it. The printing could have moved into a callback the loop calls, and you would still be handing the loop a display to hold. What moved is the decision; what stays behind is a statement of fact that nobody has to draw at all.
- "I made the loop lazy, so the caller controls it." True, and it is a consequence rather than the point: the laziness came free with
yield. A loop that collected the same events into a list and returned it would still be reporting instead of drawing — and it would still leave the user staring at nothing for forty seconds. - "I added an events API." Look at what an event is: a dict with a
type. There is no class, no registry, no subscribe call, and nothing to import. The design is a promise about order and completeness, and it costs one line per fact.
Yield what happened; let someone else decide what it looks like.
Tau's loop is an async generator of events with the same names and the same nesting as yours, and its frontends are the same two methods.
yield {"type": "message_end", "message": message}
yield {"type": "tool_execution_end", "call": call, "result": result}class AgentStartEvent(WireModel):
type: Literal["agent_start"] = "agent_start"
...
class MessageEndEvent(WireModel):
type: Literal["message_end"] = "message_end"
message: AgentMessage
...
class ToolExecutionEndEvent(WireModel):
type: Literal["tool_execution_end"] = "tool_execution_end"
tool_call_id: str
tool_name: str
result: AgentToolResult
is_error: boolTau is async: read async for as for and await f(x) as f(x) until lesson 15. This is the first block where that matters, and it will stand at the top of every one from here on.
The same shapes.
- Ten event types where you have seven; the extra three are
message_start,message_updateand a tool progress update. They are one union withtypeas the discriminator (src/tau_agent/events.py:75-87), which is what yourevent["type"]is in a language with types. - The nesting is yours, four levels deep: the run, the turn, the message, the tool call.
- The canonical order is asserted as a literal list in Tau's own test, exactly the way this lesson's tests do it (
tests/test_agent_loop.py:73-82) — and three lines later the same test asserts on the caller's list (tests/test_agent_loop.py:85), because there too the list is the product. - The prompt is announced inside the first turn, after
turn_start, as yours is (src/tau_agent/loop.py:74-81). Tau puts it on the record earlier still, beforeagent_start(src/tau_agent/loop.py:70-72), which is the same rule as yours — record first — taken further. - A frontend is
render(event)thenfinish() -> bool, written down as a protocol (src/tau_coding/rendering/base.py:20-27). Print mode's entire run is three lines — iterate, render, finish (src/tau_coding/cli.py:1363-1365) — and the bool is the process's exit status. - Tau's
FinalTextRendererkeeps the newest assistant text, prints infinish, and answersFalseafter a failure (src/tau_coding/rendering/plain.py:16-38). Tau's JSON mode draws each event in one line: dump it (src/tau_coding/rendering/json.py:23), which is yourJsonRendererwith a real serialiser behind it.
Where Tau does the opposite, and what that costs it. Your rule is mutate, then announce. Tau announces an assistant message and appends it afterwards (src/tau_agent/loop.py:146-147: the message_end for that reply was yielded further up, inside the provider sub-generator), and it yields tool_execution_end before the result's message events rather than after (src/tau_agent/loop.py:325-340), and a tool result reaches the consumer one line before it reaches the record (src/tau_agent/loop.py:164-170). So the windows your l07-c3 cell showed to be empty are, in Tau, not empty: a consumer can stop between a message being announced and being recorded, and a tool that really ran can leave nothing behind. Tau lives with that because it has a repair that runs before every request and fixes a transcript with a hole in it — which is lesson 10, and where this comes back. There is one place where the difference is forced rather than chosen: the sub-generator that streams a reply cannot hand its final message back through a return, so the loop fishes the message out of the last event it saw, and says so in a comment (src/tau_agent/loop.py:122-139).
What Tau adds. Events are typed models, not dicts, so an event that forgets a field fails where it is built rather than where it is drawn. turn_end also carries the turn's tool results, saving a consumer from collecting them (src/tau_agent/events.py:28-31). The loop counts only the time it spends waiting for the provider's next piece, and the clock is stopped while anyone downstream is drawing, so a slow consumer cannot make a provider look slow (src/tau_agent/loop.py:225-243) — a number you cannot compute from your events at all. And message_start and message_update carry a deep copy of the half-written reply after each delta (src/tau_ai/stream.py:40-41), which is what a screen needs to show words as they arrive. That is lesson 16, in the only place it can honestly come from.
Declared in Tau, and nothing Tau ships sends one. There is a tenth event, tool_execution_update, for a tool reporting progress while it runs (src/tau_agent/events.py:59-65). Two screens are ready to draw it (src/tau_coding/rendering/transcript.py:47-51, src/tau_coding/tui/adapter.py:129-131), and nothing puts one in front of them: the adapter that turns every built-in coding tool into a loop tool deletes the progress callback unused (src/tau_coding/tools.py:123), so the only callers are a test and whatever extension somebody writes. Even then the loop holds a tool's updates back until the tool has returned (src/tau_agent/loop.py:313-320), so they cannot arrive while there is still something to watch. Your seven events have no such passenger, and adding an eighth before something sends it would be the same mistake.
Where yours is weaker. Message-granular only, as the figure said, and nothing in your events says how long anything took. And you have exactly one consumer: whichever for loop is iterating. The day you want the display and something that saves every message to see the same run, one for loop will not do — lesson 11. Also worth knowing before you read Tau's repository: dev-notes/architecture/phase-3-agent-loop.md still draws the event stream with a message_delta in it (dev-notes/architecture/phase-3-agent-loop.md:64-73) and still says a provider error becomes an ErrorEvent (dev-notes/architecture/phase-3-agent-loop.md:134-139). Neither type is in events.py: the deltas are message_update, and a failed request arrives as a message_end carrying an assistant message whose stop_reason is error (src/tau_agent/loop.py:255-262) — the shape lesson 5 made you handle. Notes go stale; events.py is the answer.
One question about Tau's own machinery. It is about code you have not written, so nothing is locked and nothing waits on it.
Tau's turn_end carries the turn's tool results as well as its assistant message. Your turn_end carries the message alone. What can a frontend do with Tau's that it cannot do with yours?
message_end, and once inside tool_execution_end. What Tau's field saves is the collecting, not the knowing.turn_end comes after the tool loop in Tau too, so by then every result has been appended. An event carrying a copy of something never makes it arrive earlier.A field like this is worth having when it saves every consumer the same bookkeeping, and worth refusing when it lets two places disagree about the same fact. Tau's copy cannot disagree, because it is the same objects (src/tau_agent/loop.py:165-172). Yours would be the same objects too. The reason not to add it is smaller and duller: nothing on this course needs it yet.
src/tau_agent/events.py:15-87 · pinned to commit 9fe6a71 · view on GitHub
One more case
A fourth colleague wants a progress bar that reads tool 2 of 5 while a turn is working through a reply that asked for five files. In a sentence or two: which of your seven events give them enough to draw that, and what has to change in run_agent?
Where does the number 5 come from, and how early can anybody know it? Look at what arrives before the first tool starts.
Nothing changes in run_agent. The 5 is already there: the reply's message_end carries the assistant message, and tool_calls() on it returns five blocks — the same function you wrote in lesson 2, run by the frontend this time instead of by the loop. The 2 is a count of tool_execution_start events since that message. Two of the seven events, four or five lines, and a file nobody has to reopen.
Common answers, and what each one misses
- "Add a
totalfield totool_execution_start." It would work, and it is a field that can be wrong: now two places count the same calls. Anything a frontend can compute from an event it has already been given is a field you do not have to keep true. The rule is not "never add a field" — Tau'sturn_endadds one — it is that a computable field buys convenience and owes correctness. - "It cannot be done: events do not say how many calls a turn has." They do not say it in words. They carry the message, and the message is the thing the number is a fact about. This is the pay-off of
message_endcarrying the whole message rather than a summary of it. - "Count the
tool_execution_startevents and showtool 2with no total." Honest, and worse for the user by exactly the amount that matters: a bar that cannot fill is a spinner with numbers on it. The total was available before the first tool ran.
- You hit
- a run that showed nothing for forty seconds, then six
printcalls, then three programs that each wanted a different shape of the same facts - You built
run_agentas a generator of seven event types, balanced on every exit, announcing only what the record already holds; andFinalTextRenderer, the frontend for> answer.txt- The principle
- yield what happened; let someone else decide what it looks like
- Your harness now
- error_message
- str_arg
- int_arg
- tool_specs
- run_tool
- truncate_head
- truncate_tail
- make_read_tool
- make_write_tool
- make_bash_tool
- run_agent
- context_for_model
- FinalTextRenderer
- JsonRenderer
- Your answers
- Still open
- The transcript is still an argument you hand in and get back through
agent_end. A small web handler callsrun_agent(..., messages=[])once per line and forgets everything; fix that by passing one list, then let a user double-click Send, and two runs append to it at once. Lesson 8.