Seventy percent, on tasks the model has never seen.
DeepSWE built a coding benchmark from 113 original tasks across 91 repositories in five languages, averaging 668 lines of code per solution. Verifiers test observable behaviour, not implementation. False positives drop to 0.3% versus SWE-Bench Pro's 8.5% — and the model gap widens sharply when training contamination is off the table.
Cap the blast radius.
Anthropic published its containment playbook for Claude. Three patterns, matched to the user's ability to evaluate a bash command. The takeaway for anyone shipping agents: probabilistic defences inside the model are not enough — isolation has to live in the runtime, and the part you wrote is probably the weakest piece.
claude.ai
Ephemeral gVisor containers on isolated servers.
Minimal blast radius. Limited capabilities. Anyone, anywhere.
Claude Code
OS-level sandbox with human-in-the-loop approvals.
For developers who can read a shell command before it runs.
Claude Cowork
Sealed virtual machines, no host bleed.
For non-technical users. Pre-trust failures collapse to the VM.
Filing season as the training set.
The pitch: tax filings are exactly the workload an agent should win at, because every form is a long checklist with deterministic ground truth and a sharp annual deadline that produces fresh labelled data the moment a return is accepted by the IRS.
The interesting part is not that an agent files; it is that the agent improves from filing. The team treats each accepted return as an eval row and the rejections as a debugging signal. The error cases feed back into the prompt library, the tool wrappers, and the next month's evaluation harness, with Codex doing the modifications itself under review.
If you are building a vertical agent, the structural lesson is to design the surrounding pipeline so the task's own outputs become the next iteration's tests. Tax happens to ship them on a calendar.
"Substantive feedback in minutes instead of hours" was the line OpenAI used about Ramp last week. This one is harder: substantive feedback that updates the agent itself.
PostHog will start training on you.
A short notice, easy to miss: from 29 June, PostHog begins training proprietary models on customer session-replay data, to power replay summarisation, synthetic users, and a "self-driving" product surface. The defaults vary by where you sit on the org chart — read the memo.
- To
- Every PostHog admin, in every workspace
- From
- PostHog Inc., 27 May 2026
- Re
- Training on your replays, starting 29 Jun 2026
EU Cloud — opted out by default. No action required.
Contracted (BAA, MSA) — opted out by default.
US Cloud, no contract — opted in by default. Flip the toggle in org settings if you want to stay out.
Targets: session-replay analysis, synthetic-user testing, behaviour prediction. Output goes into the existing PostHog Code surface and what the post calls "more proactive, self-driving products."
You do not need to pick one.
A persistent question gets a clean answer: keep your backend OpenTelemetry collector, drop Sentry's SDK on the frontend, and let OTLP do the bridging. The trace-continuation step is the part to read carefully — it's what makes a click in the browser the parent of an OTel span on the server.
"That trace-continuation step is what lets a Sentry-instrumented browser action become the parent of backend OpenTelemetry work."
Hostage-taking still works on agents.
A roundup of recent adversarial research, surfaced through the channel: AI agents continue to fold under absurd framings — fictional treaties, ransom letters, fabricated authority — because language is polysemous and instructions are not capabilities. The fixes are architectural.
Scope capabilities
The agent should not be able to do the dangerous thing — full stop. No clever prompt can lift a tool it doesn't have.
Isolate credentials
One identity per task, expired on completion. Long-lived tokens are how hostage scenarios become billing events.
Treat the prompt as untrusted input
Including everything the agent reads downstream — pages, files, tool outputs. Especially those.
Skip "do not"
Instructions describe; permissions enforce. Build the second one. The first is documentation.
Stop writing code. Start writing checks.
A useful distillation of takeaways from the Code with Claude conference, reposted through the channel. The thesis: the moat moves out of the source tree and into the harness — verification, evaluation, memory, the small infrastructure that lets a cheap model do an expensive thing.
- 1.
- Pair a cheap model with an expensive advisor. Reserve the smart one for the moments that matter; let it review, not write.
- 2.
- Shift engineering effort into verification — tests, evals, sandboxes. The thing the model can't fake.
- 3.
- Auto-delete code that next month's model will make redundant. Treat the codebase as the cache, not the asset.
- 4.
- Invest in agent memory: durable, queryable, project-scoped. Context is now the bottleneck, not capability.
- 5.
- Optimise for verifiable side effects, not generated tokens. Counting tokens is back to counting punch cards.
- 6.
- Treat your IDE as a workbench for many agents, not a chat window for one. Parallel runs change everything.
- 7.
- Sell the harness, not the wrapper. Anyone can wrap an API; few will build the surrounding plant.
That's Thursday.
A benchmark with no contamination, a containment playbook, a self-improving agent, a privacy notice, a bridge between two observability worlds, a reminder about prompt-as-capability, and a small list of things to stop writing.
Selection rubric — AI tools you could adopt this week · dev tools & agentic coding · privacy & security with high signal · actionable for senior engineers and founders · no marketing fluff
Issue — 040 · published 28 May 2026 from Zürich