Input
Real, anonymized conversation logs replayed at scale
Ephemeris.
Today's issue is about measuring what models do — before they ship, after they spoof, and when the bill comes due.
Evaluation · Safety
Test the model in a wind tunnel before it ships.
OpenAI's Deployment Simulation replays real conversation data against a candidate model to predict how it will behave in production — before it reaches a single user. The pitch: surface regressions and safety drift in a sandbox while you can still fix them.
Output
A forecast of behavior and safety, measured pre-launch
Security · Email
Find out who is forging your domain.
Cloudflare's DMARC Management hits general availability: one dashboard for SPF, DKIM, DMARC and BIMI, free to every customer. It audits SPF against the RFC 7208 ten-lookup ceiling and walks you from p=none to p=reject without breaking real mail.
From: billing@your-domain.com · unauthenticated
Dev Tools · Benchmarks
A coding benchmark the best models still fail.
Cognition, the team behind Devin, published FrontierCode — 150 tasks across six languages that grade not just whether code runs but whether it is good: correctness, test quality, scope discipline, house style. On the hardest Diamond tier, the leader clears 13%. Most code agents are well below that.
13.4%
Opus 4.8
6.3%
GPT-5.5
5.2%
Opus 4.7
Diamond tier · 50 tasks · Python, Go, TypeScript, JavaScript, Java, C/C++
Observability · Cost
Nine ways to watch your LLM, ranked by free tier.
PostHog lined up nine AI-observability tools and compared the number that decides adoption for most teams: how much you can trace before you pay. The criteria worth stealing — monthly event caps, seat limits, retention, and whether evals are bundled in.
| Tool | Free events / mo | Seats |
|---|---|---|
| PostHog | 100K | Unlimited |
| Langfuse | 50K | 2 |
| LangSmith | 5K | 1 |
| Arize Phoenix | Self-host | — |
| Braintrust | Evals | Unlimited |
| Datadog LLM | Trial | Tiered |
Infrastructure · Economics
Serving a model is its own engineering problem.
A Stanford lecture with Baseten's Tuhin Srivastava reframes inference as economics: GPU utilization, heterogeneous compute, and the widening gap between open-weight models you host and frontier models you rent. For anyone shipping AI features, unit cost is a design constraint, not a footnote.
$/tok
The lecture's through-line: today's hard infra question is no longer "can we run it" but "what does each token cost to serve."
That's today.
Five picks for 17 June 2026 — a thin day, mined for signal over volume.