Explore intelligence

Recommendation · Money

Just make more things: ship weekly, use projects as a chassis for new models

Anish Acharya's core advice is to just make more things: come up with a project, ship something once a week, and use the project as a chassis to try all the new models, ship things, talk about them, and build your own intuition. He adds that it's better if the project isn't important with a capital I, and to keep investing in it with the model as a tool rather than a goal. He also pushes himself to ship something with every new model that comes out.

Lenny's Podcast · Why companies are becoming a series of loops | Anish Acharya (a16z)

Recommendation from Anish Acharya

Recommendation · Money

Focus on craft and UX as a moat since underlying models are similar

Anish Acharya observes that users can switch between similar AI coding products quickly because the underlying models are not that different — what matters is the harness and UX. He concludes there is major opportunity in user experience as a moat, or at least as a way to buy time to find something durable, and recommends focusing on craft and user experience.

Lenny's Podcast · Why companies are becoming a series of loops | Anish Acharya (a16z)

Recommendation from Anish Acharya

Method · Money

The bug fix loop: from bug report to shipped fix and customer email

Anish Acharya describes a bug fix loop: a bug report comes in, a repro gets generated, a bug fix gets created, the fix gets reviewed, and if it is high risk a human confirms it's okay to ship to prod, while if it is low risk it just gets shipped. Optionally, you email the customer to say 'hey, we fixed the bug you reported.'

Lenny's Podcast · Why companies are becoming a series of loops | Anish Acharya (a16z)

Method described by Anish Acharya

Method preserved for inspection; not a verified protocol.

Recommendation · Money

Treat every business function as an agent loop and unblock it with context

Anish Acharya recommends thinking about every function as an agent loop that goes from input to impact. Your job is to figure out where it gets blocked or goes wrong and give it more context, insight, and direction. When an agent fails, identify whether it's a knowledge gap or a data gap, supply the missing context or data, and the next time it shouldn't have to call you.

Lenny's Podcast · Why companies are becoming a series of loops | Anish Acharya (a16z)

Recommendation from Anish Acharya

Recommendation · Money

Fix for mistake 1: empower people early by writing down principles and letting them decide

Tim's answer to the benevolent-dictator mistake is to empower people early: write down your principles and guardrails, then have the courage to let people make decisions. Make people make decisions — the earlier you start, the smaller the decisions they can begin with. If you notice you are being a benevolent dictator, stop and empower people instead.

MicroConf · Scaling a SaaS to €35M ARR: 3 Mistakes I'd Never Repeat

Recommendation from Tim van der Kasteelen

This is the speaker's personal recommendation from his own experience, not a validated method.

Recommendation · Money

Go see your actual users in their real environment ('get on the truck')

Kent recommends going out and watching your actual users use your software in their real environment — his shorthand is 'get on the truck,' from a story where a boss asked an engineer whether he had ever ridden along with truck drivers using their software. Seeing users in context is humbling and reveals dozens of concrete improvements (he mentions thinking of around 30) you would never identify from your desk.

AI Engineer · Build the Right Thing: Product Engineering (Part 1) — Kent C. Dodds, EpicProduct.engineer

Recommendation from Kent C. Dodds

Recommendation · Money

Don't turn feature requests directly into implementation

Kent advises not turning feature requests directly into implementation — including not building an app that takes a user's request, sends it to an agent, and automatically implements it, which he says would produce a terrible product with no vision, taste, or judgment. Each feature request requires your product judgment, and requests shouldn't be nailed onto the product you already have.

AI Engineer · Build the Right Thing: Product Engineering (Part 2) — Kent C. Dodds, EpicProduct.engineer

Recommendation from Kent C. Dodds

Recommendation · Money

Use a collection of all three eval types, matched to the question

Most real applications use a collection of code evals, LLM-as-a-judge, and human evaluation together. Use code evals when the answer is deterministic (format validation, length limits, forbidden phrases, required fields), LLM judges when you need semantic understanding, and keep humans in the loop for unseen failure modes and to verify judges are judging correctly.

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI

Recommendation from Laurie Voss

Recommendation · Money

Read your traces and categorize what went wrong before writing a single eval

Before writing any eval, read your traces and categorize what went wrong. Figure out what to measure first. Categorize eval explanations into types of failures and count up the categories — this is called coding. If you see the same explanation across 50 different traces, you have a systematic problem, not a one-off edge case, which makes it the first thing you should fix.

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI

Recommendation from Laurie Voss

Recommendation · Money

Read a dozen or more traces end-to-end before writing any evals

The speaker advised reading a dozen or more traces end-to-end — starting with the ones where something went wrong — before writing a single evaluator, asking what the input was, what the output was, and what specifically is broken.

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI

Recommendation from Laurie Voss

Recommendation · Money

Use code evals for deterministic checks; LLM judges for the rest

Use a code evaluator when looking for a specific string in output (no LLM needed to check whether the ticker was mentioned). Use code evals for production-critical checks like output-is-JSON, length limits, and avoiding forbidden phrases such as 'as an AI language model'.

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI

Recommendation from Laurie Voss

Method · Money

Storing and resuming agent sessions with a data pack

A demonstrated workflow: connect the MCP server to your agent and create a skill that tells it when to store memories; store the conversation in a named data pack and record incident facts as searchable memories; kill the session and resume it later from a brand-new chat window without being inside a project or enabling a special memory mode; ask the same question to verify the agent pulls prior context back out of memory. When resuming, the agent searches memory first instead of creating a new conversation, and the MCP server instructs it to create a new conversation only for storing new content.

AI Engineer · Agent Memory Is Solved. Agent Learning Isn't. — Karthik Ranganathan, Yugabyte

Method described by Karthik Ranganathan

Method preserved for inspection; not a verified protocol.

endorsed

Recommendation · Money

Get your codebase agent-ready: lightweight agents.md files as living documents

Harald Kirschner recommends getting your codebase as agent-ready — a no-brainer that everybody should already be doing. Create agents.md files that are lightweight enough and give agents a good map of the codebase — where to look at, mapping that out. These are living documents that should evolve as the agent makes mistakes and works. Teams that invested in developer experience gain a lot because good documentation and onboarding for a codebase is read by the agent as well and helps the agent as much.

AI Engineer · How VS Code Went from Monthly to Weekly Releases with AI — Harald Kirschner

Recommendation from Harald Kirschner

Recommendation · Money

Encode expert knowledge into skills, e.g. the accessibility best-practices skill

Harald Kirschner recommends creating skills that encode expert knowledge — like VS Code infusing all accessibility best practices into a skill everybody uses. Previously accessibility required one person to get pulled in or give feedback; now that skill is reviewed and maintained by the area owner of accessibility to bring that expertise into everybody's hands. Think how you can encode your expert knowledge to unlock the few users who might always be asked for this input into your skill.

AI Engineer · How VS Code Went from Monthly to Weekly Releases with AI — Harald Kirschner

Recommendation from Harald Kirschner

Recommendation · Money

Maintain your own easy-to-extend evals (VSC bench) and feed developer scenarios back in

Harald Kirschner recommends that when shipping an agentic product, it's important to have your own evals of the product always at hand and easy to extend. VSC bench is VS Code's own eval harness for their agentic product. When a GitHub issue is filed to add a new eval scenario, an agent picks it up from a template and bootstraps the scenario for you. Bring developer scenarios from issues and customer conversations back into your evals so you can start hill climbing against them. Confirm improvements offline first, then run online experimentation — a classic product lifecycle.

AI Engineer · How VS Code Went from Monthly to Weekly Releases with AI — Harald Kirschner

Recommendation from Harald Kirschner

Recommendation · Money

Pull stable authentication logic out of the model's responsibility and reuse it across operations

When the authentication workflow is stable and reusable across business operations on the same portal, pull that logic out from the responsibility of the model and reuse the same authentication mechanism for many business operations. Leaving authentication in the model's responsibility forces it to re-derive a never-changing workflow on every run, adding unnecessary steps and preventing reuse.

AI Engineer · Why 99% Accurate Browser Agents Still Fail — Derek Meegan, Browserbase

Recommendation from Derek Meegan

: none

Recommendation · Money

Give the agent a deterministic OCR verification tool to check its work in real time

Meegan recommends giving the agent an opportunity to verify its work in real time via an OCR tool that deterministically extracts entities from the document and compares them against the system of record — a deterministic mechanism so the agent knows authoritatively whether it did its job right. Deploying without such a mechanism leaves the agent unable to know whether its trajectory succeeded or failed.

AI Engineer · Why 99% Accurate Browser Agents Still Fail — Derek Meegan, Browserbase

Recommendation from Derek Meegan

: none

Recommendation · Money

Encapsulate complex multi-step operations into a single deterministic tool call

When a task involves programmatically interacting with the page — e.g., downloading a file and retrieving it from storage into the agent's runtime — encapsulate the operation into a single tool call that performs download and retrieval in one transaction, instead of the model making decisions on the fly. Leaving such decisions to the model increases steps, raising unit costs and reducing performance and maintainability.

AI Engineer · Why 99% Accurate Browser Agents Still Fail — Derek Meegan, Browserbase

Recommendation from Derek Meegan

: none

Recommendation · Money

Draft a skill — an agent-facing standard operating procedure — to outline the critical path

Meegan recommends drafting a skill, structured like a standard operating procedure for humans but for an agent, that outlines the critical path. Introducing the critical path makes the agent more likely to take the correct next step (e.g., knowing it's at step two and that step three is next) because ambiguity is removed from its decision process. Running without a skill leaves ambiguity, making the correct next step less likely.

AI Engineer · Why 99% Accurate Browser Agents Still Fail — Derek Meegan, Browserbase

Recommendation from Derek Meegan

: none

Recommendation · Money

Measure correctness per transaction, not per run, and permit retries

Meegan recommends measuring agent correctness on a per-transaction basis rather than per-run, permitting retries (e.g., up to four). A hypothetical agent with a 50% per-run success rate and up to four retries climbs to a 94% per-transaction success rate. Ground the metric in the customer: customers don't care how many retries you perform, only that the workflow completed successfully and reliably.

AI Engineer · Why 99% Accurate Browser Agents Still Fail — Derek Meegan, Browserbase

Recommendation from Derek Meegan

: retry count is an example; 94% figure is a hypothetical calculation