Explore intelligence

Principle · Money

Work with research by writing evals and bringing specific use cases and sample readings

When working with research, come with very specific use cases and a clear idea of what users want and their goals; bring sample readings — understand the session, what it looked like, and why the model didn't do what it needed to do. Learn to write good evals and write them as much as possible — it's the most important thing. The ideal artifact to take to post-training is a demonstrated eval: this prompt plus these skills achieved this outcome, so the capability can be trained into the model from the beginning.

Lenny's Podcast · Stop planning for 2027: how OpenAI builds product 90 days at a time | Tara Sesha and Nan Yu (OpenAI)

Claim from Tara Sesha (OpenAI)

Principle · Money

When you don't know what to do, do nothing: get still before big decisions

Matt Paulson's counterintuitive advice is that when you don't know what to do, do nothing. Big inflection-point decisions don't get better by spending more time thinking about them, and seeking opinions from 15 different people drowns out your own voice. Instead, step back and get still through meditation, breathing exercises, music, or working out to remove noise and reconnect with your initial intuition. He notes there are neurons in your gut and heart, so information comes through outside your brain that you should listen to.

Foundr · The Near-Death Experience That Built a $400M Year Business

Claim from Matt Paulson

Principle · Money

Workarounds and effort are the strongest evidence a problem is worth solving

Most of the time, the person with the problem you want to solve is not actively experiencing it at that moment, so they'll have to think back. You want to know what users do instead — their workarounds — and how much that workaround costs them and how often the problem happens. Find out how much effort the user is already putting into solving the problem: that is gold and really strong evidence the problem is worth solving. If the user can't remember the last time the problem happened, that's a signal — not the one you want, but one you want to have. Conversely, if users say they don't remember and just gave up, that's a signal they probably won't pay or change workflows. If they are actively experiencing the problem right now, that's a good signal and you'll get really direct answers.

AI Engineer · Build the Right Thing: Product Engineering (Part 1) — Kent C. Dodds, EpicProduct.engineer

Claim from Kent C. Dodds

Principle · Money

Don't ask users to evaluate your idea or diagnose the problem (Mom Test)

Per the Mom Test, Kent C. Dodds says you don't want to ask users to evaluate your idea or diagnose the problem for you; people naturally look for someone to say the problem is huge and they'd spend untold sums to solve it, and talking about your specific solution before users agree the problem is worth solving is a mistake. Weak questions include 'Would you use this?', 'Is this a problem?', and 'What is the problem? Help me define the problem.'

AI Engineer · Build the Right Thing: Product Engineering (Part 1) — Kent C. Dodds, EpicProduct.engineer

Claim from Kent C. Dodds

Principle · Money

The right number of credentials in a sandbox is always zero

Jim Clark's maxim: the right number of credentials to have in a sandbox is always zero — credentials should never be in there. If there are absolutely no credentials in a sandbox, the blast radius of an agent doing something incorrect is reduced. He also says never have any credentials anywhere in any harnesses.

AI Engineer · How Many Credentials Should Your AI Agent Have? Zero. — Jim Clark, Docker

Claim from Jim Clark

Creator perspective · Money

Scope credentials to the moment of need: Git signing keys only when committing

Jim Clark notes that in an hour of agent work, the agent might need his Git signing keys for only about two minutes of that time. When committing, the agent needs the Git commit tree, the ability to make the commit, and signing keys — but nothing else. He would like the sandbox an agent runs a task in to be modeled after the intent of what the task is doing: if the agent is not making a commit, commit signing keys should not be inside that sandbox. If agents know the intent of what you're trying to do, use that intent to model the capabilities you give them.

AI Engineer · How Many Credentials Should Your AI Agent Have? Zero. — Jim Clark, Docker

Claim from Jim Clark

Principle · Money

Keep judge output simple: binary labels, not numeric scales

Keep judge output choices simple — binary is best, two clear labels. Numeric scales like 1–5 are tempting but LLMs are bad at them because they can't distinguish a three from a four. If you really need more categories, use a partial result like a 0.5 score, but by default go for zero or one. Under the hood the judge is given a tool it calls with correct or incorrect, so you don't need to specify output in the prompt.

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI

Claim from Laurie Voss

Principle · Money

Don't use a correctness eval when the agent uses live data beyond the judge's training cutoff

In the demo, the correctness eval scored zero on every row — it completely failed. The correctness eval failed because it judges against the LLM's training data, while the agent used live web search for current 2026 stock prices the judge couldn't possibly know. Don't use a correctness eval when your agent uses live data beyond the judge model's training cutoff — it will complain about information from the future it couldn't know. Instead, use a faithfulness eval when your agent uses live data, because the judge actually has context and is on a level playing field with the agent.

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI

Claim from Laurie Voss

Creator perspective · Money

Know which evals are guardrails (ship blockers) vs North Star metrics

Know which of your evals are guardrails (ship blockers) and which are North Star metrics (aspirational), because it changes how you act on results and set up production monitors. Some evals are guardrails and ship blockers: if an agent hallucinates a stock price, that is a hard fail — don't ship. A guardrail regression should page someone in the middle of the night. A North Star metric dip is something you look at in a weekly report.

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI

Claim from Laurie Voss

Creator perspective · Money

Don't build a single 'God evaluator'; split into one evaluator per dimension

Don't build a single 'God evaluator' that checks accuracy, tone, completeness, policy compliance, and formatting all in one prompt — it is a nightmare to calibrate. A God evaluator that fails makes it impossible to know which of its five dimensions failed. Having a God evaluator emit 15 different labels becomes too complicated to judge or score by. Split evaluation into one evaluator per dimension instead of one mega evaluator; the combination gives a richer picture than any single eval.

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI

Claim from Laurie Voss

Principle · Money

Code evals: deterministic, cheap, and the first line of defense

The next step after knowing what's wrong is code evals — no model configuration, no API calls, just Python. A code evaluator is a simple Python function that returns an evaluation result with a label and a score; it needs no LLM and no API call and returns instantly. Code evaluators are completely deterministic functions that take input and output and decide whether the output is good according to some definition of good — anything with grading logic that always gives the same answer for the same input can be a code evaluator. A code eval doesn't have to be a simple string operation — the grading logic just has to be deterministic; it can query a database or hit an API. Use simple deterministic code evaluators as the first line of defense in your evaluation stack (Swiss cheese model).

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI

Claim from Laurie Voss

Principle · Money

Renewal is the prime expansion moment — but only with demonstrated value

Treat renewal as a commercial milestone where you present accumulated value evidence and ask for more (years, business units, products). Never ask for more money at renewal if you've only met the account cold — e.g., second meeting in eight months — without demonstrated value; that's like a stranger knocking on the door selling old brooms.

30 Minutes to President's Club | No-Nonsense Sales · #604 - Upsell & Expansion Masterclass: 7 Lever to Grow Any Account

Claim from host

Principle · Money

Four key valuation drivers: growth, ARR, churn, profit (in order)

The four key things that impact valuation for a B2B SaaS business, in order, are growth, ARR/revenue, churn, and profit. Founders should optimize them in this order.

MicroConf · When to Sell Your SaaS Business: How to Get a 5x Better Offer

Claim from unknown

Creator perspective · Money

The core of the PM role is hypothesis definition and testing, not trappings like docs and presentations

Tara Seshan states that the core of the PM role has always been about asking the most essential question about your product: what determines whether it works or not. Many trappings of the job (execution on time, writing docs, presentations) have fallen away, and the hypothesis definition and testing loop has become the most important thing at the company. Everyone (EMs, engineers, data scientists, designers) now focuses on this.

Lenny's Podcast · AI’s third era: the rise of persistent AI coworkers | Tara Seshan (OpenAI’s product lead)

Claim from Tara Seshan

Recommendation · Money

Start shipping and let your moat develop over time

Anish Acharya advises founders to start shipping and let their moat develop over time rather than trying to design it upfront. He pairs this with the script 'Moats are most often discovered, not designed,' warning against overthinking upfront strategy.

Lenny's Podcast · Why companies are becoming a series of loops | Anish Acharya (a16z)

Recommendation from Anish Acharya

Recommendation · Money

Reorganize the whole company around AI, not just tool access

Anish Acharya distinguishes between merely using AI and reorganizing your entire company around AI, recommending the latter. He suggests asking: if AI were infinitely intelligent and astonishingly cheap, how would you reorganize the company? He also says to give the average employee more credit for embracing AI tools, citing a 'Jedi Academy' concept where everyone at a company, including mechanics, is taught to use the new tools.

Lenny's Podcast · Why companies are becoming a series of loops | Anish Acharya (a16z)

Recommendation from Anish Acharya

Recommendation · Money

Put yourself in the environment where the thing you want is happening

Matt Owens calls embedding yourself in the environment where the thing you want is happening 'the secret' to his rapid skill growth — being in a growing scene fuels learning and creative pace.

Design Better · Matt Owens: Athletics co-founder on keeping your creative fire lit

Recommendation from Matt Owens

endorsed

Recommendation · Money

Build a business that gives people permission to go off script and experiment

Matt Owens argues you have to keep one foot in the possible space and build a business that gives people permission to go off script and experiment.

Design Better · Matt Owens: Athletics co-founder on keeping your creative fire lit

Recommendation from Matt Owens

endorsed

Recommendation · Money

Define what success looks like before categorizing failures

The speaker advised defining requirements — what a good report looks like for your agent — before categorizing failures, noting that many teams instead ship an agent and react to complaints without ever defining the bar up front.

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI

Recommendation from Laurie Voss

Recommendation · Money

Pivot to AI-Native to Survive Existential Threats

When facing existential threats from AI-native startups and large incumbents, a company must pivot to being AI-native rather than gradually layering in AI features. Ignore industry experts who claim customers fear new technology; validate directly with customers instead. Push through internal and external resistance, neglect other burning fires, and go all-in on AI.

The Official SaaStr Podcast: SaaS | Founders | Investors · SaaStr 874: From 0% to 83% AI-First Customers in 2 Years. How Owner's CEO Rebuilt a $100M Vertical SaaS Company

Recommendation from Adam Guild

Consequential creator advice requires independent verification before Today can promote it.