Explore intelligence

Recommendation · Money

Split evaluations into one evaluator per dimension

Building one big mega evaluator that checks everything at once is a nightmare to calibrate — when a 'God evaluator' says fail, you can't tell which of the five dimensions failed. Split evaluations into one evaluator per dimension; the combination of multiple focused evals gives a richer picture than any single eval ever could.

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI

Recommendation from Laurie Voss

Method · Money

Add observability to an agent app and verify traces are flowing

To add observability to an agent app, ask the coding agent to add observability with Arize AX to the application (the speaker asked: 'Can you add observability with Arize AX to this application? You've got secrets in .env.local already.'); to get observability out of an application you only need to turn on tracing — tell the app you're using Arize, provide your API key and endpoint, and send the traces it already generates to your endpoint; implementing observability in an OpenAI Agents SDK app is simpler than expected because the SDK is already instrumented with OpenInference; after adding observability, restart the app and run a query to verify traces are flowing, and to verify tracing is configured, check your .env.local file for the tracing project name and look for that project in the

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 201 — Laurie Voss, Arize AI

Method described by Laurie Voss

Method preserved for inspection; not a verified protocol.

Commercial context: The creator presents this as their own offering and directs viewers to get it.

Method · Money

Have the coding agent analyze traces, find problems, and fix a specific problem, then verify the fix

To have the agent analyze your traces, ask it to pull your traces and look for things that are going wrong ('Can you pull my traces and look for things that are going wrong?'); to fix a specific problem found in traces, ask the agent about that problem in particular and whether it can fix it ('What about that problem in particular? Can you fix that problem in particular?'); after the agent fixes the problem, verify the fix by re-running the failing query and checking the traces show correct input, output, and response; throughout the demo the speaker never touched the code — all he did was say 'look for problems' and 'fix that problem in particular'.

AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 201 — Laurie Voss, Arize AI

Method described by Laurie Voss

Method preserved for inspection; not a verified protocol.

Recommendation · Body

Focus each session on one quality: acceleration day or high-speed day, not everything at once

Dr. Morin advises that on acceleration day, you focus only on the acceleration — everything afterwards is not important. When coaches cue and emphasize everything every time in sprinting, there is no way to get quality over the entire spectrum — this is the main problem of beginning coaches. He structures one weekly session focused on acceleration and two sessions focused on high-speed running, with the second high-speed session being a hamstring prevention session rather than a max-velocity session. For acceleration work, he uses about the same metrics as high-speed work (100-150 meters of exposure), noting it's a principle, not a recipe.

Garage Strength · Why Most Athletes Train Speed Wrong w/ Sprint Scientist Dr. Jean-Benoit Morin

Recommendation from Dr. Jean-Benoit Morin

Consequential creator advice requires independent verification before Today can promote it.

Recommendation · Body

Strengthen trunk and hamstrings in highly bent positions for athletes whose sport demands bent positions

Dr. Morin recommends that for athletes who must stay in bent positions for their sport (like running backs and basketball point guards), rather than changing their running technique, you strengthen the trunk and hamstrings in highly bent positions. He advises using very intense trunk flexion exercises, choosing exercises that involve both trunk flexion and huge hip flexion, and using unilateral exercises as much as possible to replicate the lunge and sprint movement. He cites the drop and catch RDL as an example of a very intense trunk and hip flexion exercise, and notes resisted sprinting is useful because it forces the athlete to be more bent forward with higher hip flexion, strengthening the posterior chain with kinematics close to future actions. The same bent-position strengthening is

Garage Strength · Why Most Athletes Train Speed Wrong w/ Sprint Scientist Dr. Jean-Benoit Morin

Recommendation from Dr. Jean-Benoit Morin

Recommendation · Body

Assess acceleration objectively with metrics, then combine quantitative and qualitative diagnosis

Dr. Morin advises that when an athlete shows good top speed, strength, and broad jump but weak acceleration, the first step is to assess acceleration objectively with metrics — maximum force output and maximum propulsive power output on the ground. You can also bring kinematic analysis including contact time to diagnose acceleration weaknesses such as trunk collapse or overly vertical push. He stresses combining quantitative numbers with qualitative movement assessment — how does he move? He also notes that changing the trunk angle during acceleration likely addresses braking issues in the first three to seven steps (per the host's example with a running back), and that an athlete already elite at high jump will very likely not improve acceleration by jumping higher.

Garage Strength · Why Most Athletes Train Speed Wrong w/ Sprint Scientist Dr. Jean-Benoit Morin

Recommendation from Dr. Jean-Benoit Morin

Recommendation · Body

Strengthen athletes in bent positions instead of changing their sport movement

For athletes who must stay in bent positions (like point guards), you would never tell them to drive less horizontally because it is dangerous for the hamstrings; instead, teach them to be much stronger with their trunk and hamstrings in exercises that stimulate highly bent positions.

Garage Strength · Why Most Athletes Train Speed Wrong w/ Sprint Scientist Dr. Jean-Benoit Morin

Recommendation from Dr. Jean-Benoit Morin

Recommendation · Money

Verify a verifier by training on its filtered data

Another way to verify verifier quality is to train on data filtered by it: hold the number of training examples constant (3K or 9K trajectories) and filter them by whether the verifier judged them as passed. When doing SFT, train only on successful trajectories (where the verifier marked them true).

AI Engineer · Your LLM Judge Is a Confident Liar: Building Better Verifiers — Browserbase

Recommendation from Miguel Pagano

Recommendation · Money

Use speculative decoding: a speculator guesses tokens and the target model grades them in parallel

Charles Frye recommends speculative decoding as a key technique: take a separate model (the speculator) to guess the next couple of tokens, pass them through the target model, and grade them in parallel — turning decodes into tiny prefills. With rejection sampling applied, the target model's output is guaranteed to be the same as sequential decoding, up to numerics. As you improve the quality of the speculator and the number of tokens it can guess in a row, you get a basically linear speedup in decode throughput.

AI Engineer · What Is an Inference Engine, Anyway? — Charles Frye, Modal

Recommendation from Charles Frye

Recommendation · Money

Use CUDA graph capture to turn per-kernel CPU launch work into a single graph launch

Charles Frye recommends CUDA graph capture as a key technique for reducing CPU-side launch overhead: track all kernels in the model forward path, turn them into one big DAG of operations, and launch them from a single graph instead of launching each kernel individually. This turns per-kernel CPU launch work into a single CPU-side decision of which graph to launch, relaxing constraints in the model forward pass.

AI Engineer · What Is an Inference Engine, Anyway? — Charles Frye, Modal

Recommendation from Charles Frye

Recommendation · Money

Keep systems small; expand cautiously and verify at each step

Blaine Billingsley recommends staying vigilant about keeping your system small — the bigger things get, the more likely you won't be able to guarantee their success. Make sure your product solves one thing absolutely perfectly at small scale before cautiously expanding, verifying it doesn't fail at each step up.

Design Better · Blaine Billingsley: OpenAI designer on the collaboration problem AI hasn't solved yet

Recommendation from Blaine Billingsley

Method

Steroid management of suspected differentiation syndrome: dexamethasone 10 mg twice daily for three days

Dr Ghayas Issa advises that at any suspicion of differentiation syndrome, give high-dose steroids: dexamethasone 10 milligrams twice a day for three days, regardless of whether symptoms improve, because the goal is to prevent the syndrome from recurring. A dose of steroids can be diagnostic — patients often feel 100% better with the first dose, confirming you're on the right track. Because differentiation syndrome can be deadly, it is better to err on the side of giving steroids than to wait to confirm the diagnosis and risk it being too late to correct.

Research To Practice · Acute Myeloid Leukemia — An Interview with Dr Ghayas Issa on the Role of Menin Inhibitors

Method described by Dr Ghayas Issa

Method preserved for inspection; not a verified protocol.

Consequential creator advice requires independent verification before Today can promote it.

Method

Workup of suspected differentiation syndrome on menin inhibitors

Dr Ghayas Issa outlines the evaluation for suspected differentiation syndrome: check the blood count for an increase in white count (sometimes associated with differentiation); look at whether leukemia cells are turning into normal cells; a chest x-ray may be done because pleural or pericardial effusions can occur; an echo may be performed if there is a pericardial effusion; and kidney function should be evaluated.

Research To Practice · Acute Myeloid Leukemia — An Interview with Dr Ghayas Issa on the Role of Menin Inhibitors

Method described by Dr Ghayas Issa

Method preserved for inspection; not a verified protocol.

Consequential creator advice requires independent verification before Today can promote it.

Method

Evaluating suspected differentiation syndrome

Dr Ghayas Issa describes the workup for suspected differentiation syndrome: check blood counts for an increase in white count (sometimes associated with differentiation) and look for leukemia cells turning into normal cells; obtain a chest x-ray because pleural or pericardial effusions can occur; perform an echocardiogram if there is a pericardial effusion; and check kidney function.

Research To Practice · Acute Myeloid Leukemia — An Interview with Dr Ghayas Issa on the Role of Menin Inhibitors

Method described by Dr Ghayas Issa

Method preserved for inspection; not a verified protocol.

Consequential creator advice requires independent verification before Today can promote it.

Method · Money

Armand's three-part cold call structure: opener, problem proposition, and objection handling

Armand describes the call structure he taught SDRs: an opener, then when the prospect says no, leading into a 'problem proposition' that leads with the problem before the solution (illustrated by his Carta example about CFOs finding it ridiculous to pay lawyers to issue stock options). For the spreadsheet objection, he used the 'Mr. Miyagi method' of agreeing with the prospect and selling a test drive. He reports that these three things got SDRs good on the phones, moving cost per appointment from $50 to $200 and meetings per SDR from four to eight.

30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)

Method described by Armand (guest)

Method preserved for inspection; not a verified protocol.

Method · Money

Five weekly meetings that run an SDR team

Armand describes five meetings that can run 90% of SDR teams: a Monday morning meeting (MMM) that injects energy and focuses on activity, Teardown Tuesdays and Thursdays (30-minute sessions reviewing three cold calls, two cold emails, or two to three accounts, giving coaching without adding much calendar time), Wednesday one-on-ones for individual coaching, metrics review, and role play, and Friday team dials that end the week with energy and focus on activity plus culture. The week is bookended by energy-focused meetings on Mondays and Fridays with skill-focused sessions on Tuesdays and Thursdays.

30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)

Method described by Armand (guest)

Method preserved for inspection; not a verified protocol.

Recommendation · Money

Win over each rep privately before the team all-hands

Before addressing the team publicly, conduct 16 hours of back-to-back one-on-ones with every team member, using a 'between you, me, and the wall' conversation to privately share the turnaround plan. Winning over individuals first means peer pressure helps win over the rest of the group.

30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)

Recommendation from Armand Farrokh

anecdotal

Method · Money

SDR hiring process: recruiter screen, mock cold call, and triple onsite

Armand describes a revamped SDR interview process. A recruiter screen cuts about 50% of the applicant pool. The second step is a mock cold call, where the first call is expected to be bad; the interviewer looks for whether the candidate's gears are turning and how much they researched, gives no more than two points of specific feedback (e.g., leading with a problem proposition instead of a pitch), and treats no change on the second call as a massive red flag. After the mock cold call the interviewer should know if they have a 'triple A' hire. The final stage is a triple onsite of three back-to-back interviews: a mock cold email (written live or drafted beforehand, testing whether the candidate applied prep and used their brain), a rep filtering test (narrowing 40,000 companies to 25, with

30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)

Method described by Armand (guest)

Method preserved for inspection; not a verified protocol.

Recommendation · Money

Remove waste from the prospect list instead of increasing activity

A prospecting turnaround can often be achieved not by increasing activity or improving phone skills, but by removing waste in the prospect list. Categorize personas into A/B/C tiers and replace all C-tier prospects with more A-tier prospects without increasing overall activity. Don't reach out to personas that are a poor fit for your product (e.g., selling stock option management software to a Chief HR Officer).

30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)

Recommendation from Armand Farrokh

endorsed

Recommendation · Money

Generate many theories concurrently — Cleric's primary technique

Pienaar recommends generating many different theories simultaneously to give the agent breadth and diversity of ideas, calling this Cleric's primary technique. The rationale: agents are excellent at relative ranking but terrible at absolute scoring of their own theories, so multiple candidate ideas beat asking for one scored answer.

AI Engineer · Your AI Agent Is Confidently Wrong About Production — Willem Pienaar, Cleric

Recommendation from Willem Pienaar