Inside Outset: January / February

From Pixels to Tokens

January marked a shift in how we think about research at Outset — not just in what we shipped, but in how teams are beginning to evaluate modern product experiences.

For decades, UX research operated in a relatively predictable world: pixels, screens, and flows. Interfaces were fixed, user paths were known, and experiences varied only slightly from one person to the next. But as AI becomes embedded in more products, that foundation is changing.

Today’s most important user experiences are dynamic, probabilistic, and conversational. They unfold differently for every user, shaped by language, context, and interpretation. As Chris Monnier, Principal UX Researcher on Microsoft Copilot, puts it: today’s most important products are built on tokens, not pixels.

This shift raises a fundamental question for research teams: how do you evaluate experiences that don’t behave the same way twice?

Rethinking how we evaluate AI-driven experiences

As products move from fixed interfaces to open-ended, language-driven interactions, traditional UX methods start to show their limits. Testing screens and flows alone doesn’t capture whether users understand an AI system, trust its responses, or feel confident acting on what it produces.

That’s the gap UX Evals are designed to address.

UX Evals provide a structured way to evaluate AI-driven product experiences with real users, focusing on comprehension, usability, and trust across dynamic interactions. Instead of treating evaluation as a one-off, late-stage step, teams can run evaluations earlier and more consistently — catching issues before they compound.

We’ll be unpacking how this approach works in practice, including how Microsoft is building a research model for this new class of product experience, in our upcoming UX Evals webinar.

A broader view of continuous feedback

This focus on dynamic experiences also showed up in one of our more experimental projects in January: launching the world’s first NPS for a city.

Using Outset, we ran a study capturing how San Francisco residents feel about the city today — how it’s changed, what they love, what they’d fix, and which places deserve more attention. Those responses formed the foundation for a live NPS score, currently sitting at 55 and updating weekly as new feedback comes in.

Paired with bus stop ads across the city inviting locals to scan a QR code and add their voice, the project explored what continuous, public feedback looks like at city scale. You can explore the project and see what SF thinks of itself at voicesofsf.com.

What we shipped in January

Alongside these experiments, January also brought several platform updates aimed at helping teams run clearer, higher-quality research:

• Research Objectives help teams align studies upfront so every interview ladders back to the decisions that matter.

• AI Quality Screening adds built-in safeguards to maintain rigor as research scales.

• Custom Moderators allow teams to tailor Outset’s AI interviewer to their goals, audience, and tone.

• UX Evals introduce a new way to evaluate product experiences designed for AI-driven interactions.

Together, these updates reflect the same underlying belief: as products change, research methods have to evolve with them.

We’re excited to keep exploring what effective research looks like in a world of dynamic, language-driven experiences — and to share what we’re learning along the way.