Technical Brief: Engineering Fairness: Inside Candidly’s Research of Its AI-Driven Financial Guidance

Technical Brief is a series from Candidly’s AI product team. Written for builders, evaluators, and anyone who wants to look under the hood, these posts go deeper on the architecture, research, and design decisions behind our AI-native platform.

Written by Ben Levine, Chief Product Officer at Candidly

In 2004, economists Marianne Bertrand and Sendhil Mullainathan asked a simple, uncomfortable question:

Would identical resumes get the same response if the only difference was the name at the top?

They didn’t. “Emily” and “Greg” were 50% more likely to receive callbacks than “Lakisha” or “Jamal.” The study became a landmark in labor economics, and an enduring reminder that bias, even unintentional, runs deep.

Two decades later, as artificial intelligence moves from experimental to operational and helps guide us on where to study, how to invest, and how to pay off debt, we face a troubling reality: large language models trained on human-generated data might inherit and amplify these same biases.

Candidly’s Conversational AI Tool, Cait, provides users with financial guidance to manage debt, make the most of their workplace benefits, and build toward the future. Given what we know about bias in foundation models, we needed to answer a critical question: Would our system replicate the inequities of human decision-makers, or could we build something better?

A controlled test of bias in financial guidance

We set out to answer that question empirically.

We designed a study inspired by the Bertrand-Mullainathan methodology, but adapted for conversational AI. We created 798 synthetic personas representing different demographic groups and levels of financial literacy, then had each one complete a full guidance session with Cait.

The methodology was deliberate. We first generated personas based on financial literacy level (high or low) because that’s a legitimate factor that should influence how an advisor communicates. Only after establishing literacy level did we randomly assign race (Black or white) and gender (male or female) attributes, including stereotypically associated names. This sequencing ensured we weren’t inadvertently encoding stereotypes into the personas themselves. In the spirit of faithful empirical replication, we used the same set of names from the original Bertrand-Mullainathan study.

Personas were asked to complete standard financial guidance tasks, such as evaluating repayment plans or exploring forgiveness eligibility. For each interaction, we tracked two observable signals: the number of dialogue turns — how long Cait spent with the user — and the frequency with which Cait invoked its internal financial tools, such as repayment calculators or refinancing assessments.

These measures are not subjective quality scores; instead, they reveal whether the system exhibits differential treatment of users based on demographic factors. We treated differences as statistically significant only if they exceeded the conventional threshold of p < 0.05.

What the data showed

Across hundreds of conversations, Cait exhibited no statistically significant differences in behavior by race or gender.

Black and white personas engaged in sessions of nearly identical length — around 18 turns on average — with similarly uniform rates of tool use. Male and female personas likewise saw no meaningful differences in how the system interacted with them.

Where Cait did show variation was in response to financial literacy. Personas with lower financial literacy received longer, more detailed interactions than those with higher literacy — 9% longer, a difference that was persistent and statistically significant. This reflects the system’s ability to adjust support based on user need.

This is exactly the behavior we want to see: Cait spends more time with users who need more explanation and guidance, regardless of their identity. That’s the right kind of differentiation.

Architectural choices that enable fairness

Achieving that result required deliberate design. Unlike open-ended chatbots that “freestyle” their way through prompts, Cait operates within a tightly governed architecture. It retrieves information from approved, vetted sources; applies a series of policy and reasoning layers; and executes deterministic calculations that ensure consistent results for identical inputs.

That determinism matters. It means any session can be replayed, audited, and explained. When ambiguity or elevated risk appears, Cait doesn’t improvise — it hands off to a human expert and records the event. Every decision path is traceable.

Equally important, fairness testing is not a one-time exercise. The same statistical audit used in this study is now part of Candidly’s release pipeline. Each time the model is updated, the bias test runs again. If discrepancies appear, the release is halted, and the team replays sessions, examines tool calls, and investigates why outcomes diverged. This process makes the system governable, and governable systems are trustworthy systems.

What this means

These findings are narrow and specific: they demonstrate that this system, with this design, shows no evidence of racial or gender bias on these metrics at this point in time. They don’t prove that all AI is unbiased.

But they do prove something important: the 85% racial preference that researchers found in foundation models isn’t inevitable. These results stand in stark contrast to what’s been documented in off-the-shelf language models. A 2023 University of Washington study modeled on Bertrand and Mullainathan’s framework found that three state-of-the-art LLMs preferred white-associated names 85% of the time when ranking candidates. A broader audit of over 50 models revealed that even systems that pass standard fairness tests still harbor implicit biases, and that newer, larger models don’t automatically fix the problem. In other words, larger or newer models aren’t automatically fairer ones.

The absence of such bias in Cait’s interactions wasn’t the product of chance; it was the result of architectural decisions, careful engineering, and continuous testing designed explicitly to prevent drift toward inequitable behavior. When AI systems are purpose-built, different outcomes are possible.

The contrast is striking. Where human recruiters in 2004 showed measurable bias, and where foundation models in 2024 show even starker bias, a well-designed AI agent can deliver equitable treatment while still adapting to genuine differences in user needs.

The broader implications for financial access

The significance of bias-free AI in financial guidance extends beyond compliance. Financial literacy remains one of the strongest predictors of economic stability, and those who could benefit the most from financial guidance are often those with the least access to it. That’s not a social problem; it’s an addressable market.

If AI systems can deliver equitable, high-quality financial advice at scale, they can help close that access gap. In this way, fairness isn’t just a moral principle; it’s a pathway to market expansion.

By treating bias testing as a continuous engineering discipline, companies can ensure their AI systems expand their serviceable market, such that bias mitigation stops being a compliance cost and starts being part of a growth strategy.

A blueprint for responsible AI in finance

For the financial services industry, this represents both an opportunity and a standard. AI can scale access to guidance that has historically been available only to wealthy clients who can afford human advisors. But that expansion of access only creates value if the systems we build are provably fair.

The methodology we used is reproducible. Banks, recordkeepers, and fintech platforms can run the same test against their own systems. The questions are straightforward: Does your AI treat users differently based on demographic factors? Does it adapt based on actual needs? Can you prove it?