From GPT-4.1 to GPT-6.1: what changed when Mama started thinking

Mama started on GPT-4.1, which doesn't think before it answers. Every model since does, and how much it thinks turned out to be the setting we had to get right.

Yesterday we moved Mama’s financial coach onto GPT-6.1 Sol.

She started out on GPT-4.1 in April 2025, and she’s been through most of what OpenAI has released since. Looking back over the changes, most of what we had to learn comes down to one thing. GPT-4.1 doesn’t think before it answers. Every model after it can, and how much it thinks is a setting you have to get right.

The temperature days

With GPT-4.1 you shaped the replies with temperature and top_p. Our first commit had the temperature at 1.7, which is very high, and over the first two months it came down to 0.7.

The main replies ran on GPT-4.1 mini. The classifier that decides what a message is about ran on nano, at a temperature of 0.1.

Moving to a thinking model

In August 2025 we moved to GPT-5, and temperature and top_p were simply gone.

In their place were two new settings. Verbosity controls how long the answer should be. Reasoning effort controls how hard the model thinks before it answers. We ran it at the lowest effort, “minimal”, with verbosity on low.

I don’t think users noticed much change in Mama’s tone. What changed was the depth of her answers. She gave much better answers, with a lot more behind them.

In November the coach moved to GPT-5.1 at effort “low”, so it would think a little more on the money questions, while everything else stayed where it was.

The July migration

In July this year OpenAI started shutting the older models down. GPT-4.1 nano goes this month, and GPT-5, 5 mini and 5 nano go in December.

So everything moved to the GPT-5.4 family, except the coach, which went to GPT-5.6 Terra. Every model name now lives in a single file, and a test fails the build if an old model or an old setting shows up anywhere else in the code.

The settings had changed again.

“minimal” was gone. The lowest effort is now “none”, which means no thinking at all. “none” is also the default, where GPT-5 had defaulted to “medium”, so any call that never set an effort explicitly went from thinking to not thinking without anyone deciding that. We set “low” on those by hand.

Temperature came back, but only when the effort is “none”. Anywhere else, the API rejects it.

The prompts didn’t move with the models

There’s one thing we should have done at each of these changes and didn’t. Mama’s prompts were written for GPT-4.1, and they stayed much the same through GPT-5, 5.1 and 5.4. By right, when you change the model you should go back over the prompts as well, because each family reads instructions a little differently.

We knew that. The problem was that our evals weren’t up to par, so there was no way to tell whether a prompt change had made things better or worse, and no safe way to make one. That only changed recently, when we built evals we could run on every change and keep a history of. It’s what let us rewrite Mama’s voice prompt last month with some confidence.

One budget for thinking and talking

The change that bit us was the token limit.

With a thinking model, max_output_tokens is a single budget shared by the thinking and the reply. Our coach had a cap of 500. A normal reply is under a hundred tokens, so that looked generous.

But on questions that needed a plan, the thinking alone sometimes ran past 500, and the reply was cut off wherever the budget ran out. Last month one went out to a user as “For a low-to” and nothing else. We found nine of those in a week.

The cap is 1,500 now. A reply that gets cut off is retried once with more room, and a fragment never goes out.

The quick-reply chips, the suggested follow-ups under Mama’s replies in the app, had the same problem the other way round. With a cap of 200, the thinking used all of it, and the model never got as far as suggesting any chips.

Deciding when to think

Thinking isn’t always worth it, so it’s set per call.

Logging expenses runs at “none”, because most messages are “lunch RM15” and need no thought. But “Toll RM2.13 and another toll RM2.30” was getting recorded as one expense instead of two. When we tested it, “none” got 6 out of 160 of those messages wrong, and “low” got none wrong.

So the effort now depends on the message:

export function expenseTurnReasoningEffort(
  mode: ModeType,
  message: string,
  /** A total-or-separate question is held (pendingExpenseData.awaitingSplit). */
  heldSplit = false
): Extract<ReasoningEffort54, "none" | "low" | "medium"> {
  if (mode !== "expense_creation" && mode !== "mixed_creation") return "none"
  if (mode === "expense_creation") {
    if (singleItemAmounts(message)) return "medium"
    if (heldSplit) return "low"
  }
  return countAmounts(message) >= 2 ? "low" : "none"
}

One amount, no thinking. Two or more, and it thinks a little.

We tried the same thing for reading bank statements, and it made things worse. At “low” the model dropped a row that had the same date and amount as another one, and took twice as long to do it. Statements stayed at “none”.

Yesterday’s move

Which brings us to Sol.

The coach went from GPT-5.6 Terra to GPT-6.1 Sol, still at effort “low”. On the same set of coach turns, Sol produced no reasoning tokens at all at that setting, and the replies were as good or better. They held the Malay, and none were cut off.

It’s a touch slower, but it’s cheaper. Cached input is half the price per million tokens, and output is a little less too.

Roast mode stays on Terra. Comedy needs a beat of thinking, and Sol at “low” didn’t do any.