How we made Mama's message classifier eight times faster

Every message to Mama goes through a classifier first. Over 16 months it went from GPT-4.1 nano to ILMU, a detour through Jev, and now OpenAI's Decisions API.

When someone sends Mama a message like “Lunch RM15”, she needs to record an expense. “How much did I spend on food?” means looking up their spending history. And “Should I pay off my car loan first?” is a question for the financial coach.

To us, the difference is obvious. To an AI, it’s another decision to make.

That’s why every message sent to Mama goes through a classifier before anything else happens. Its job is to figure out what the user wants and send the message to the right place. We have about 17 different modes, covering everything from recording expenses to answering financial questions. The classifier also returns a confidence score, so when it isn’t sure, Mama can ask the user to clarify rather than guess.

It’s a small part of the system, but one that runs on every single message. A slow classifier makes Mama feel slow, even if everything after it is working perfectly.

Over the past 16 months, we’ve changed how this works quite a few times. We started with GPT-4.1 nano, spent several months using Malaysia’s own ILMU model, experimented with Jev, and have now moved to OpenAI’s Decisions API.

Along the way, we learnt that choosing the right model is only half the problem.

Starting with GPT-4.1 nano

The first version went live in June 2025. We had a set of simple rules to catch obvious messages, and anything those rules couldn’t recognise went to GPT-4.1 nano.

A month later, we reversed the order and let the model classify messages first. By November, we’d split the process into two calls. The first identified the general area of the message, and the second picked the specific mode within it.

We also put together a test set of messages with known answers. That gave us a way to measure whether a change actually improved classification instead of relying on a few conversations that seemed to work.

The two-step approach made sense at the time, but it also meant waiting for two model calls before Mama could even begin dealing with the message.

Giving ILMU a try

In February 2026, we moved classification to ILMU, the Malaysian AI model developed by YTL AI Labs.

I’d been interested in trying ILMU since it was announced. Having a model developed here in Malaysia was something I genuinely wanted to support, and YTL was offering free tokens for developers to experiment with.

It turned out to be pretty good at classification. It was fast, understood the kinds of messages our users were sending, and performed well on our test set. It wasn’t a thinking model at the time, so we didn’t use it for Mama’s coaching, but as a classifier it made sense.

We kept OpenAI as a fallback in case ILMU was unavailable.

Or at least, that was the idea.

In May, we discovered that our fallback wasn’t nearly as useful as we thought. ILMU would occasionally hang when establishing a cold connection. Because we were calling it through the OpenAI SDK, which has a default timeout of ten minutes and two retries, the system could spend over two minutes waiting before it ever tried the fallback.

Imagine sending “Lunch RM15” and waiting that long for something that should take a second.

The fix was straightforward once we’d identified the problem. We reduced the timeout to ten seconds and disabled retries. If ILMU didn’t respond in time, we wanted to move on, not keep waiting.

It was a useful reminder that having a fallback and having a fallback that actually works are two different things.

When the free tokens ran out in July, we moved classification back to OpenAI, this time using GPT-5.4 nano.

That was a bit of a shame because ILMU had actually been performing better in our tests. It was getting around 90% of messages right with a median response time of 2.1 seconds. Nano was in the mid-80s and taking closer to 2.9 seconds with our two-call setup.

We continued improving the OpenAI version. In September, we went through the classifier’s rulebook and cut it from roughly 6,000 tokens to 1,500. We also got rid of the two-step routing and went back to a single call.

That made a much bigger difference than we expected. Accuracy climbed into the mid-90s, and median classification time dropped to 1.2 seconds.

We’d spent months adding complexity to make the classifier more reliable, only to find that simplifying it worked better.

Jev was fast. Getting to it wasn’t.

Towards the end of September, I came across Jev from TypeSafe.

Jev is what they call a decision model. Instead of generating text like a typical LLM, you give it a question and it returns something structured, such as a probability, a choice or a score.

That sounded like a good fit for some of the smaller decisions Mama makes.

We weren’t planning to replace the main router with it. Our classifier has rules with a specific order of priority, where the first matching rule wins. Jev doesn’t have a way to enforce that kind of sequence.

But we had other questions worth testing. For example, when Mama asks someone to confirm an expense and they reply “ok”, does that mean they’re agreeing to what Mama proposed?

For a question that simple, calling a full language model feels like overkill.

Our first tests went through an API aggregator. Jev was about twice as fast as OpenAI, but the slower requests were still taking well over the half-second target we had in mind.

So we tried calling Jev directly to find out where the time was going.

The model itself was quick, taking around 175 milliseconds. But we were spending roughly another 265 milliseconds on network overhead, and establishing a fresh secure connection could take around 495 milliseconds from Kuala Lumpur.

For comparison, OpenAI has an edge in Kuala Lumpur, where a new connection took around 50 to 70 milliseconds in our tests.

We also found that our Node.js fetch connections were being dropped after sitting idle for four seconds. With traffic being relatively quiet, we kept having to establish new connections and pay that initial cost again.

There wasn’t anything particularly wrong with Jev. The model was doing its job quickly. We just couldn’t get to it quickly enough for what we needed.

In the end, we didn’t integrate it.

That experiment changed how we benchmark these services, though. We now care much more about the time from our server to the actual answer, including connection setup and network delays. A model’s inference time alone doesn’t tell us what our users will experience.

Then OpenAI released Decisions

In early October, OpenAI opened up its Decisions API.

The concept was very similar to what we’d been testing with Jev. Instead of asking a language model to generate an answer and then extracting a decision from it, we could ask for the decision directly.

You can ask whether something is true, choose from a fixed set of options, or score something against a set of criteria. The API charges for input rather than generated output.

We had it running in production two days later.

The main classifier now sends the message, the relevant conversation context and our compact rulebook to Decisions. It asks which of our supported modes the message belongs to, along with a few other signals we need.

The call looks like this:

const decision = await this.ai.decisions.create(
  {
    model: DECISIONS_MODEL,
    input: `${buildRouterContext(context)}\nMessage: "${message}"`,
    questions: [
      {
        type: "choice",
        name: "mode",
        instructions: COMPACT_SINGLE_ROUTER_RULEBOOK,
        choices: ROUTER_CHOICES
      },
      ...SIGNAL_QUESTIONS
    ]
  },
  { timeout: DECISIONS_TIMEOUT_MS, maxRetries: 0 }
)

We haven’t removed the old classifier. If Decisions returns a confidence score below 0.7, we pass the message to the previous router. And if the API fails, we fall back rather than letting the error reach the user.

On our test set of 119 messages, Decisions classified 116 correctly. The previous router managed 114.

Two extra correct answers isn’t a dramatic improvement on its own. The real difference was speed.

Decisions was taking around 0.3 seconds, compared with 2.4 seconds for the old router in the same comparison. That’s roughly eight times faster.

Around 15% of messages still get passed to the old router, but most can now be classified without that extra wait.

There’s another useful benefit. We can ask multiple questions in the same Decisions request without adding much overhead. So while it’s choosing the message’s mode, it can also tell us whether a coaching question actually needs to search our knowledge base.

That’s one less decision we need to make later.

It’s still a beta API, and we’ve seen one failed call in a sample of about 76, so I’m not comfortable relying on it without a fallback. But the performance improvement has been substantial.

The tricky part was understanding “ok”

We also started using Decisions for something that sounds even simpler than classification: figuring out whether a user has agreed to something Mama just proposed.

Let’s say Mama has prepared an expense and asks the user to confirm it.

If the user replies “ok”, that’s usually a yes. But “ok but RM35 la” is different. They’re agreeing in principle while correcting the amount. We shouldn’t record the original expense unchanged.

Our first attempt didn’t work particularly well.

When we asked Decisions to judge the message “ok” on its own, it returned a probability of just 0.03. It had no idea what the user was agreeing to.

Once we included Mama’s original proposal in the input, it got all 20 of our initial test cases right.

Then we added eight more cases where the user appeared to agree but also wanted to change something. The first version of our instructions incorrectly treated all eight as straightforward confirmations.

We had to be much more specific about what agreement meant:

“Does the user’s reply agree to Mama’s proposal exactly as proposed, in any language (English, Malay, Chinese, Manglish, emoji)? A reply that agrees but also changes or adds anything (amount, date, merchant, category, work or personal, tags) is not agreement.”

With that change, it passed all 28 test cases.

It’s a small example, but I think it captures something important about using these models. Even for a seemingly obvious question, the context you provide and the exact decision you’re asking for make a huge difference.

Where we’ve ended up

Looking back at the classifier’s performance, the progression is quite clear:

Not every change was an improvement. We switched models, added rules, removed rules, tried alternatives and occasionally discovered that something we’d built wasn’t working the way we thought it was.

The biggest improvements didn’t always come from a better model, either. Fixing a timeout, reducing a prompt, removing an unnecessary API call and paying attention to network latency all made a difference.

Decisions is the fastest classifier we’ve used so far, but we’ll keep the old router around until we’re confident it can handle more of the traffic reliably.

For someone using Mama, none of this is supposed to be noticeable. They type “Lunch RM15”, and Mama records their lunch.

That’s how it should feel.