Intelligent Model Routing for LLM Reasoning

A few weeks ago, Arcee AI introduced Conductor: an intelligent model routing system. What makes Conductor different from other model routing services is the fact that it evaluates the complexity, topic, type of task, language, etc of the user query using machine learning models and then routes the query to LLM (or internal Arcee SLMs) which it finds the most appropriate to answer the question. Thus saving heavily on token costs.

An example on how to use Arcee Conductor from command line is provided below:

The API is very familiar! Isn’t it? 🙂

Today, I want to talk about the brand new "Auto Reasoning" mode in Arcee Conductor.

Sometimes you want the language models to "think" about the response. The “thinking” part uses a lot of tokens and so does the resulting answer. wouldn't it be nice to route to appropriate LLM or Arcee SLM based on the query without sacrificing the accuracy of the answer? Thus, we introduce “auto-reasoning” mode in Conductor. Usage is the same as “auto” mode except instead of “model” : “auto”, we use “model”: “auto-reasoning”.

Here is an example input:

And the output is:

As you can see, for this simple query, the selected model was “arcee-ai/maestro-reasoning” which is Arcee’s own internal state-of-the-art small language model (SLM) with reasoning capabilities. What if the query is a bit complicated? Let’s take a look at it in the Conductor UI (conductor.arcee.ai).

In this case, DeepSeek-R1 was selected based on the complexity of the query:

Again, for a simpler query, like “what should i wear for running in rainy weather?”, we see that Arcee Maestro is selected:

Conductor “knows” which query is complicated and which isn’t. And it doesnt guess! It uses advanced AI and Machine Learning algorithms to figure that out.

Conductor currently routes your reasoning requests to:

• Arcee Maestro

• O3-mini high

• Claude 3.7 Sonnet Thinking

• DeepSeek R1

Thus, no matter how complicated or easy your query is, it always gets answered.

Check out Arcee Conductor here:https://www.arcee.ai/product/arcee-conductor and if you have any questions, feel free to ask :)