Perplexity’s Post

Should "Cost-Per-Correct" (CPC) be a key LLM ops metric? I built and ran a token harness across a dataset of 200 5th-grade level math problems, and evaluated them with GPT-5 across a grid of token caps (64, 128, 256) x shots (0, 2, 8) x reasoning levels (minimal, medium, high). TL;DR: - CPC = (total $ spend) / (# correct outputs). Lower is better! - Reasoning shouldn't be “always-on.” - Token efficiency is the name of the game. Full write-up + repo (link in the comments).

What could be better than GPT-5 release? MCPAdapt now supports Structured Output with Smolagents. This is a game changer, drastically reducing CodeAgent iterations (think tokens, think $) and error rate. For a little while MCP tools could only return one type of output: string (excl. binaries). And for most LLM systems that was ok as they mostly work with text. However the wole idea of CodeAgents from #HuggingFace is to reason in interpretable code. Having tools that only return string in this context is quite limitating as it would often result in several wasted iterations fixing parsing errors. Fortunately MCP added structured output into the spec allowing tools to define more precisely the type of their output. We leveraged this to hint CodeAgent about the type of the returned object. In my few tests it systematically reduced the number of iterations and errors of the Agent. The screenshot shows the same prompt triggering an mcp server with a single tool returning a dict with a laptop specs. Without using structured output on the left and with structured output on the right. You can observe on this very simple case 5 iterations vs 3 iterations. 12k tokens vs 6k tokens for the exact same answer. Without structured output, after parsing error, the Agent even decides to resort to regex to extract the data from the string...

I made this simple tool for detecting language-based biases in LLMs. You write a question in your language, but behind the scenes, the tool translates it into a different "processing" language, asks the LLM, then translates the answer back. This way, you can see how the LLM's answer changes depending on the language used to think in: https://lnkd.in/dsq2C3eR

We explore why Tool-Integrated Reasoning (TIR) enhances LLMs in our new work, "Understanding Tool-Integrated Reasoning," and introduce a new algorithm, Advantage Shaping Policy Optimization. This study proves TIR expands LLM capabilities! https://lnkd.in/gg9GR7jF

learn how to build an LLM from scratch, @rasbt's repo is really a gem. it has notebooks with diagrams and explanations that will teach you 100% of: > attention mechanism > implementing a GPT model > pretraining and fine-tuning GitHub link:- https://lnkd.in/ecmxgema

Not every input token in an LLM prompt contributes equally. And many times, tokens can even create negative value. Some tokens clarify intent, preserve context, or help the model reason accurately. Others introduce ambiguity or redundancy, adding noise that makes the LLM’s job harder. OpenAI and Anthropic have trained our brains to think of tokens as equal ($0.15 per million input tokens). But what if you were penalized for bad tokens (more expensive) and rewarded for good tokens (less expensive)? I’ve been thinking about how to measure the value of each input token based on two factors: how useful it is, and how much it's affected by noise in context. Here’s a simple formula I’ve been experimenting with: Value Adjusted = Value Base × (Quality Boost) × (Noise Penalty) Or more specifically: Value Adjusted = Value Base (ie. $0.00001 per input token) × [1 + α × (q - 0.5)] × [1 - γ × noise × (1 - q)^δ] Where: q is a quality score for the token (between 0 and 1) noise reflects contextual uncertainty or ambiguity α, γ, and δ are tuning parameters This allows higher-quality tokens to be rewarded and lower-quality ones to be penalized more aggressively, especially when noise is high. The idea is to capture how valuable each token is to the model, and how fragile that value becomes in less clear contexts. Take this prompt: "So basically the thing I was wondering is like, you know, how do I, um, change the AC capacitor?" This is full of filler tokens that carry little useful information and introduce noise. The true signal is something much simpler: "How do I change my AC capacitor?" This is a basic example, but the problem is magnified when you have conflicting tokens that then requires reasoning (ie. more expensive models). By assigning value to each token, we could improve how prompts are scored, compressed, or even priced. It might also help with long-context models where not all tokens should count equally. Is anyone aware of any models like this? Can you please share them with me?

Using Regex, we can easily match or search for patterns in a text. Before searching for a pattern, we have to specify one using some well-defined syntax. In this problem, you are given a pattern. You have to check whether the syntax of the given pattern is valid.

🌌 Day 151 of #gfg160 Challenge 🌌 📌 Problem: Alien Dictionary 🎯 Difficulty: Hard Today’s challenge was really interesting — figuring out the order of letters in an alien language 👽. We are given a sorted dictionary of words, and from this we must deduce the lexicographic order of characters. 🔍 Problem Summary Words are sorted as per alien rules We need to determine a valid character ordering If no valid order exists (contradictions), return "" Example 1: Input: ["baa", "abcd", "abca", "cab", "cad"] Output: "bdac" (valid order) Example 2: Input: ["caa", "aaa", "aab"] Output: "cab" Example 3: Input: ["ab", "cd", "ef", "ad"] Output: "" (no valid ordering possible) 💡 My Approach This boils down to a graph problem: 1️⃣ Treat each character as a node. 2️⃣ Build edges between first differing characters in adjacent words. 3️⃣ Detect contradictions with cycle detection (DFS recursion stack). 4️⃣ Use topological sorting to produce the final order. 📚 Key Takeaways This problem is a classic topological sort use case. Cycle detection is crucial — contradictions mean no valid ordering. It’s a great mix of graphs, recursion, and lexicographic reasoning. 💬 This problem reminded me how language rules can be modeled with graphs. Curious — would you solve this using DFS recursion (like above) or with BFS (Kahn’s Algorithm)? #gfg160 #geekstreak2025 #GraphAlgorithms #TopologicalSort #AlienDictionary #DSA #ProblemSolving #CodingChallenge #GFG

Lost in all the hype: the art of using LLMs as classifiers. Which is especially useful when building search. We can talk about agents and AGI all day, but one of the best applications of an LLM is the ability to classify some text (query, paragraph, etc) to a set of labels. It seems relatively trivial, but there are a lot of techniques you can use to improve the accuracy of your classifier * Using structured outputs for a small set of labels * Tuning the prompt between precision / recall -- ie letting the LLM say "I don't know" depending on its confidence in the answer * For large label set, intentionally letting the LLM generate/hallucinate a fake label that you resolve to real ones via embedding search A lot of it is managing the amount of freedom you give an LLM and how exactly you then constrain that freedom to the set of labels you care about. There's more of course in Cheat at Search with LLMs :) Where you learn to use LLMs for query and content classification in search. Hope to see you there.