"Exploring Extreme Quantization for LLMs: A New Framework"

For nearly seven decades, Dijkstra’s algorithm has reigned supreme as the gold standard for finding shortest paths in graphs. Born from a 20-minute mental exercise at an Amsterdam café in 1956, Edsger Dijkstra’s creation has been the backbone of everything from GPS navigation to network routing protocols. But that reign just ended. A research team led by Ran Duan at Tsinghua University has achieved what many considered impossible: they’ve broken the fundamental “sorting barrier” that has limited shortest-path algorithms for 40 years. Their new deterministic O(m log^(2/3) n)-time algorithm for single-source shortest paths represents a breakthrough that challenges textbook assumptions about algorithmic limits.

Here, I am posting about the A* algorithm, which is built on top of Dijkstra’s Algorithm. It works better when a heuristic value is available. These are the basic building blocks for developing more advanced pathfinding algorithms with modern improvements.

When improving LLM Accuracy, it’s common to assume the process is linear: Prompt Engineering → RAG → Fine-Tuning. In reality, it’s more nuanced. A better approach is to start with prompt engineering. This gives you a baseline to evaluate performance. From there, your next step depends on what you find: 1. Context Optimization (RAG)   This is useful when the model lacks domain knowledge because it wasn’t part of its training set, or when its knowledge is outdated. Adding retrieval helps the model provide more accurate and relevant responses. 2. Model Optimization (Fine-Tuning)   This is the right choice when the model is inconsistent, produces incorrect formatting, or doesn’t match the tone and style you need. Fine-tuning helps lock in consistent behavior. The important point is that this isn’t a strict sequence—it’s a decision based on what’s missing in the model’s performance.

Revised paper on deductive program synthesis. Program derived from a logical specification using theorem-proving technology. Correctness established and rationale explained as a byproduct of the proof. In a case study, dozens of unification algorithms were generated automatically.

How to Train Your LLM? Memory Decoder Crushes RAG For years, LLM domain adaptation has been stuck in a compromise: the immense costs and "catastrophic forgetting" of DAPT, or the frustrating latency and clunky overhead of RAG. But a new approach is here, and it feels like a generational leap. Discover the Memory Decoder, a brilliant, plug-and-play memory component that bypasses the limitations of its predecessors. By learning to imitate a retriever, this compact module supercharges your LLM, delivering both superior performance and unparalleled efficiency. Can a small, dedicated "memory chip" truly make a 0.5B model outperform a 72B-parameter behemoth? The research says yes. Read on to find out how this paradigm shift could make the old methods obsolete. https://lnkd.in/gmBHXTcb

Goodbye RAG, Hello Memory Decoder. The Memory Decoder fundamentally reimagines how we infuse specialized knowledge into large language models. It takes the key advantages of its predecessors—the deep domain knowledge of DAPT and the plug-and-play nature of RAG—and wraps them in a single, efficient, and versatile architecture that avoids their biggest pitfalls. https://lnkd.in/gDzYuW3x

📊 If you wonder why #LLM benchmarks indicate that rivals go almost head-to-head, the article from Sebastian Raschka, PhD provides a clear answer: every good idea migrates from model to model, thanks to #open #weights and #source initiative. As often mentioned in the article linked below, architectures become very common, having only second-order differences. 🔎 However, for multibillion scale projects even a small percent of advantage brings significant benefits in absolute measures. This article is devoted to such details. I hope you will like it. The subject isn't simple and describes nine models in full details, so the text appears to be a long read naturally. ❓ A Q-K crossing as an argument to function of mixing over V is a key component of #Transformer technology. What keeps my curiosity alive is a wide presence of FC layers and normalizers across all architectures. These additions serve as correctors to the main math. They require significant part of total number of parameters. They take significant time to compute. They can't be reduced or completely dropped out from the model. So, the question is what is missed or wrong in the method that we need to adjust base computations across all the model repeatedly? 📰 "The Big LLM Architecture Comparison" https://lnkd.in/dkjQXysD 📷 The picture below was taken from this article

The research system is built on an orchestrator-worker pattern, a common design in computing where one central unit directs the process and supporting units carry out specific tasks. https://lnkd.in/gPJk2SBF