Building the Foundations for Real AI Models

🌀 THE RIFF | Edition #8

Yesterday I caught up with a friend and colleague to talk tech, policy, and the future of humanity—specifically, how to separate AI hype from reality. We lingered on “large” versus “small” language models and realised the word small now carries baggage. Are we talking about smaller than yesterday’s small (through pruning, distillation, quantisation, or tight fine-tuning LLMs), or about small by intent—domain-specific models designed for bounded problems? And how do we get there?

Small vs Smaller: what actually changes?

The learning principles don’t change: these models form statistical associations over the knowledge they’re exposed to and use those associations to solve new prompts. What does change is scope and footprint.

• Introducing SLLMs (Smaller Large Language Models): same deep-learning building blocks as LLMs, but trained or adapted for a narrower domain, with a smaller compute and memory footprint and tighter knowledge base. Open source is a good start.
• How they get smaller: pruning, distillation, quantisation, LoRA-style fine-tuning, retrieval boundaries, and data curation—each trading generality for efficiency, predictability, and cost control.

Let’s focus on tools for data curation.

About “memory” and agentic AI

With the need to control data synthesis, “memory” has re-entered the conversation via product features, but agent architectures with memory are decades old (Kalogeropoulos DA, Carson ER, Collinson PO., 2003). The novelty isn’t that memory exists; it’s how today’s models blend statistical association with tool use, (federated) learning loops, and persistent context. The risk is mislabelling marketing features as breakthroughs.

Foundations first: data quality and representational efficiency

If SLLMs are to be genuinely useful (and safe), the responsibility shifts upstream:

• Curate high-quality, context-rich datasets relevant to the domain. If they are fragmented (and they are) introduce context safeguards through standards.
• Design for representational efficiency so models don’t “learn to be fair later”; fairness and non-discrimination should be properties of the data and process, not afterthoughts.
• Treat this as a socio-technical problem: standards, governance, and participation matter as much as parameters and FLOPs.

Health commons and civic engagement

For health especially, the right “unit” of engagement is often a population health cohort—communities organised around shared problems. Think health commons: civic groups, clinicians, and researchers co-producing datasets, guardrails, and evaluation criteria. This is how we ground model knowledge in lived reality.

Regulation lens: capability, generality, and FLOPs

The EU AI Act and the GPAI Code of Practice introduce two useful levers: capability and generality. Broadly:

• Generality distinguishes reusable “foundation-like” models from purpose-built AI systems tied to a single, regulated application.
• Capability (incl. compute/FLOPs thresholds) signals when a model’s scale and potential impact make it high-risk GPAI.

Today’s big LLMs clearly land in high-risk GPAI territory. But smaller models may fall below certain thresholds while still posing meaningful bias, safety, and quality risks in deployment. That is precisely why we need standards (not just policy) that travel well across sizes: evaluation protocols, data provenance, domain-specific benchmarks, and civic sandboxes to test socio-technical fit.

This is what this all means for investors, entrepreneurs and policy makers, building on MIT's "The GenAI Divide State of AI in Business 2025" Report:

What investors should keep in mind

• Retire the old habits. High failure rates mirror past “transformation” cycles. AI won’t fix weak data, brittle workflows, or absent governance. Invest in healthy data ecosystems and standards first.
• RAG is a method, not a magic wand. Retrieval-augmented setups raise training/validation complexity and cost. The answer isn’t more glue code; it’s standards and sandboxing that make knowledge integration repeatable and testable with real populations.
• Smaller is the new small. Domain SLLMs can deliver lower investment, maintenance, and ecological footprints while solving concrete problems (e.g., protecting staff mental wellbeing in hospitals). Startups like vertical law models (e.g., Harvey) point to this trajectory.
• Follow the real AI economy. There’s often a gap between official adoption and actual worker usage. Value is produced at the edge—by people solving real work. Fund teams that co-design with end users and treat AI as a latent discovery tool that reveals problems and options, not a hammer looking for nails.

What startups—and policy-makers—should prioritise

• Be a good builder. Winners pick narrow, high-value use cases, embed deeply into workflows, and scale through continuous learning, not feature sprawl.
• Design the organisation, not just the model. Partnerships often beat solo builds. Decentralise authority with clear ownership for data, safety, and change management.
• Support the workforce through the transition. Until systems achieve reliable contextual adaptation, impact shows up as external cost optimisation more than internal restructuring. Meet real workforce needs; they hold the keys to adoption.
• Close the learning gap. The blocker isn’t model size; it’s the inability to learn from real workflows. Build feedback channels, post-deployment evaluation, and civic sandboxes that link model outputs to real outcomes.

Where this goes next

Deployment turns models into systems—and that’s where regulation actually bites. In the next RIFF, I’ll unpack “put-to-service”: deployment patterns, evidence requirements, post-market monitoring, and how SLLMs can meet (and raise) the bar for safety, equity, and performance.

If we want real AI, we need to get the foundations right: data, standards, civic participation, and models sized to the problems that truly matter.

This isn’t just the responsibility of technologists—it’s a shared challenge for investors, startups, entrepreneurs, the third sector, and policymakers alike.

At the Global Health Digital Innovation Foundation, we’re working on several initiatives to build NextGen GenAI for health and care. If you’d like to join us in shaping solutions that make a real impact, do reach out.

Stay tuned — I’ll be unpacking this in upcoming editions.

⚡Welcome to The Riff

A sharp, human-centred take on where digital health and AI are headed next—offering signal over noise, with an eye on equity, sustainability, and real-world impact.

Each edition riffs on a theme—from drift in AI systems and digital bias in healthcare, to sandboxes, standards, and smarter models of care. It’s rooted in active work across policy, ethics, and innovation ecosystems—but always grounded in people, practice, and possibility.

Whether you’re shaping the future of health systems, building technology, or asking better questions, The Riff is your lens into what’s emerging, what’s working, and what we need to talk about next.