💊 DATA Pill #167 - Durable AI Loops, Flink Agents, TDD with dbt, S3 Vectors

Hi,

This week’s picks deliver real-time agents, crash-proof AI workflows, and smarter data engineering with dbt, Flink, and vector-native S3.

ARTICLE

AI Agents Must Act, Not Wait: A Case for Event-Driven Multi-Agent Design | 6 min | AI | Sean Falconer, Andrew Sellers | Personal Blog

A case for building reactive, event-driven multi-agent systems instead of static prompt chains.

In MORE LINKS you will read:

• Durable AI Loops: Fault Tolerance across Frameworks and without Handcuffs

{ MORE LINKS }

TUTORIALS

Test Driven Development (TDD) with dbt: Test First, SQL Later | 5 min | Data Engineering | Dumky de Wilde | Xebia Blog

Write tests before models and catch logic errors early.

In MORE LINKS you will read:

• Data pipeline troubleshooting: Root cause analysis through lineage metadata
• Build a Data Lakehouse with Apache Iceberg, Polaris, Trino & MinIO
• Snowflake to BigQuery migration - introduction

{ MORE LINKS }

TOOL

Apache Flink Agents | Agentic AI

Build fault-tolerant, long-running AI agents directly on Flink using native state and streaming.

In MORE LINKS you will read:

• Amazon S3 Vectors

{ MORE LINKS }

DATA TUBE

The slow death of scaling and what comes next | 1 h 2 min | ML | Sara Hooker | Cohere

Sara Hooker explores the limits of scale in machine learning and what’s coming next for open research and efficient models.

In MORE LINKS you will watch:

• AI prompt engineering in 2025: What works and what doesn’t

{ MORE LINKS }

PINNACLE PICKS

Your last week top picks:

Embedding User-Defined Indexes in Apache Parquet Files | 7 min | Data Engineering | Qi Zhu, Jigao Luo, Andrew Lamb | Apache DataFusion Blog

DataFusion introduces custom Parquet indexing for faster queries on large datasets.

NVIDIA Says Small Language Models Are The Future of Agentic AI | SLM | 5 min | Cobus Greyling | Personal Blog

Small models are faster, safer, and better suited for real-time AI. NVIDIA explains why they may outpace large LLMs in practical applications

The Agent Factory - Episode 2: Multi-Agent Systems, Concepts & Patterns | 23 min | Gen AI | Vlad Kolesnikov, Shir Meir Lador | Google Cloud Tech

Vlad Kolesnikov and Shir Meir Lador explain how to design collaborative agents using swarms, supervisors, and context engineering.

____________________

Have any interesting content to share in the DATA Pill newsletter?

➡ Join us on GitHub

➡ Dig previous editions of DataPill

Adam from the GetInData is Now Xebia