Building Real-time Table Editor with Databricks Apps and Lakebase

Building Real-time Table Editor with Databricks Apps and Lakebase

With the recent launch of Lakebase, Databricks users can now incorporate OLTP (Online Transaction Processing) features directly into the platform, alongside its traditional strengths in data analytics and AI.

This advancement enables the creation of comprehensive data applications capable of managing transactional and analytical workloads within a single platform. I would like to explore this in fine detail in this blog post.

I've received several customer requests about creating and deploying a real-time table editor. Although it's a standard business interface, challenges arise when multiple users need to update specific cells in a table dynamically.

This post will discuss potential approaches for implementing such interfaces. The source code is available in this repository, allowing you to deploy and adapt it as a starting point.

Architecture Overview

The architecture of this real-time table editor uses a standard three-layer architecture, which is already widely adopted in many applications:

The standard three-layer model of an application

The application follows a clean separation of concerns across three primary layers:

  1. Browser Client Layer - A React-based frontend built with modern tooling including TanStack Router for routing and TanStack Table for data grid functionality

  2. Application Backend Layer - A FastAPI-powered server that handles business logic, API endpoints, and real-time event streaming

  3. Database Layer - Lakebase providing OLTP capabilities, including advanced features like triggers and notifications

This structure allows each layer to be developed independently while preserving contracts through a strongly typed API.

Realtime table editor

Tables are a specific yet standard way to display information from a database. However, many users of business applications may prefer different user interfaces, such as a single view per object. Despite this, tables remain one of the most common methods for presenting business-related data.

Below is a table, which we'll refer to as the "Dessert Inventory," that I have prepared for this application:

Dessert inventory table view.

What's cool is that we can provide users a capability to interactively edit the records of the table, for example:

Importantly, we indeed need to persist the changes into the database. There are multiple ways to do this, yet REST API is one of the most standard, convenient and type-safe approach.

Three-Model Pattern for Type Safety

To ensure type safety, we need to start with types. The best way to introduce consistent typing in Python is by using Pydantic.

However, Pydantic alone does not provide an ORM layer, which allows for the quick storage and retrieval of well-typed objects from a database.

This is where SQLModel becomes valuable, as it helps developers store well-typed objects in the database without the need to manually describe the conversion logic:

Note this code has a bit of duplication - in fact it can be done better, but sometimes in real-life applications the input, database and output model might differ. In case you have common attributes that required across all models, you can introduce a base class which will be inherited by all other models.

Having these types, we can strongly type the FastAPI responses.

Backend CRUD Implementation

The backend uses the defined models and sqlmodel session to perform database operations:

Since the types returned from FastAPI are added to the OpenAPI schema spec, we can generate a client out of them for the frontend. For this task I'm using orval:

Now in case if I change a type or add any other API route, I don't need to additionally implement a browser client - it will be auto-generated for me by orval.

Frontend Integration with Auto-Generated Clients

The frontend leverages Orval to generate type-safe API clients directly from the FastAPI OpenAPI specification:

This approach provides compile-time type checking and automatic IDE completion, significantly reducing development time and runtime errors.

Using components to display API data

Components in React are a great way to deliver consistent and reupsable UI elements. With the automatically generated frontend code and suspense capabilities, writing UI interfaces becomes a bliss.

Here is a quick example - assume we would like to display a nicely looking user profile in a sidebar:

Take a look at the sidebar in bottom right corner

We can easily add this as a feature to our application on the backend:

Orval will generate for us a convenient useProfileSuspense function, which returns strongly typed response:

The component encapsulates the loading logic, and Suspense allows us to display a nice skeleton while API it returning the response.

Real-time Updates with PostgreSQL NOTIFY and Server-Sent Events

The most compelling feature of this application is its real-time synchronization across multiple browser clients. This is achieved through a combination of PostgreSQL's NOTIFY mechanism and Server-Sent Events (SSE):

Notify from Postgres into SSE into Browser

Database-Level Triggers

Lakebase is based on Postgres, and this allows us to implement database triggers that automatically notify connected clients of any data changes:

This trigger fires automatically on any INSERT, UPDATE, or DELETE operation, capturing the operation type and relevant data.

Server-Sent Event Streaming

The FastAPI backend implements an SSE endpoint that listens for PostgreSQL notifications and streams them to connected browser clients:

This implementation efficiently handles multiple concurrent connections while providing graceful cleanup and error handling.

Frontend Real-time Integration

The React frontend creates an EventSource connection to receive real-time updates and automatically updates the UI state:

This creates a reactive user interface where changes made by one user are immediately reflected across all connected clients without any polling or manual refresh operations.

This demo shows 2 separate browser window, synchronizing the updates in less then a blink of an eye (I haven't measured precise latency but visually it appears to be <100ms):

Developer experience

I'm not a big fan of JS/TS and especially React - I find sometimes the logic quite cumbersome, and every time when I need to add a new useEffect / useState.

Fortunately in 2025 the tech stack for writing the UI part is quite advanced. In particular I found this combination quite powerful:

The power of shadcn/ui comes from the fact that components are not imported, but rather generated, so they become parts of your codebase. Some developers might find this an issue, since code is becoming duplicated.

I see this more as a benefit in modern world development, since VSCode or Cursor can work very well with the codebase, and easily pick the necessary components from it. Given this, generating a robust and nicely looking interface becomes less of a coding task, and more of a prompting task.

Since client-related code is also automatically generated via orval, there is no need to describe logical blocks on the client side, which makes prompting the Ui interfaces a simple and easy task even for developers who are not familiar with JS/TS and Reach ecosystem.

N.B. - Apps indeed support both JS/TS and Pythonic frontend frameworks, and this solution could've been entirely implemented in Python. I wanted to demostrate the robustness of full-stack development on Databricks Apps, hence the framework choice.

I've added several useful cursor rules into the repository for reference.

Summary

The combination of Databricks Apps and Lakebase represents a significant advancement in building data-centric applications. It allows users to create full-stack data-driven applications.

The classic three-layered architecture - browser client, application backend, and database - provides a clean separation of concerns while enabling sophisticated features like real-time synchronization. 

Lakebase allows developers to leverage familiar patterns like triggers and notifications, while the Databricks Platform provides the infrastructure, security, and governance needed for enterprise applications.

The full source code can be found on my GitHub, in this repository.

Please hit like, share and subscribe to me on LinkedIn - this is the best way support me as an author 🙂

Alexander Nagaitsev

Senior Software Data Engineer at Apple

6d

nice article, indeed interesting

Like
Reply
Andreas Niehaus (Terörde)

Solutions Architect at Databricks

1w

Really cool!

Youri Immerzeel

Data Engineering Capability Lead - Data Services & Foundation - FrieslandCampina

1w

Mayank Srivastava if a bizz or other nontechnical user has to update some reference metadata this is really nice!

To view or add a comment, sign in

Others also viewed

Explore topics