Implementing Schema Validation in MongoDB with Pydantic

Samuel Folasayo
MongoDB Schema Validation with Pydantic
Data Governance, Integrity and Consistency with Pydantic & MongoDB
Joe Nyirenda

What is MongoDB Schema Validation?
Ensures data complies with a predeﬁned set of rules(data types etc.)
Maintains data integrity
Helps enforces data governance
Can help optimise schema size
It can be applied when creating or updating collections.

What is Pydantic?
Python library used for data validation and parsing
Enforces data types and constraints
Deﬁning models based on Python type annotations
Provide error handling
Provides conversion between different data formats, like JSON or
dictionaries

Pros and Cons of Pydantic + Schema Validation
Pros
Enforces data validity before storing
Ensures code reliability
Double-layer validation
Greater security
Data integrity
Robust validation
Cons
Potential overhead
Complexity in setup
Inﬂexible schema

Conﬁguring Schema Validation
Can be performed at collection
creation time or to an already existing
collection
db.createCollection ("users", {
validator: {
$jsonSchema : {
bsonType: "object",
required: ["name", "email"],
properties: {
name: {
bsonType: "string",
description: "must be a string and is required" ,
maxLength: 50
},
email: {
bsonType: "string",
description: "must be a string and is required" ,
pattern: "^S+@S+.S+$" // Enforces valid
email pattern
},
age: {
bsonType: "int",
description: "must be an integer and optional" ,
minimum: 18,
maximum: 120
}
}
}
},
validationAction: "error"
})

Apply Schema to Existing Collection
Use collMod command to update
schema validation rules if collection
exists
db.runCommand({
collMod: "users",
validator: {
$jsonSchema: {
bsonType: "object",
required: ["name", "email"],
properties: {
name: {
bsonType: "string",
description: "must be a string and is required",
maxLength: 50
},
email: {
bsonType: "string",
description: "must be a string and is required",
pattern: "^S+@S+.S+$"
},
age: {
bsonType: "int",
description: "must be an integer and optional",
minimum: 18,
maximum: 120
}
}
}
},
validationAction: "error"
})

After applying the schema, try inserting documents to see how MongoDB enforces
the validation rules.
Testing the Schema
db.users.insertOne({
name: "John Doe",
email: "john@example.com",
age: 30
})
This will work because it complies with the schema.

MongoDB enforces the validation rules
Testing the Schema: invalid document
Invalid Document Insertion:
db.users.insertOne({
name: "John Doe",
email: "john@example.com",
age: 17
})
This will fail because age is less than 18.

Demo: minimalist web form
Tools:
FastAPI: Python web framework
for building APIs
Motor: Asynchronous MongoDB
driver.
Pydantic: Validating and parsing
data.
Jinja2: HTML templating engine

Deﬁne the structure and validation rules for incoming data
Example for a user model:
Deﬁning Pydantic Models
class UserModel(BaseModel):
name: str = Field(..., max_length=50)
email: EmailStr
age: Optional[int]= Field(None, ge=18, le=120)

Motor connects asynchronously to MongoDB:
Connecting to MongoDB
from motor.motor_asyncio import AsyncIOMotorClient
from fastapi import FastAPI, HTTPException
app = FastAPI()
# MongoDB connection
client = AsyncIOMotorClient("mongodb://localhost:27017")
db = client.mydatabase
users_collection = db.users

Here, we’ll create two main routes:
GET route to serve the form HTML.
Building Routes
from fastapi import Request, Form
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
# Initialize Jinja2 template rendering
templates = Jinja2Templates(directory="templates")
@app.get("/", response_class=HTMLResponse)
async def get_form(request: Request):
return templates.TemplateResponse("form.html", {"request": request})

post route to handle form submission,
Validate the data using the Pydantic model
Save validated data to MongoDB
Building Routes
@app.post("/submit")
async def submit_form(
request: Request,
name: str = Form(...),
email: str = Form(...),
age: Optional[int] = Form(None)
):
try:
# Validate the data using the Pydantic model
user = UserModel(name=name, email=email, age=age)
# Insert into MongoDB
await users_collection.insert_one(user.dict())
# Success response
return templates.TemplateResponse("success.html", {"request":
request, "user": user.dict()})
except Exception as e:
raise HTTPException(status_code=400, detail=str(e))

Handling Validation Errors in Code
from fastapi import FastAPI, HTTPException
from motor.motor_asyncio import AsyncIOMotorClient
from pydantic import BaseModel, EmailStr, Field
app = FastAPI()
client =
AsyncIOMotorClient("mongodb://localhost:27017")
db = client.mydatabase
users_collection = db.users
class UserModel(BaseModel):
name: str = Field(..., max_length=50)
email: EmailStr
age: Optional[int] = Field(None, ge=18, le=120)
@app.post("/submit")
async def submit_user(user: UserModel):
try:
await users_collection.insert_one(user.dict())
return {"message": "User inserted successfully"}
except Exception as e:
raise HTTPException(status_code=400,
detail=str(e))

Form Template (templates/form.html):
Displaying HTML Form
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0"
>
<title>User Form</title>
</head>
<body>
<h1>Enter User Information
</h1>
<form action="/submit" method="post">
<label for="name">Name:</label>
<input type="text" id="name" name="name" required><br><br>
<label for="email">Email:</label>
<input type="email" id="email" name="email" required><br><br>
<label for="age">Age:</label>
<input type="number" id="age" name="age"><br><br>
<input type="submit" value="Submit">
</form>
</body>
</html>

Success Template (templates/success.html):
Displaying HTML Form
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Form Submitted</title>
</head>
<body>
<h1>Submission Successful</h1>
<p>User Details:</p>
<ul>
<li><strong>Name:</strong> {{ user.name }}</li>
<li><strong>Email:</strong> {{ user.email }}</li>
<li><strong>Age:</strong> {{ user.age }}</li>
</ul>
</body>
</html>

Conclusion
Pydantic provides an extra layer of validation
Schema validation enforces data type and domain speciﬁc rules
for data model
Schema validation great for mature applications

Implementing Schema Validation in MongoDB with Pydantic

More Related Content

Similar to Implementing Schema Validation in MongoDB with Pydantic (20)

More from techprane (17)

Recently uploaded (20)

Implementing Schema Validation in MongoDB with Pydantic