Data normalization & memoized denormalization

Data normalization &
memoized denormalization
Branislav Holý
1

Data normalization
● A process that transforms denormalized data to normalized data
2
const article = {
id: 'article-1',
title: 'My Awesome Article',
author: {
id: 'user-1',
name: 'John'
}
};

Data normalization
3
const article = {
id: 'article-1',
author: {
id: 'user-1',
name: 'John'
}
};
const entities = {
users: {
'user-1': {
id: 'user-1',
name: 'John'
}
},
articles: {
...
}
};

Data normalization
4
const article = {
id: 'article-1',
author: {
id: 'user-1',
name: 'John'
}
};
const entities = {
users: {
'user-1': {
id: 'user-1',
name: 'John'
}
},
articles: {
'article-1': {
id: 'article-1',
author: 'user-1'
}
}
};

Data normalization
5
const article = {
id: 'article-1',
author: {
id: 'user-1',
name: 'John'
},
viewers: [{
id: 'user-2',
name: 'Alice'
}, {
id: 'user-1',
name: 'John'
}]
};

Data normalization
6
const article = {
id: 'article-1',
author: {
id: 'user-1',
name: 'John'
},
viewers: [{
id: 'user-2',
name: 'Alice'
}, {
id: 'user-1',
name: 'John'
}]
};
const entities = {
users: {
'user-1': { ... },
'user-2': { ... }
},
articles: {
'article-1': {
id: 'article-1',
author: 'user-1',
viewers: ['user-2', 'user-1']
}
};

Should I use it?
● Yes… if your data are entities that have relations between each other
7

Advantages
● Single source of truth
8

const article = {
id: 'article-1',
author: {
id: 'user-1',
name: 'John'
},
viewers: [{
id: 'user-2',
name: 'Alice'
}, {
id: 'user-1',
name: 'John'
}]
};
Advantages
○ Denormalized data
■ name in multiple places
9

const entities = {
users: {
'user-1': {
id: 'user-1',
name: 'John'
},
'user-2': { ... }
},
articles: {
'article-1': {
id: 'article-1',
author: 'user-1',
viewers: ['user-2', 'user-1']
}
};
Advantages
○ Denormalized data
■ name in multiple places
○ Normalized data
■ name in a single place
1

Advantages
● Easy to write
○ We have to modify a single place only
11

Disadvantage
● Difficult to read
○ We cannot use:
normalizedArticle.user.name
12

Disadvantage
○ We cannot use:
○ The user field is a string!
13

Disadvantage
○ We cannot use:
○ The user field is a string!
○ But we can denormalize the article object!
… you’ll see later.
14

Normalizr
● It is a library that converts denormalized data to normalized data
● It uses schema to describe relations between entities
● https://github.com/paularmstrong/normalizr
15

How to normalize data?
● Create schemas for your entities
● Call normalize()
16

Schema
import { schema } from 'normalizr';
const user = new schema.Entity('users');
const article = new schema.Entity('articles', {
author: user,
viewers: [user]
});
17

Schema
import { schema } from 'normalizr';
const user = new schema.Entity('users');
const comment = new schema.Entity('comments', {
commenter: user
});
const article = new schema.Entity('articles', {
author: user,
comments: [comment]
});
18

normalize() function
import { normalize } from 'normalizr';
const normalizedData = normalize(denormalizedData, schema);
○ denormalizedData - nested data (API response)
○ schema - normalizr schema
19

20
const denormalizedArticles = [
{
id: 'article-1',
author: { id: 'user-1', name: 'John' },
comments: [{
id: 'comment-1',
text: 'Such a great article!',
commenter: { id: 'user-2', name: 'Alice' }
}, {
id: 'comment-2',
text: 'Thanks!',
commenter: { id: 'user-1', name: 'John' }
}]
},
{
id: 'article-2',
title: 'My Terrible Article',
comments: []
}
]

21
{
id: 'article-1',
comments: [{
id: 'comment-1',
}, {
id: 'comment-2',
text: 'Thanks!',
}]
},
{
id: 'article-2',
comments: []
}
]
normalize(denormalizedArticles, [article]);

22
{
id: 'article-1',
comments: [{
id: 'comment-1',
}, {
id: 'comment-2',
text: 'Thanks!',
}]
},
{
id: 'article-2',
comments: []
}
]
{
entities: {
users: {
'user-1': { id: 'user-1', name: 'John' },
'user-2': { id: 'user-2', name: 'Alice' }
},
comments: { ... },
articles: { ... }
},
result: ['article-1', 'article-2']
}

23
{
id: 'article-1',
comments: [{
id: 'comment-1',
}, {
id: 'comment-2',
text: 'Thanks!',
}]
},
{
id: 'article-2',
comments: []
}
]
{
entities: {
users: { ... },
comments: {
'comment-1': {
id: 'comment-1',
commenter: 'user-2'
},
'comment-2': {
id: 'comment-2',
text: 'Thanks!',
commenter: 'user-1'
}
},
articles: { ... }
},
}

24
{
id: 'article-1',
comments: [{
id: 'comment-1',
}, {
id: 'comment-2',
text: 'Thanks!',
}]
},
{
id: 'article-2',
comments: []
}
]
{
entities: {
users: { ... },
comments: { ... },
articles: {
'article-1': {
id: 'article-1',
author: 'user-1',
comments: ['comment-1', 'comment-2']
},
'article-2': {
id: 'article-2',
author: 'user-1',
comments: []
}
}
},
}

How to denormalize data?
● Use the same schemas
● Call denormalize()
○ denormalize(['article-1', 'article-2'], [article], entities);
25

○ It denormalizes the whole data structure
○ It does not memoize previously normalized data
26

○ It denormalizes the whole data structure
○ It does not memoize previously normalized data
● Use selectors!
27

Reselect
● A selector is a function that gets data from a state
○ const selector = state => state.name; // a simple selector without reselect
● Selectors can compute derived data from a state
○ They keep a single source of truth
● Selectors (with Reselect) are efficient (memoized)
○ They are not recomputed unless one of their arguments is changed
● Selectors (with Reselect) are composable
○ They can be used as input to other selectors
28

Reselect
● The selectors created with
createSelector are memoized
● The first arguments of
createSelector are other
selectors
● The last argument is a function which
has results of the previous selectors
in its input arguments
29
import { normalize } from 'normalizr';
const appState = {
counter: 0,
user: { name: 'John' }
};
const getCounter = state => state.counter;
const getUser = state => state.user;
const getName = createSelector(
getUser,
user => user.name
);
const getUpperCaseName = createSelector(
getUserName,
userName => userName.toUpperCase()
);
const getNameWithCounter = createSelector(
getUpperCaseName,
getCounter,
(userName, num) =,> `${userName} ${num}`
);

Selectors for Users
30
{
entities: {
users: {
'user-1': { id: 'user-1', name: 'John' },
'user-2': { id: 'user-2', name: 'Alice' }
},
comments: { ... },
articles: { ... }
},
}
const getUsers = entities => entities.users;

Selectors for Comments
31
const getNormalizedComments = entities =>
entities.comments;
{
entities: {
users: { ... },
comments: {
'comment-1': {
id: 'comment-1',
commenter: 'user-2'
},
'comment-2': {
id: 'comment-2',
text: 'Thanks!',
commenter: 'user-1'
}
},
articles: { ... }
},
}

Selectors for Comments
const getComments = createSelector(
getNormalizedComments,
getUsers,
(normalizedComments, users) => Object.keys(normalizedComments).reduce(
(comments, commentId) => {
const normalizedComment = normalizedComments[commentId];
const userId = normalizedComment.commenter;
return {
...comments,
[commentId]: {
...normalizedComment,
commenter: users[userId]
}
};
},
{}
)
);
32

Selectors for Articles
33
entities.comments;
const getComments = createSelector(...);
const getNormalizedArticles = entities =>
entities.articles;
{
entities: {
users: { ... },
comments: { ... },
articles: {
'article-1': {
id: 'article-1',
author: 'user-1',
comments: ['comment-1', 'comment-2']
},
'article-2': {
id: 'article-2',
author: 'user-1',
comments: []
}
}
},
}

Selectors for Articles
const getArticles = createSelector(
getNormalizedArticles,
getComments,
getUsers,
(normalizedArticles, comments, users) => Object.keys(normalizedArticles).reduce(
(articles, articleId) => {
const normalizedArticle = normalizedArticles[articleId];
const userId = normalizedComment.commenter;
return {
...articles,
[articleId]: {
...normalizedArticle,
author: users[userId],
comments: normalizedArticle.comments.map(commentId => comments[commentId])
}
};
},
{}
));
34

Selectors
35
entities.comments;
const getComments = createSelector(...);
const getNormalizedArticles = entities =>
entities.articles;
const getArticles = createSelector(...);
const normalizedData = {
entities: {
users: { ... },
comments: { ... },
articles: { ... }
},
};
const denormalizedArticles = getArticles(
normalizedData.entities
);
const denormalizedData = normalizedData.result
.map(articleId => denormalizedArticles[articleId]);

36
Thank you!
Don’t forget to use…

37
Thank you!

Advanced normalizr features
41
● Custom ID attribute
● Process strategy
● Merge strategy

Custom ID attribute
42
● User object: { userId: 'user-1', name: 'John' }

Custom ID attribute
43
● User object: { userId: 'user-1', name: 'John' }
● Schema:
const user = new schema.Entity('users', {}, {
idAttribute: 'userId'
});

Custom ID attribute for M:N
44
[
{
id: 'user-1',
name: 'John',
skills: [
{ level: 10, skill: { id: 'skill-1', name: 'JavaScript' } },
{ level: 1, skill: { id: 'skill-2', name: 'C++' } }
]
},
{
id: 'user-2',
name: 'Alice',
skills: [
{ level: 8, skill: { id: 'skill-1', name: 'JavaScript' } }
]
}
]

45
const skill = new schema.Entity('skills');
const userSkill = new schema.Entity(
'userSkills',
{ skill },
{
idAttribute: (value, parent) => `${value.skill.id}-${parent.id}`
}
);
const user = new schema.Entity('users', {
skills: [userSkill]
});

46
{
skills: {
'skill-1': {
id: 'skill-1',
name: 'JavaScript'
},
'skill-2': {
id: 'skill-2',
name: 'C++'
}
},
userSkills: { ... },
users: { ... }
},
result: ['user-1', 'user-2']
}

47
{
skills: { ... },
userSkills: {
'skill-1-user-1': { level: 10, skill: 'skill-1' },
'skill-1-user-2': { level: 8, skill: 'skill-1' }
},
users: { ... }
},
}

48
{
skills: { ... },
users: {
'user-1': {
id: 'user-1',
name: 'John',
skills: ['skill-1-user-1', 'skill-2-user-1']
},
'user-2': {
id: 'user-2',
name: 'Alice',
skills: ['skill-1-user-2']
}
}},
}

Process strategy
49
● Pre-process entities before the normalization
● Modify the shape of entities
● Add additional data

Process strategy
50
[
{
id: 'user-1',
name: 'John',
skillLevels: [10, 1],
skills: [
{ id: 'skill-1', name: 'JavaScript' },
{ id: 'skill-2', name: 'C++' }
]
},
{
id: 'user-2',
name: 'Alice',
skillLevels: [8],
skills: [{ id: 'skill-1', name: 'JavaScript' }]
}
]

Process strategy
51
const skill = new schema.Entity('skills');
const userSkill = new schema.Entity('userSkills', { skill }, { idAttribute });
const user = new schema.Entity(
'users',
{ skills: [userSkill] },
{
processStrategy: ({ id, name, skillLevels, skills }) => ({
id,
name,
skills: skills.map((skill, index) => ({
level: skillLevels[index],
skill
}))
})
}
);

Process strategy
52
{
skills: { ... },
userSkills: {
'skill-1-user-2': { level: 8, skill: 'skill-1' }
},
users: { ... }
},
}

Merge strategy
53
● Merge entities with the same IDs

Merge strategy
54
{
skills: {
'skill-1': {
id: 'skill-1',
name: 'JavaScript'
},
'skill-2': {
id: 'skill-2',
name: 'C++'
}
},
users: { ... }
},
}
We want user IDs here.
We want user IDs here.

Merge strategy
55
const user = new schema.Entity(
'users',
{ skills: [userSkill] },
{
processStrategy: ({ id, name, skillLevels, skills }) => ({
id,
name,
skills: skills.map((skill, index) => ({
level: skillLevels[index],
skill: {
...skill,
users: [id]
}
}))
})
}
);

Merge strategy
56
const skill = new schema.Entity(
'skills',
{},
{
mergeStrategy: (entityA, entityB) => ({
...entityA,
...entityB,
users: [...entityA.users, ...entityB.users]
})
}
);

Merge strategy
57
{
entities: {
skills: {
'skill-1': {
id: 'skill-1',
name: 'JavaScript',
users: ['user-1', 'user-2']
},
'skill-2': {
id: 'skill-2',
name: 'C++',
users: ['user-1']
}
},
users: { ... }
},
}

58
Thank you!

59
Thank you!

Data normalization & memoized denormalization

More Related Content

What's hot (18)

Similar to Data normalization & memoized denormalization (20)

Recently uploaded (20)