Spring Data y Mongo DB en un proyecto Real

Spring Data y MongoDB
en un proyecto real.

David Gómez
@dgomezg

sábado 18 de febrero de 12

David Gómez
rchitect & Trainer
Sw A mas.com
siste
extrema-
dgomez@
zg
@dgome


Spring CORE

Rich Web Applications
with Spring

Enterprise Integration


#springio
#extrema


Hablaremos de

Un caso real
¿Por qué NoSQL? ¿Por qué MongoDB?
MongoDB ‘a pelo’
Spring Data al rescate
¡Cuidado!


Un caso real.
Control de tráﬁco marítimo


Emisor AIX
3-5 min


Receptor GPS

Radar (3 s)
Goniometro

Estación Meteo,
Emisor AIX
Cámaras,
3-5 min
...


El problema: La escalabilidad


Por cada barco:
Recepción y procesamiento de señales (nº)
Transacciones,
Cálculo de alarmas
Gestión de Históricos


Señales
(1 AIS x 3 min + 1 radar x 3 seg) x 100 barcos
Alarmas
Persistencia de Líneas de Varada
(10.000+ ptos x 100+ líneas)
40 Alarmas
Zonas, Elementos Navegación
Persistencia: Total: 87 tablas + históricos(x2)


¿Por qué NoSQL?

Teorema CAP (Eric Brewer)

Source: Nathan Hurst’s Blog


¿Por qué NoSQL?


Consistency

C



¿Por qué NoSQL?


Consistency Availability

C A



¿Por qué NoSQL?



C A

P
Partition Tolerance



¿Por qué NoSQL?



C A
Sólo
2

P
Partition Tolerance



¿Por qué NoSQL?



A
Oracle, MySQL,
C
Sólo
2

P
Partition Tolerance



¿Por qué NoSQL?



A
Oracle, MySQL,
C
Sólo
M Hyp is, M
on er em

2
go Ta c
Re

DB ble ac
d

, D , H heD
at Ba B
aS se
to
re
,

P
Partition Tolerance



¿Por qué NoSQL?
RDBMS
Orientadas a documento
Teorema CAP (Eric Brewer) orientadas a columna
Key-Value


A
Oracle, MySQL,
C
Sólo
M Hyp is, M

,
on er em

uc ra, ort
2

aK
go Ta c

Co nd m
Re

Ri
DB ble ac

B, a de
d

B,
D ass ol
, D , H heD

hD
ple C o,V
at Ba B

am
aS se
to

yn
re

D
,

P
Partition Tolerance Sim



Pero... ¿NoSQL?

(cc) Photo by mwin - http://www.ﬂickr.com/photos/wingler/3429634150


Pero... ¿NoSQL?

NoSQL !== no SQL
Not only SQL

(cc) Photo by mwin - http://www.ﬂickr.com/photos/wingler/3429634150


NoSQL: ACID vs BASE

Atomicity, Basically
Available,
Consistency,
Soft state,
Isolation,
Eventual
Durability consistency

(cc) Photo by zhouxuan12345678 - http://www.ﬂickr.com/photos/53921113@N02/5645102295/


Tipos.

orientadas a Columna orientadas a Documento

Key-value orientadas a Grafos


¿Por qué MongoDB?

Orientada a documento
Almacenados en Colecciones
(bajo una clave)
Los documentos pueden ser heterogéneos

Formatos textuales: JSON & BSON

(cc) Photo by Ampersand Duck - http://www.ﬂickr.com/photos/ampersandduck/4941185476


MongoDB

Escrita en C++
Orientada a documento
Formato JSON (o BSON)
Un poco SQL (queries, índices, Referencias
externas)
Particionado horizontal (sharding)
Consultas Javascript

(cc) Photo by Ampersand Duck - http://www.ﬂickr.com/photos/ampersandduck/4941185476


Orientadas a Documento

Documentos formateados en JSON


Orientadas a Documento

Documentos formateados en JSON
{ "_id" : "224000999",
"_class" : "com.vts.model.Vessel",
"flag" : "ALBANIA",
"name" : "Sample NOT_AVAILABLE Vessel 224000999",
"callsign" : "SV224000999",
"toBow" : 25,
"toStern" : 5,
"toPort" : 5,
"toStarboard" : 5,
"comments" : "Sample vessel created automatically for test purposes"
}


El API

Consola y API JavaScript:


El API

beleriand:bin dgomez$ ./mongo
MongoDB shell version: 1.8.1
connecting to: test
>


El API

connecting to: test
> use vts
switched to db vts
>


El API

connecting to: test
> use vts
switched to db vts
> show collections
>
Event
WeatherData
system.indexes
vessel
>


El API

connecting to: test
> use vts
switched to db vts
> show collections
>
Event
WeatherData
system.indexes
vessel
> db.Event.find()
>
{ "_id" : ObjectId("4e0b5b211446446f6be3bb1a"), "_class" :
"com.vts.model.events.SystemEvent", "timestamp" :
ISODate("2011-06-29T17:04:33.039Z") }
{ "_id" : ObjectId("4e0b5b3d144676f49946443f"), "_class" :
"com.vts.model.events.SystemEvent", "timestamp" :
ISODate("2011-06-29T17:05:01.394Z") }
>


El API

> show collections
Event
WeatherData
system.indexes
vessel
>


El API

> show collections
Event
WeatherData
system.indexes
vessel
> db.alarms.save({"type" : "speed", "severity" : "warn", "vesselSSID" :
>
"224000999"})
>


El API

> show collections
Event
WeatherData
system.indexes
vessel
>
"224000999"})
> show collections
>
Event
WeatherData
alarms
system.indexes
vessel
>


El API

> show collections
Event
WeatherData
system.indexes
vessel
>
"224000999"})
> show collections
>
Event
WeatherData
alarms
system.indexes
vessel
> db.alarms.find()
>
{ "_id" : ObjectId("4f3acca3ed4cc8078fd1a4be"), "type" : "speed", "severity" :
"warn", "vesselSSID" : "224000999" }
>


Documentos

Identiﬁcados por un “_id” (generado o asignado)
Textual (JSON)
Binario (BSON)*
Documentos grandes divididos en
chunks


Programación de clientes

Multitud de lenguajes (drivers)
Transformación a JSON realizada por el driver




{ "_id" : "224000999",
"flag" : "ALBANIA",
"name" : "Sample NOT_AVAILABLE Vessel 224000999",
"callsign" : "SV224000999",
"toBow" : 25,
"toStern" : 5,
"toPort" : 5,
"toStarboard" : 5,
"comments" : "Sample vessel created automatically for test purposes" }

sample.py




{ "_id" : "224000999",
{ "_id" => "224000999",
"flag""flag" => "ALBANIA",
: "ALBANIA",
"name""name" => "Sample NOT_AVAILABLE 224000999",
: "Sample NOT_AVAILABLE Vessel Vessel 224000999",
"callsign" : "SV224000999",
"callsign" => "SV224000999",
"toBow" : 25, => 25,
"toBow"
"toStern" : 5, => 5,
"toStern"
"toPort" : 5, => 5,
"toPort"
"toStarboard" : 5, => 5,
"toStarboard"
"comments" : "Sample"Sample createdcreated automatically for test purposes" }
"comments" => vessel vessel automatically for test purposes" }

sample.py Ruby




{ "_id" : "224000999",
{ "_id" => "224000999", "224000999",
array( "_id" =>
"flag""flag" "flag" => "ALBANIA",
: "ALBANIA",
=> "ALBANIA",
"name""name" "name" => "Sample NOT_AVAILABLE Vessel 224000999",
: "Sample"Sample NOT_AVAILABLE 224000999",
=> NOT_AVAILABLE Vessel Vessel 224000999",
"callsign" : "callsign" => "SV224000999",
"callsign" => "SV224000999",
"SV224000999",
"toBow" : 25, => 25, => 25,
"toBow""toBow"
"toStern" : 5, => 5, => 5,
"toStern"
"toStern"
"toPort" : 5, => 5, => 5,
"toPort"
"toPort"
"toStarboard" : 5, => 5, => 5,
"toStarboard"
"toStarboard"
"comments" : "comments" => "Sample vessel created automatically for test purposes"}
"comments" => "Sample createdcreated automatically for test purposes" }
"Sample vessel vessel automatically for test purposes" }

sample.py sample.php
Ruby




BasicDBObject doc = new BasicDBObject();
{ "_id" : "224000999",
{ "_id" => "224000999", "224000999",
array( "_id" => doc.put("flag", vessel.getFlag());
"flag""flag" "flag" => "ALBANIA",
: "ALBANIA",
=> "ALBANIA",doc.put("name", vessel.getName());
"name""name" "name" => "Sample NOT_AVAILABLE Vessel 224000999",
: "Sample"Sample NOT_AVAILABLE 224000999",
=> NOT_AVAILABLE Vessel Vessel 224000999",
doc.put("callsign", vessel.getCalSign());
"callsign" : "callsign" => "SV224000999",
"callsign" => "SV224000999",
"SV224000999",
doc.put("toBow", vessel.getBow());
"toBow" : 25, => 25, => 25,
"toBow""toBow"
"toStern" : 5, => 5, => 5,
"toStern"
"toStern" doc.put("toPort", vessel.getStern());
"toPort" : 5, => 5, => 5,
"toPort"
"toPort" doc.put("toStern", vessel.getPort());
"toStarboard" : 5, => 5, doc.put("toStartboard", vessel.getStarboard());
"toStarboard"
"toStarboard" => 5,
"comments" : "comments" => doc.put("comments", vessel.getComments()); purposes"}
"comments" => "Sample createdcreated automatically for test purposes" }
"Sample vessel vessel vessel created automatically for test
"Sample automatically for test purposes" }

sample.py sample.php
Ruby
sample.java


Cliente Java con
MongoDB
‘a pelo’


1. Obtener el Driver

Importar el driver


1. Obtener el Driver

Importar el driver

<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongo-java-driver</
artifactId>
<version>2.7.0-rc1</version>
</dependency>


2. Conexión

Obtener la referencia a la BD y la conexión


2. Conexión

Obtener la referencia a la BD y la conexión

Mongo m = new Mongo( "127.0.0.1" , 27017 );
DB db = m.getDB( "playground" );


3. Inserciones
Crear e insertar un objeto


3. Inserciones
Crear e insertar un objeto
BasicDBObject doc = new BasicDBObject();
doc.put("flag", vessel.getFlag());
doc.put("name", vessel.getName());
doc.put("callsign", vessel.getCalSign());
doc.put("toBow", vessel.getBow());
doc.put("toPort", vessel.getStern());
doc.put("toStern", vessel.getPort());
doc.put("toStartboard", vessel.getStarboard());
doc.put("comments", vessel.getComments());

DBCollection coll = db.getCollection("vessel");
coll.insert(doc);


4. Consultas (I)

Obtener todos los documentos de una colección


4. Consultas (I)

DBCollection vessels = db.getCollection("vessel");

DBCursor cursor = vessels.find();

List<Vessel> vessels = new ArrayList<>();
while (cursor.hasNext()) {
DBObject jsonVessel = cursor.next();
Vessel vessel = new Vessel();
vessel.setMmsi(jsonVessel.get(“mmsi”));
vessel.setLength(jsonVessel.get(“length”));
// ...

vessels.add(vessel);
}


4. Consultas (I)


DBCursor cursor = vessels.find();

List<Vessel> vessels = new ArrayList<>();
while (cursor.hasNext()) { class DBCollection {
public abstract
DBObject jsonVessel = cursor.next();
public final DBCursor find(){
Vessel vessel = new Vessel();
public final DBCursor find( DBObject ref )
vessel.setMmsi(jsonVessel.get(“mmsi”));
public final DBCursor find( DBObject ref , DBObject keys )
vessel.setLength(jsonVessel.get(“length”));
// ...
public final DBObject findOne( Object obj )
vessels.add(vessel); DBObject findOne( Object obj, DBObject fields )
public final
}
public DBObject findAndModify( DBObject query , DBObject update )

}


4. Consultas (II)
Consultas “SQL-like”


4. Consultas (II)
En RDBMS:
SELECT * FROM VESSEL WHERE FLAG = “ALBANIA”;

Las consultas se hacen también en JSON


4. Consultas (II)
En RDBMS:

{ “flag” : “ALBANIA” }


4. Consultas (II)
En RDBMS:


connecting to: test
> use vts
switched to db vts
> db.vessel.find( { "flag" : "ALBANIA" });
{ "_id" : "224312999", "_class" : "com.vts.model.Vessel", "flag" : "ALBANIA",
"name" : "Sample TOWING Vessel 224312999", "callsign" : "SV224312999", “length”:
30, “beam” : “10”, "toBow" : 25, "toStern" : 5, "toPort" : 5, "toStarboard" : 5,
>


4. Consultas (II)
En RDBMS:


connecting to: test
> use vts
switched to db vts DBObject query = new BasicDBObject();
query.put("flag", "ALBANIA");
> db.vessel.find( { "flag" : "ALBANIA" });
{ "_id" : "224312999", "_class" : "com.vts.model.Vessel", "flag" : "ALBANIA",
DBCursor albanianShips = vessels.find(query);
"name" : "Sample TOWING Vessel 224312999", "callsign" : "SV224312999", “length”:
30, “beam” : “10”, "toBow" : 25, "toStern" : 5, "toPort" : 5, {
while (albanianShips.hasNext()) "toStarboard" : 5,
// ...
> }


Operadores de consulta

Otros operadores


Operadores de consulta

Otros operadores
$all
$and
$exists
$gt
$in
$lt
$mod
$ne
$nin
$nor
$or
$size
$type


Consultas complejas
Concatenación de consultas
RDBMS:


Consultas complejas
SELECT * FROM VESSELS WHERE length < 90 AND length > 30
RDBMS: AND (flag = “ALBANIA” or flag = ”SAN MARINO”);


Consultas complejas

{
“length” : { $lt : 90, $gt : 30},
Mongo: $or : [
{ “flag” : “ALBANIA”},
{ “flag” : “SAN MARINO”} ]
}


Consultas complejas

{
“length” : { $lt : 90, $gt : 30},
Mongo: $or : [
{ “flag” : “ALBANIA”},
{ “flag” : “SAN MARINO”} ]
}

DBObject query = new BasicDBObject();
DBObject condition = new BasicDBObject("$lt",90);
condition.put("$gt",30);
query.put("length", condition);

DBObject [] flags = {
Mongo new BasicDBObject("flag", "ALBANIA"),
new BasicDBObject("flag", "SAN MARINO")
};
Driver: query.put("$or", flags);

talks.find(query);


DBObjects desde JSON

com.mongo.util.JSON


DBObjects desde JSON

com.mongo.util.JSON
Object query = JSON.parse("{ 'length' : { $lt : 90, $gt : 30}, " +
'$or' : [ { 'flag' : 'ALBANIA}, " +
"
" { 'flag' : 'SAN MARINO'} ] }");

DBCursor cursor = db.find((DBObject) query);


Además...

Gestión de excepciones
Gestión recursos
DB, Collección, Cursores, DBObjects...

Transformaciones Objeto JSON


Spring Data MongoDB


Spring Data
Objetivo:
Proporcionar un mecanismo homogéneo y
completo para el acceso a BD NoSQL
Multiples proyectos:
Soportados:
MongoDB, Redis, Neo4j, Hadoop, GemFire,
RiaK
En proceso:
CouchDB, Hbase, Cassandra


Spring Data y MongoDB

XML namespace para conﬁgurar driver Mongo
MongoTemplate
Conversión de excepciones automática
Conversión Conﬁgurable
JMX monitoring


Importando Spring Data


Importando Spring Data

<repository>
<id>spring-release</id>
<name>Spring Maven Release Repository</name>
<url>http://repo.springsource.org/libs-release</url>
</repository>

<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-mongodb</artifactId>
<version>1.0.1.RELEASE</version>
</dependency>


Namespace


Namespace
<?xml
version="1.0"
encoding="UTF-‐8"?>
<beans
xmlns="http://www.springframework.org/schema/beans"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-‐instance"

xmlns:mongo="http://www.springframework.org/schema/data/mongo"

xmlns:context="http://www.springframework.org/schema/context"

xsi:schemaLocation="http://www.springframework.org/schema/beans

http://www.springframework.org/schema/beans/spring-‐beans.xsd

http://www.springframework.org/schema/data/mongo

http://www.springframework.org/schema/data/mongo/spring-‐mongo-‐1.0.xsd

http://www.springframework.org/schema/context

http://www.springframework.org/schema/context/spring-‐context.xsd">

</beans>


Namespace
<?xml
version="1.0"
<beans










<mongo:db-‐factory
dbname="vts"/>

</beans>


Namespace
<?xml
version="1.0"
<beans










<mongo:db-‐factory
dbname="vts"/>

<bean
id="mongoTemplate"
class="org.springframework.data.mongodb.core.MongoTemplate">

<constructor-‐arg
name="mongoDbFactory"
ref="mongoDbFactory"
/>

</bean>

</beans>


MongoTemplate
<bean
id="mongoTemplate"

<constructor-‐arg
name="mongoDbFactory"
ref="mongoDbFactory"
/>
</bean>

Identiﬁca la colección
Convierte query a JSON
Convierte respuesta (“_class”)
Gestiona las conexiones y cursores
Convierte las excepciones


MongoTemplate - Consultas



public class MongoTemplate implements MongoOperations, ApplicationContextAware {

public <T> T findOne(Query query, Class<T> entityClass)

public <T> List<T> find(Query query, Class<T> entityClass)

public <T> T findById(Object id, Class<T> entityClass)

public <T> List<T> findAll(Class<T> entityClass)

}




public <T> T findOne(Query query, Class<T> entityClass)

public <T> List<T> find(Query query, Class<T> entityClass)

public <T> T findById(Object id, Class<T> entityClass)

public <T> List<T> findAll(Class<T> entityClass)

}
public <T> T findOne(Query query, Class<T> entityClass, String collectionName)
....


Consultas
RDBMS: SELECT * FROM VESSEL WHERE FLAG = “ALBANIA”;


Consultas

Mongo: { “flag” : “ALBANIA” }


Consultas



query.put("flag", "ALBANIA");

Mongo DBCursor albanianShips = vessels.find(query);
while (albanianShips.hasNext()) {
Driver: DBObject result = albanianShips.next();
// Create Vessel object from DBObject
}


Consultas


Query query = new Query(Criteria.where("flag").is("ALBANIA");

List<Vessel> vessels = mongoTemplate.find(query, Vessel.class);

Utilizado para
Identiﬁcación de la colección
Serialización JSON


Serialización automática


Serialización automática

{
"_id"
:
"224000999",

"_class"
:
"com.vts.model.Vessel",

"flag"
:
"ALBANIA",

"name"
:
"Sample
NOT_AVAILABLE
Vessel
224000999",

"callsign"
:
"SV224000999",

"toBow"
:
25,

"toStern"
:
5,

"toPort"
:
5,

"toStarboard"
:
5,

"comments"
:
"Sample
vessel
created
automatically
for
test
purposes"
}


Consultas complejas


Consultas complejas
ORDER BY lastUpdate;


Consultas complejas

{
“length” : { $lt : 90, $gt : 30}
}


Consultas complejas

{
“length” : { $lt : 90, $gt : 30}
}

DBObject condition = new BasicDBObject("$lt",90);
condition.put("$gt",30);
query.put("length", condition);

DBObject [] flags = {
new BasicDBObject("flag", "ALBANIA"),
new BasicDBObject("flag", "SAN MARINO")
};
query.put("$or", flags);

vessels.find(query);


Consultas complejas

{
“length” : { $lt : 90, $gt : 30}
}

Criteria criteria = Criteria.where("length");
criteria.gte(from);
criteria.lte(to);

Query query = new Query(criteria);
query.sort().on("lastUpdate", Order.ASCENDING);

List<Vessel> vessels = mongoTemplate.find(query, Vessel.class);


MongoTemplate - inserciones


MongoTemplate - inserciones


public void save(Object objectToSave)

public void save(Object objectToSave, String collectionName)

}


AbstractRepository


AbstractRepository

public abstract class AbstractMongoRepository<T> {

@Inject
protected MongoTemplate mongoTemplate;

/** Clase que utiliza Mongo para identificar el nombre de la coleccion */
private Class<T> persistentClass;

protected AbstractService(Class<T> persistentClass) {
this.persistentClass = persistentClass;
}

public String determineCollection() {
return persistentClass.getSimpleName();
}
}


AbstractRepository
public abstract class AbstractMongoRepository<T> {

public void save(T e) {
mongoTemplate.save(e, determineCollection());
}

public void save(Collection<T> entities) {
for (T e : entities) {
save(e);
}
}

public void removeById(String id) {
Object e = findById(id);
if (e != null) {
mongoTemplate.remove(e, determineCollection());
}
}

public T findById(String id) {
T t = mongoTemplate.findById(id, persistentClass);
if (t == null) {
throw new ObjectNotFoundException(persistentClass, id);
}
return t;
}
}


VesselRepository


VesselRepository

public class VesselRepository extends AbstractMongoRepository<Vessel> {

public VesselRepository() {
super(Vessel.class);
}

}


A tener en cuenta


El caso de BitSet
Cuidado con los campos transient


El caso de BitSet
Cuidado con los campos transient
public class BitSet implements Cloneable, java.io.Serializable {

/** The internal field corresponding to the serialField "bits". */
*/
private long[] words;

/** The number of words in the logical size of this Bitset */
private transient int wordsInUse = 0;

private void expandTo(int wordIndex) {
int wordsRequired = wordIndex+1;
if (wordsInUse < wordsRequired) {
ensureCapacity(wordsRequired);
wordsInUse = wordsRequired;
}
}
}


Conversión Manual


Conversión Manual
public
class
BitSetReadConverter
implements
Converter<DBObject,
BitSet>
{

@Override

public
BitSet
convert(DBObject
source)
{

BasicDBList
words
=
(BasicDBList)source.get("words");

BitSet
bitset
=
new
BitSet();

int
index
=
0;

for
(Object
word
:
words)
{

long
value
=
(Long)word;

while
(value
!=
0L)
{

if
((value
&
1L)
!=
0)
{

bitset.set(index);

}

++index;

value
=
value
>>>
1;

}

}

return
bitset;

}
}


Conversión Manual
Porque a veces lo automático no funciona


Conversión Manual
Porque a veces lo automático no funciona
<?xml version="1.0" encoding="UTF-8"?>
<beans>

<bean id="mongoTemplate"
<constructor-arg name="mongoDbFactory" ref="mongoDbFactory" />
<constructor-arg name="mongoConverter" ref="mappingConverter" />
</bean>

<mongo:db-factory dbname="vts"/>

<mongo:mapping-converter>
<mongo:custom-converters>
<mongo:converter>
<bean class="com.vts.db.BitSetReadConverter"/>
</mongo:converter>
</mongo:custom-converters>
</mongo:mapping-converter>

</beans>


Queries con multiples valores



Query query = new Query(
Criteria
.where("length").gte(20)
.and("length").lt(60));



Criteria
.where("length").gte(20)
.and("length").lt(60));

DEBUG MongoTemplate - find using query: { "length" : { "$lt" : 60} }



Criteria
.where("length").gte(20).lte(60));



Criteria
.where("length").gte(20).lte(60));

DEBUG MongoTemplate - find using query: { "toBow" : { "$gte" : 20 , "$lte" : 60} }


Conclusiones
Reducción del número de tablas (87x2 vs 5)
Optimización en el rendimiento de escrituras
Simpliﬁcación de los DAOs con Spring-Data

Otras ventajas:
Documentos en formato de Respuesta REST
interfaz Javascript en la consola


Q&A
David Gómez
@dgomezg


Spring Data y Mongo DB en un proyecto Real

Más contenido relacionado

La actualidad más candente (20)

Destacado (20)

Similar a Spring Data y Mongo DB en un proyecto Real (20)

Más de David Gómez García (20)

Último (20)

Spring Data y Mongo DB en un proyecto Real