DZone Java 8 Block Buster: Query Databases Using Streams

Meetup Gothenburg Dec
28 2015
Per-Åke Minborg
minborg@speedment.com

Palo Alto Office Speedment Java One
Presentation

Before Speedment
Connector
JDBC
ORM
JPA
Application
PROS
Widely spread standard
• Many providers
• Many developers
You can work with a
relational database as if it
were object oriented
 Increased productivity
 Provides a certain
degree of abstraction
(you can replace your
DBMS)
CONS
 Slows down the application
and gives unpredictable
performance
 You can not access all
Java 8 features
Time consuming to write
applications for existing
databases, You must still
write SQL, HQL, et. Al
Higher the risk for errors
Java

ORM
JPA
Speedment- a Java
Development Tool
Connector
JDBC
Application
Object Oriented functional programming
 Fast to develop
 Fast Execution
 Easy to test
 Secure
 Make use of all Java 8 features
Domain Model reflects your existing
datasource
Automatic generation of Code (”error
free”)
Java
Speedment
Java 8
Application

Speedment- a Java
Development Tool

Code Examples
• Embrace Java 8 paradigms with stream(), filter(), sort(), limit() etc.
• Forget about SQL, JQL et. al

Database Table “hare”
mysql> explain hare;
+-------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(45) | NO | | NULL | |
| color | varchar(45) | NO | | NULL | |
| age | int(11) | NO | | NU | |
+-------+-------------+------+-----+---------+----------------+
4 rows in set (0.01 sec)

Code Generation
public interface Hare extends Entity<Hare> {
public final static ReferenceComparableField<Hare, Integer> ID = …
public final static ReferenceComparableStringField<Hare> NAME = …
public final static ReferenceComparableStringField<Hare> COLOR = …
public final static ReferenceComparableField<Hare, Integer> AGE = …
Integer getId();
String getName();
String getColor();
Integer getAge();
Hare setId(Integer id);
Hare setName(String name);
Hare setColor(String color);
Hare setAge(Integer age);
/** Graph-like traversal methods eliminating JOINs */
Stream<Carrot> findCarrotsByOwner();
Stream<Carrot> findCarrotsByRival();
Stream<Carrot> findCarrots();
}

DZone Java 8 Block Buster: Query Databases Using Streams

Querying using Java 8
Streams
List<Hare> oldHares = hares.stream()
.filter(Hare.AGE.greaterThan(4))
.collect(toList());
SELECT * from `hare`
WHERE
`age` > 4;

Stream Fundamentals
Detailed description Slide 35-45

Depends on Terminating
Operation
Optional<Hare> oldHare = hares.stream()
.findAny();
SELECT * from `hare`
WHERE
`age` > 4 LIMIT 1;

Depends on Terminating
Operation
long noOldHares = hares.stream()
.count();
SELECT count(*) from `hare`
WHERE
`age` > 4;

Entities are linked
Optional<Carrot> carrot = hares.stream()
.filter(Hare.NAME.equal(”Spire”))
// Carrot is a foreign key.
.flatmap(Hare::findCarrots)
.findAny();

Multi-threading
hares.stream()
.parallel()
.forEach(doExpensiveOperation());

Persistence
Hare harry = hares.newInstance()
.setName("Harry")
.setColor("Gray")
.setAge(3)
.persist();
// OR JPA-style
entityManager.persist(harry);

Delete Using Streams
users.stream()
.filter(User.ID.greaterThan(4))
.forEach(User::delete);
DELETE FROM `user`
WHERE `id` > 4;

Initialization
Speedment speedment = new HareApplication().build();
Manager<Hare> hares = speedment.managerOf(Hare.class);

Speedment Insane Mode
•Extreme Performance, real time big Data
•Convert large databases into In-JVM-memory Java Objects
•Scales Linearly -More CPUs, More Insane

In Memory Initialization
Speedment speedment = new HareApplication()
.with(OnHeapReadOnlyCacheComponent.class)
.build();

How does it Work?
23
1 In-JVM-memory cache
2 Java Objects in Graph View
3 Separates Reads and Writes
Reflection in
real time
Application
JVM
Read
Write

Why will in-JVM-Memory give
Insane Speed?
Database In JVM Memory
Compare latencies using the Speed of Light

Why will in-JVM Memory give
Insane Speed?
• Keep everything in-JVM
• No need to check if an object is in cache
• We can organize objects knowing that we have them all
• We do not even have to look at individual objects -> O(1)
• Everything is Java
Size
Time
O(1)
O(n)

Speedment – OSS and
Enterprise

Cloud Deployment
RDBMS
Speedment
Speedment
Speedment
Speedment
Speedment
• Spin up nodes as you
grow
• Partition data over nodes

How can I fit my full Database
in JVM Memory?
~24 GB
>1 TB
>4 TB
∞
On-Heap
Off-Heap
SSD
SQL
Database Size
(After Compression)

Scale out with our partner
Hazelcast
• Set up a cluster in the Cloud
• Scale out your existing database
• Set up Terabytes of RAM

Properties:
• The API will remain the same regardless of selected Storage Engine
• Massively concurrent and lock free
• Scale with CPU performance and number of cores
• Eventually consistent
• Transaction safe, if you want

Configuration
• Groovy script like Gradle
• Config file is placed at the same location as the POM
dbms {
schema {
table {
name = ”hare";
column {
name = "name";
}
}
}
}

Ease of use:
• Single Maven dependency in POM
• Maven targets
• Works with Gradle
• GUI to automatically derive the groovy config file from existing databases

Properties:
• The database ”owns” the data
• Data is reflected into Materialized Object Views (MOVs)
• MOVs and application resides in the same JVM
• Most operations are O(1)
• persons.byName(“Bob") -> Map<PK,Person> with 1000 persons in 100 ns
(30 m)
• TCP/IP RTT 100 us -> +1000 "finds" per TCP RT.
• In-memory DBs with zero query latency are snails by definition

http://localhost:4567/trades/1001/200?
from=0&to=1441147840000&callback=Ext.data.JsonP.callback1
Ext.data.JsonP.callback1([
{'start':1420131600000,'stop':1420236681000,'average':100876,'open':100000,'close':102209,'high':104060,'low':98714},
… (200 rows in total) ...
{'start':1441147839000,'stop':1441147839000,'average':142165,'open':142165,'close':142165,'high':142165,'low':142165}
]);
Objectives:

Stream Fundamentals Per-Åke Minborg
minborg@speedment.com

Setup
36
DB App
Pwd ?
In-JVM
Cache

Stream<Trade>stream= trades.stream();
37
DB
Data
Source
Trade5 Trade4 Trade3 Trade2 Trade1

trades.stream().parallel();
38
TradeN
Trade4
Trade3
Trade2
Trade1
TradeX
Thread 1
Thread 2
Thread N
DB
Data
Source

trades.stream().filter(Predicate<Trade> predicate);
39
Trade2 Trade1
predicate.
test(trade)Trade5 Trade4
Trade3
false
true

trades.stream().map(Function<Trade,String>mapper);
40
Json2 Json1Trade5 Trade4
mapper.
apply(trade)
e.g.Trade::toJson

trades.stream().collect(Collectorcollector);
41
TradeN
Trade4
Trade3
Trade2
Trade1
TradeX
apply(trade)
apply(trade)
apply(trade)

Complete Stream
42
DB
Data
Source

Finding first and last date/time in an interval
43
from to
selected

trades.stream().parallel()
.filter(DATE.between(from, to))
.filter(STOCK.equal(stock))
.mapToLong(Trade::getDate)
.summaryStatistics();
44
2 3 4
2 3 4
2 3 4
5
1
2
3
4
5
DB
Data
Source

45
DB
Data
Source 1
4
4
4
5
2
3
trades.stream().parallel()
.filter(DATE.between(from, to))
.filter(STOCK.equal(stock))
.mapToLong(Trade::getDate)
.summaryStatistics();
1
2
3
4
5

Generating a REST API
Introduction to Speedment and Sencha
Emil Forslund
emil@speedment.com

Outline
●The Speedment + Sencha Software Stack
●Using the Pivotgrid Plugin for Speedment

Overview of the Software
Stack
DB
Javascript Framework
Server
Framework
DB Abstraction Layer

Database Layer
●Holds all the data collected from various sources
●High Level of Normalization
Sale
Product
Salesperson
Customer
Country
Region
Office
Cost Price
Count

Database Abstraction Layer
●Presents an Object-Oriented view of the database
●Write expressive and powerful queries
●Optimize reading by using in-memory caching
●Better testability and higher productivity
speedment.managerOf(Sale.class)
.stream()
.filter(Sale.PRODUCT.equal(105))
.collect(toList());

Server Framework
●A lightweight webserver
●Separates client/server-responsabilitys
●Execute selected business logic as response to REST commands
●Low Level of Abstraction
http://localhost:8123/salesinfo/
?c=callback
&filter=[{...}]

Javascript Framework
●Presents data in user-friendly components
●User can navigate the data in the browser
●Automatically requests additional data from the server

Introduction to the Pivot Grid
for Speedment
●Generate a full-fledged server application from your database
●Select exactly which data to expose in the API
●Define aliases for columns and tables
●Map between different data types
●Create virtual columns calculated in real-time

Step 1 – Create a New Project

Step 2 – Find Pivotgrid
Archetype

Example Application:
Salesinfo

DZone Java 8 Block Buster: Query Databases Using Streams

More Related Content

What's hot (20)

Similar to DZone Java 8 Block Buster: Query Databases Using Streams (20)

More from Speedment, Inc. (6)

Recently uploaded (20)

DZone Java 8 Block Buster: Query Databases Using Streams