How HarperDB Works

2
HarperDB removes much of the complexity
involved with interacting and reacting to Big
Data at scale. The tool allows developers of
any skill level to interact with large amounts of
data while also scaling to meet the needs of
larger enterprise applications through its
leverage of Node.js, a REST API interface, and
fully-indexed model.
Let’s jump in to show you how exactly the
product works.

THE BASICS
Most products that provide both NoSQL and SQL capability
use a concept called multi model. Multi model either
duplicates your data to present it in both a JSON format as
well as a table & row structure or it utilizes significant
amounts of RAM to do transformations in memory.
HarperDB’s single model means there is no transformation of
data required and as a result, you will see significant savings
on storage and RAM all while preserving speed.
3

SIMPLE REST API
Using this single model architecture and an
easy-to-use REST API, write to a schema-less
model with either JSON or SQL.
4

EXPLODED
DATA MODEL
Instead of storing
this data as columns
and rows or a
document store,
HarperDB explodes
the data into
individual indices.
5

NATIVELY
INDEXED
STATUS
Each attribute is stored
separately, allowing for
HarperDB to be fully
indexed without creating
additional overhead.
6

SQL AND NOSQL
CAPABILITIES
You can then query or
search HarperDB using SQL
or NoSQL on any column.
Using joins and multiple
conditions and operators,
HarperDB coalesces these
indices back into a single
object and returns the
results in JSON or column
row format.
7

Now that you’ve learned some of the
features of the product, here’s a practical
example on how to get started after
9
installing the product.

EXAMPLE
COMMANDS
FOR USING
HARPERDB
10

1. CREATE SCHEMA
11
Many databases start with a schema, and HarperDB is no different. You can't
add or update data without a table, and you can't create a table without a
schema. Here's an example of a schema we're creating called "dev" and the
appropriate curl command:
curl --request POST
--url http://localhost:9925/
--header 'authorization: Basic YWRtaW46MTQwMA=='
--header 'content-type: application/json'
--data '{
"operation":"create_schema",
"schema":"dev"
}'
Notice the authorization header
added to the call. Every call must
have an authorization header.

2. CREATE TABLE
12
With the schema created, it's time to tackle the table. Tables in HarperDB
have no defined structure other than the table name and hash.
The hash can be named whatever you like. From there, you can dynamically
add columns/attributes to each table on insert of a record.

13
Let's create a table called "dog." Here is the curl command:
curl --request POST
--data '{
"operation":"create_table",
"schema":"dev",
"table":"dog",
"hash_attribute":"id"
}'

14
And just for our purposes, go ahead and create another table called "breed."
Here's the curl command:
curl --request POST
--data '{
"operation":"create_table",
"schema":"dev",
"table":"breed",
"hash_attribute":"id"
}'

4. NOSQL INSERT
15
With our "dev" schema and our two tables - "breed" and "dog" - created, we
can begin to add some data to play with.
HarperDB has a number of different ways to insert data, the most basic
being a simple NoSQL Insert.

16
We'll start by inserting a single record into HarperDB in the "dog" table in
the "dev" schema. Our record looks like:
{
"id" : 1,
"doc" : "Penny",
"owner_name": "Kyle",
"breed_id":154,
"age":5,
"weight_lbs":35,
"adorable":true
}

17
You'll notice we have included the hash "id"
defined in the "create_table" call in the
object. This must always be included when
inserting or updating an object. The rest of
the object is a simple attribute/value pairs
that we have defined in the object itself. No
schema configuration is needed for inserting
these attributes.

18
Here's the curl command:
curl --request POST
--data '{
"operation":"insert",
"schema" : "dev",
"table":"dog",
"records": [
{
"id" : 1,
"doc" : "Penny",
"breed_id":154,
"age":5,
"weight_lbs":35,
"adorable":true
}
]
}'
You will notice that the "records"
attribute on the operation object is
an array. An insert can be called
with one more records.

5. NOSQL INSERT
MULTIPLE ROWS
19
Now that we've added a single row, lets call the same command, but use
multiple records:

20
curl --request POST
--data '{
"operation":"insert",
"schema" : "dev",
"table":"dog",
"records": [
{
"id" : 2,
"dog_name" : "Harper",
"owner_name": "Stephen",
"breed_id":346,
"age":5,
"weight_lbs":55,
"adorable":true
},

21
{
"id" : 3,
"dog_name" : "Alby",
"owner_name": "Kaylan",
"breed_id":348,
"age":5,
"weight_lbs":84,
"adorable":true
},
{
"id" : 4,
"dog_name" : "Billy",
"owner_name": "Zach",
"breed_id":347,
"age":4,
"weight_lbs":60,
"adorable":true
},

22
{
"id" : 5,
"dog_name" : "Rose Merry",
"owner_name": "Zach",
"breed_id":348,
"age":6,
"weight_lbs":15,
"adorable":true
},
{
"id" : 6,
"dog_name" : "Kato",
"breed_id":351,
"age":4,
"weight_lbs":28,
"adorable":true
},

23
{
"id" : 7,
"dog_name" : "Simon",
"owner_name": "Fred",
"breed_id":349,
"age":1,
"weight_lbs":35,
"adorable":true
},
{
"id" : 8,
"dog_name" : "Gemma",
"owner_name": "Stephen",
"breed_id":350,
"age":3,
"weight_lbs":55,
"adorable":true
}
]
}'
You will notice the command is
identical however we have simply
added multiple objects to the
records array.

6. SQL SELECT
24
OK, so you have all your tables, rows, and schemas lined up in the system, but
you want to pull out some structured data. Time to migrate over to another
tool or sacrifice precious memory and storage to transform the data, right?
Maybe with other databases, but HarperDB was built from the ground up with
a patent-pending data chain unique to the industry that makes extracting easy
and simple.
Rather than saving your data as a document or in tabular/columnar format,
HarperDB atomizes every record into its individual attributes and saves them
discretely on disk as unique indices.
With this functionality, you can extract unstructured data from structured
tables/columns no problem with the simplest of commands such as SQL
Select.

25
Here's a SQL Select command from our "dev" schema and "dog" table
where the required operation uses standard SQL:
curl --request POST
--url http://localhost:9925
--header 'authorization: Basic
SERCX0FETUlOOjE0MDA='
--data '{
"operation":"sql",
"sql":"select * from dev.dog"
}'
And here's the response:

26
[
{
"age": 5,
"breed": "Mutt",
"id": 1,
"name": "Harper",
"weight_lbs": 55
},
{
"id": 2,
"name": "Simon"
},
{
"age": 5,
"breed": "Mutt",
"id": 3,
"name": "Penny",
"owner": "kyle b",
"weight_lbs": 35
}
]

7. BULK UPLOADS
FROM CSV
27
On top of all the great features offered by HarperDB, you can also insert
CSVs directly into the system.
See this file - breeds.csv - as an example of how the data should look.
Then simply place that file or your own file somewhere on your file system.
Make sure you keep track of the path. To load the csv, you simply give the
path of the file, table and schema and HarperDB will load the file. The first
row of the CSV must be the column/attribute names, and the hash must be
included as a column.

28
curl --request POST
--data '{
"operation":"csv_file_load",
"schema":"dev",
"table":"breed",
"file_path":"/path/to/your/file/your_file.csv"
}'
Here's the curl command for uploading CSVs:

HOW IT FEELS
TO USE
HARPERDB
29
You've seen the nuts and bolts of the product as well as
the basic commands, but you might be wondering what it
feels like to operate HarperDB and why you should switch.

THE DETAILS
30
A client’s data-intensive web-based application relied on
several persistent stores to warehouse and analyze SEC (U.S.
Securities and Exchange Commission) data. Relational
databases, a data warehouse, stored procedures, triggers, R
analytics, and a 57-gigabyte document mirror of the SEC’s
archives were components of the solution.
Let's explore a use case from Mycos Technologies,
a software development agency.

31
The original cloud document solution provided an extensive
library for the development team to learn and use, the promise
of global distribution, and service-level backed performance
metrics. The primary obstacle of the original document solution
was cost during the development phase.
As a cloud service, with no capability for self-hosting for a
distributed development team, the client was forced to scale
the solution to a monthly cost exceeding $1500 or face
throttling during standard CRUD operations. The “starter”
offering was limited in storage to 10-gigabytes and
consequently, a non-starter.

HARPERDB IS
CHOSEN
32
HarperDB initially appealed as an
alternative because of its cost-
effectiveness. It became the database
solution of choice for other reasons
discovered in the process.

HARPERDB IN ACTION
33
Mycos Technologies found the following when using HarperDB:
It can be deployed locally or in the cloud, or both.
Its scalability was only constrained by the backing hardware and other variables that we
could control.
In informal benchmarks, its response times were comparable or better than alternative
solutions.
There was no platform-specific client library or driver to version, wrapper, or learn.
We were not compelled to use a legacy data model to access our data.
We could leverage SQL syntax when developers needed fine-grained CRUD operations. We
could leverage NoSQL for our batch operations.
The only maintenance and tuning required can be done by server administrators.
Indexing and collection partitioning are handled automatically.
Most important to us, the developer learning curve was hours instead of weeks. All
developers who had used REST could be productive, almost immediately. The query
syntax was intuitive.

WHY HARPERDB?
"In our view, HarperDB is a compelling solution
primarily becasue it factors-out platform-specific
client component "helping" us access our data. We
know our data, and with SQL and NoSQL
functionality, HarperDB gives us unimpeded access."
34
John Douglas, Director, Mycos Technologies

There you have it, you now know the ins and
outs of HarperDB and how it feels to switch.
Have more questions? Check out our
Zendesk or contact us today.
Ready to get started?
DOWNLOAD OUR COMMUNITY EDITION

How HarperDB Works

More Related Content

What's hot (20)

Similar to How HarperDB Works (20)

Recently uploaded (20)

How HarperDB Works