Overview of MONOMI
1 like499 views
The document discusses a system called Monomi, which extends CryptDB for processing analytical queries over encrypted data, aiming to protect data against server compromises with a modest overhead. It presents split execution optimization techniques, such as per-row precomputation and space-efficient encryption, to enhance performance while ensuring security. The proposed architecture and query planning are evaluated through experiments demonstrating effective execution of complex queries without directly exposing sensitive data.
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
key:[$1]
LocalProjection
exprs: [$0, sum($1)]
LocalGroupFilter
filter: sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1]
LocalProjection
exprs: [sum($0) * 0.0001]
RemoteSQL
SELECT
ps parkey DET,
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
GROUP BY ps partkey DET
LocalDecrypt
pos: [$0]
RemoteSQL
SELECT
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-13-320.jpg)
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
key:[$1]
LocalProjection
exprs: [$0, sum($1)]
LocalGroupFilter
filter: sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1]
LocalProjection
exprs: [sum($0) * 0.0001]
RemoteSQL
SELECT
ps parkey DET,
GROUP(precomp DET
Precomputed
multiplication
)
FROM ... WHERE
n name DET = 0xabcdef
GROUP BY ps partkey DET
LocalDecrypt
pos: [$0]
RemoteSQL
SELECT
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-14-320.jpg)
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
key:[$1]
LocalProjection
exprs: [$0, sum($1)]
LocalGroupFilter
filter: sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1
Reference to
the columns
of the child
operator
]
LocalProjection
exprs: [sum($0) * 0.0001]
RemoteSQL
SELECT
ps parkey DET,
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
GROUP BY ps partkey DET
LocalDecrypt
pos: [$0]
RemoteSQL
SELECT
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-15-320.jpg)
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
key:[$1]
LocalProjection
exprs: [$0, sum($1)]
LocalGroupFilter
filter: sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1]
LocalProjection
exprs: [sum($0) * 0.0001]
RemoteSQL
SELECT
ps parkey DET,
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
Deterministic
encryption of the
value :1
GROUP BY ps partkey DET
LocalDecrypt
pos: [$0]
RemoteSQL
SELECT
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-16-320.jpg)
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
key:[$1]
LocalProjection
exprs: [$0, sum($1)]
LocalGroupFilter
filter: sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1]
LocalProjection
exprs: [sum($0) * 0.0001]
RemoteSQL
SELECT
ps parkey DET,
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
GROUP BY ps partkey DET
LocalDecrypt
pos: [$0]
RemoteSQL
SELECT
GROUP
Concatenation of all values
from each GROUP BY group
(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-17-320.jpg)
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
key:[$1]
LocalProjection
exprs: [$0, sum($1)]
LocalGroupFilter
filter: sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1]
LocalProjection
exprs: [sum($0) * 0.0001]
RemoteSQL
SELECT
ps parkey DET,
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
GROUP BY ps partkey DET
LocalDecrypt
pos: [$0]
RemoteSQL
SELECT
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-18-320.jpg)
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
key:[$1]
LocalProjection
exprs: [$0, sum($1)]
LocalGroupFilter
filter: sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1]
LocalProjection
exprs: [sum($0) * 0.0001]
RemoteSQL
SELECT
Outmost SELECT
ps parkey DET,
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
GROUP BY ps partkey DET
LocalDecrypt
pos: [$0]
RemoteSQL
SELECT
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-19-320.jpg)
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
key:[$1]
LocalProjection
exprs: [$0, sum($1)]
LocalGroupFilter
filter: sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1]
LocalProjection
exprs: [sum($0) * 0.0001]
RemoteSQL
SELECT
ps parkey DET,
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
GROUP BY ps partkey DET
LocalDecrypt
pos: [$0]
RemoteSQL
SELECT
Innermost SELECT
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-20-320.jpg)
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
key:[$1]
LocalProjection
exprs: [$0, sum($1)]
LocalGroupFilter
filter: sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1]
LocalProjection
exprs: [sum($0) * 0.0001]
RemoteSQL
SELECT
ps parkey DET,
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
GROUP BY ps partkey DET
LocalDecrypt
Decrypts the data
at the client
pos: [$0]
RemoteSQL
SELECT
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-21-320.jpg)
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
key:[$1]
LocalProjection
exprs: [$0, sum($1)]
LocalGroupFilter
filter: sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1]
LocalProjection
exprs:
Multiplication
by constant
[sum($0) * 0.0001]
RemoteSQL
SELECT
ps parkey DET,
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
GROUP BY ps partkey DET
LocalDecrypt
pos: [$0]
RemoteSQL
SELECT
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-22-320.jpg)
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
key:[$1]
LocalProjection
exprs: [$0, sum($1)]
LocalGroupFilter
filter:
Filter referring to
the HAVING clause
sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1]
LocalProjection
exprs: [sum($0) * 0.0001]
RemoteSQL
SELECT
ps parkey DET,
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
GROUP BY ps partkey DET
LocalDecrypt
pos: [$0]
RemoteSQL
SELECT
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-23-320.jpg)
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
key:[$1]
LocalProjection
Selection of ps partkey and
of the summation of the pre-
computed expression
exprs: [$0, sum($1)]
LocalGroupFilter
filter: sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1]
LocalProjection
exprs: [sum($0) * 0.0001]
RemoteSQL
SELECT
ps parkey DET,
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
GROUP BY ps partkey DET
LocalDecrypt
pos: [$0]
RemoteSQL
SELECT
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-24-320.jpg)
![Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion
SPLIT QUERY PLAN
Example (TPC-H Q11)
SELECT ps partkey,
SUM (ps supplycost * ps availqty)
AS VALUE
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1
GROUP BY ps partkey
HAVING SUM (ps supplycost * ps availqty) > (
SELECT SUM (ps supplycost * ps availqty)
* 0.0001
FROM partsupp JOIN supplier JOIN nation
WHERE n name = :1 )
ORDER BY value DESC;
LocalSort
Sorting referring to the
ORDER BY clause
key:[$1]
LocalProjection
exprs: [$0, sum($1)]
LocalGroupFilter
filter: sum($1) > subquery0()
LocalDecrypt
pos: [$0, $1]
LocalProjection
exprs: [sum($0) * 0.0001]
RemoteSQL
SELECT
ps parkey DET,
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
GROUP BY ps partkey DET
LocalDecrypt
pos: [$0]
RemoteSQL
SELECT
GROUP(precomp DET)
FROM ... WHERE
n name DET = 0xabcdef
11 / 31](https://image.slidesharecdn.com/monomi-160701042816/85/Overview-of-MONOMI-25-320.jpg)
Overview of MONOMI
- 1. Processing Analytical Queries over Encrypted Data Stephen Tu, M. Frank Kaashoek, Samuel Madden, and Nickolai Zeldovich 39th International Conference on Very Large Data Bases Riva del Garda, Trento, Italy, August 2013 SWIM Seminar May 19th, 2015 Mateus Cruz
- 2. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion OUTLINE 1 Introduction 2 Split Execution 3 Optimization Techniques 4 Designer and Planner 5 Experiments 6 Conclusion 2 / 31
- 3. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion OUTLINE 1 Introduction 2 Split Execution 3 Optimization Techniques 4 Designer and Planner 5 Experiments 6 Conclusion 3 / 31
- 4. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion OVERVIEW System called MONOMI Extension of CryptDB Analytical queries over encrypted data Data protected against server compromises Modest overhead Slowdown of 1.03 to 2.33× 4 / 31
- 5. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion MAIN IDEAS Split client/server execution Optimization techniques Per-row precomputation Space-efficient encryption Grouped homomorphic addition Pre-filtering Designer Physical data layout Planner Efficient execution plan for queries 5 / 31
- 6. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion USED ENCRYPTION SCHEMES 6 / 31
- 7. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion ARCHITECTURE 7 / 31
- 8. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion OUTLINE 1 Introduction 2 Split Execution 3 Optimization Techniques 4 Designer and Planner 5 Experiments 6 Conclusion 8 / 31
- 9. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion INTUITION The server cannot execute all queries Limitations of cryptosystems Cannot transfer all data to the client Large amount of data Divide execution Execute as much as possible on server Transfer data to the client when it is not possible to execute on the server 9 / 31
- 10. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion LIMITATIONS OF SERVER EXECUTION Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS value FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; 10 / 31
- 11. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion LIMITATIONS OF SERVER EXECUTION Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS value FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM Addition and comparison in- volve incompatible encryption schemes (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; 10 / 31
- 12. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion LIMITATIONS OF SERVER EXECUTION Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS value FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) No efficient encryption scheme allows multiplica- tion of two encrypted values * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; 10 / 31
- 13. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort key:[$1] LocalProjection exprs: [$0, sum($1)] LocalGroupFilter filter: sum($1) > subquery0() LocalDecrypt pos: [$0, $1] LocalProjection exprs: [sum($0) * 0.0001] RemoteSQL SELECT ps parkey DET, GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef GROUP BY ps partkey DET LocalDecrypt pos: [$0] RemoteSQL SELECT GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 14. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort key:[$1] LocalProjection exprs: [$0, sum($1)] LocalGroupFilter filter: sum($1) > subquery0() LocalDecrypt pos: [$0, $1] LocalProjection exprs: [sum($0) * 0.0001] RemoteSQL SELECT ps parkey DET, GROUP(precomp DET Precomputed multiplication ) FROM ... WHERE n name DET = 0xabcdef GROUP BY ps partkey DET LocalDecrypt pos: [$0] RemoteSQL SELECT GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 15. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort key:[$1] LocalProjection exprs: [$0, sum($1)] LocalGroupFilter filter: sum($1) > subquery0() LocalDecrypt pos: [$0, $1 Reference to the columns of the child operator ] LocalProjection exprs: [sum($0) * 0.0001] RemoteSQL SELECT ps parkey DET, GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef GROUP BY ps partkey DET LocalDecrypt pos: [$0] RemoteSQL SELECT GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 16. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort key:[$1] LocalProjection exprs: [$0, sum($1)] LocalGroupFilter filter: sum($1) > subquery0() LocalDecrypt pos: [$0, $1] LocalProjection exprs: [sum($0) * 0.0001] RemoteSQL SELECT ps parkey DET, GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef Deterministic encryption of the value :1 GROUP BY ps partkey DET LocalDecrypt pos: [$0] RemoteSQL SELECT GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 17. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort key:[$1] LocalProjection exprs: [$0, sum($1)] LocalGroupFilter filter: sum($1) > subquery0() LocalDecrypt pos: [$0, $1] LocalProjection exprs: [sum($0) * 0.0001] RemoteSQL SELECT ps parkey DET, GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef GROUP BY ps partkey DET LocalDecrypt pos: [$0] RemoteSQL SELECT GROUP Concatenation of all values from each GROUP BY group (precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 18. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort key:[$1] LocalProjection exprs: [$0, sum($1)] LocalGroupFilter filter: sum($1) > subquery0() LocalDecrypt pos: [$0, $1] LocalProjection exprs: [sum($0) * 0.0001] RemoteSQL SELECT ps parkey DET, GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef GROUP BY ps partkey DET LocalDecrypt pos: [$0] RemoteSQL SELECT GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 19. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort key:[$1] LocalProjection exprs: [$0, sum($1)] LocalGroupFilter filter: sum($1) > subquery0() LocalDecrypt pos: [$0, $1] LocalProjection exprs: [sum($0) * 0.0001] RemoteSQL SELECT Outmost SELECT ps parkey DET, GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef GROUP BY ps partkey DET LocalDecrypt pos: [$0] RemoteSQL SELECT GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 20. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort key:[$1] LocalProjection exprs: [$0, sum($1)] LocalGroupFilter filter: sum($1) > subquery0() LocalDecrypt pos: [$0, $1] LocalProjection exprs: [sum($0) * 0.0001] RemoteSQL SELECT ps parkey DET, GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef GROUP BY ps partkey DET LocalDecrypt pos: [$0] RemoteSQL SELECT Innermost SELECT GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 21. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort key:[$1] LocalProjection exprs: [$0, sum($1)] LocalGroupFilter filter: sum($1) > subquery0() LocalDecrypt pos: [$0, $1] LocalProjection exprs: [sum($0) * 0.0001] RemoteSQL SELECT ps parkey DET, GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef GROUP BY ps partkey DET LocalDecrypt Decrypts the data at the client pos: [$0] RemoteSQL SELECT GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 22. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort key:[$1] LocalProjection exprs: [$0, sum($1)] LocalGroupFilter filter: sum($1) > subquery0() LocalDecrypt pos: [$0, $1] LocalProjection exprs: Multiplication by constant [sum($0) * 0.0001] RemoteSQL SELECT ps parkey DET, GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef GROUP BY ps partkey DET LocalDecrypt pos: [$0] RemoteSQL SELECT GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 23. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort key:[$1] LocalProjection exprs: [$0, sum($1)] LocalGroupFilter filter: Filter referring to the HAVING clause sum($1) > subquery0() LocalDecrypt pos: [$0, $1] LocalProjection exprs: [sum($0) * 0.0001] RemoteSQL SELECT ps parkey DET, GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef GROUP BY ps partkey DET LocalDecrypt pos: [$0] RemoteSQL SELECT GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 24. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort key:[$1] LocalProjection Selection of ps partkey and of the summation of the pre- computed expression exprs: [$0, sum($1)] LocalGroupFilter filter: sum($1) > subquery0() LocalDecrypt pos: [$0, $1] LocalProjection exprs: [sum($0) * 0.0001] RemoteSQL SELECT ps parkey DET, GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef GROUP BY ps partkey DET LocalDecrypt pos: [$0] RemoteSQL SELECT GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 25. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPLIT QUERY PLAN Example (TPC-H Q11) SELECT ps partkey, SUM (ps supplycost * ps availqty) AS VALUE FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 GROUP BY ps partkey HAVING SUM (ps supplycost * ps availqty) > ( SELECT SUM (ps supplycost * ps availqty) * 0.0001 FROM partsupp JOIN supplier JOIN nation WHERE n name = :1 ) ORDER BY value DESC; LocalSort Sorting referring to the ORDER BY clause key:[$1] LocalProjection exprs: [$0, sum($1)] LocalGroupFilter filter: sum($1) > subquery0() LocalDecrypt pos: [$0, $1] LocalProjection exprs: [sum($0) * 0.0001] RemoteSQL SELECT ps parkey DET, GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef GROUP BY ps partkey DET LocalDecrypt pos: [$0] RemoteSQL SELECT GROUP(precomp DET) FROM ... WHERE n name DET = 0xabcdef 11 / 31
- 26. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion OUTLINE 1 Introduction 2 Split Execution 3 Optimization Techniques 4 Designer and Planner 5 Experiments 6 Conclusion 12 / 31
- 27. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion PER-ROW PRECOMPUTATION Prior computation of certain expressions Materialized using additional columns Decision made by the designer module Example SUM (ps supplycost * ps availqty) 13 / 31
- 28. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPACE-EFFICIENT ENCRYPTION Minimize ciphertext expansion FFX mode of operation: n bits to n bits Pack multiple columns in a row Pack multiple rows into a single Paillier Packed ciphertexts are kept in separate files on the local file system 14 / 31
- 29. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion GROUPED HOMOMORPHIC ADDITION Packed aggregates computed with a single modular multiplication (a1||...||an) + (b1||...||bn) = (a1 + b1)||...||(an + bn) E(a1||...||an)×E(b1||...||bn) = E((a1+b1)||...||(an+bn)) 15 / 31
- 30. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion PRE-FILTERING Minimize data sent to the client Apply filtering to encrypted data Example SELECT l orderkey FROM lineitem GROUP BY l orderkey HAVING SUM(l quantity) > :1 16 / 31
- 31. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion PRE-FILTERING Minimize data sent to the client Apply filtering to encrypted data Example SELECT l orderkey FROM lineitem GROUP BY l orderkey HAVING SUM(l quantity) > :1 Incompatible schemes for SUM and comparison (>) 16 / 31
- 32. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion PRE-FILTERING Minimize data sent to the client Apply filtering to encrypted data Example SELECT l orderkey det, PAILLIER SUM(l quantity paillier) FROM lineitem GROUP BY l orderkey det HAVING MAX(l quantity ope) > encrypt ope(m) OR COUNT(*) > (:1 / m) 16 / 31
- 33. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion PRE-FILTERING Minimize data sent to the client Apply filtering to encrypted data Example SELECT l orderkey det, PAILLIER SUM(l quantity paillier) FROM lineitem GROUP BY l orderkey det HAVING MAX(l quantity ope) > encrypt ope(m Maximum value of the column l quantity ) OR COUNT(*) > (:1 / m) 16 / 31
- 34. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion OUTLINE 1 Introduction 2 Split Execution 3 Optimization Techniques 4 Designer and Planner 5 Experiments 6 Conclusion 17 / 31
- 35. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion INTUITION Optimizations are not always better Designer Best physical design – Encryption schemes – Precomputed expressions Planner Best query plan at runtime 18 / 31
- 36. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion DESIGNER: INPUT AND OUTPUT Input Representative query workload – Q0, Q1, ..., Qn – Chosen by the administrator Sample data Space constraint factor (optional) Output Physical design of the server – Set of encrypted columns to materialize 19 / 31
- 37. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion DESIGNER: ALGORITHM 1 Consider all operations in query Qi Check what expression would allow execution on the server EncSeti – Set of value, scheme pairs for Qi Example WHERE x = :1 generates a x, DET pair, referring to the x column ORDER BY x + y generates a x + y, OPE pair, referring to a precomputed x + y value 20 / 31
- 38. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion DESIGNER: ALGORITHM 2 The designer invokes the planner to determine the best way to execute Qi The planner computes PowSeti – Contains the subsets of EncSeti The planner constructs an execution plan for Qi for each element of the power set 3 The planner uses a cost model to estimate the fastest execution plan 20 / 31
- 39. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion COST MODEL Sum of three components Execution time on the server Data transfer time Post-processing on client (decryption) Constraints are considered using an Integer Linear Programming (ILP) formulation 21 / 31
- 40. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion OUTLINE 1 Introduction 2 Split Execution 3 Optimization Techniques 4 Designer and Planner 5 Experiments 6 Conclusion 22 / 31
- 41. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion IMPLEMENTATION 8.000 lines of Scala for designer/planner 4.000 lines of C++ for client library OpenSSL for cryptography Each table is mapped to an encrypted table Copies of columns (different cryptosystems) Do not support Views Pattern matching with two or more patterns Example LIKE ’%foo%bar%’ 23 / 31
- 42. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion ENVIRONMENT Client Four 4-core 2.2GHz Intel Xeon E5520 24GB RAM Server Four 4-core 2.4GHz Intel Xeon E5530 24GB RAM Multiple cores used for decryption Postgres 8.4 Memory limit: 8GB TPC-H scale 10 dataset 24 / 31
- 43. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion OVERALL EFFICIENCY Median overhead of 1.24× 25 / 31
- 44. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion TECHNIQUE PERFORMANCE Cumulative use of optimization techniques 26 / 31
- 45. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SPACE OVERHEAD 27 / 31
- 46. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SENSITIVITY TO DESIGNER INPUT Choosing representative queries Aggregation over expressions Expressions requiring precomputation Very selective WHERE on large relations 28 / 31
- 47. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion SECURITY Plaintext is never revealed OPE is used infrequently Leaks order 29 / 31
- 48. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion OUTLINE 1 Introduction 2 Split Execution 3 Optimization Techniques 4 Designer and Planner 5 Experiments 6 Conclusion 30 / 31
- 49. Introduction Split Execution Optimization Techniques Designer and Planner Experiments Conclusion CONCLUSION Novel system: MONOMI Analytic queries over confidential data New optimization techniques Use of designer and planner Modest overheads Execution: 1.24× Space: 1.72× 31 / 31