Memory Optimization
Sripathi Krishnan, CTO, HashedIn Technologies
Copyright © 2017 HashedIn Technologies Pvt. Ltd.
- Ensure the objects don’t have redundant data
- Users have recommended products
- If you cache users, ensure
recommended_products is just a reference
Copyright © 2017 HashedIn Technologies Pvt. Ltd.
- Default serialization libraries are not
efficient
- Use Protobuf / Message Pack / Others
- JSON must be compressed
Copyright © 2017 HashedIn Technologies Pvt. Ltd.
- Keys in redis have lots of overheads
- For smaller strings, combine them into a larger
hash
- Tweak hash-max-ziplist-entries and
hash-max-ziplist-value appropriately
Copyright © 2017 HashedIn Technologies Pvt. Ltd.
- Objects are usually saved in hashmap, but the
keys are redundant if every object has the
same keys
- “NamedTuple”: store objects in a list instead
- “C like struct”: Use Redis Bitfield command to
store fixed length values in a single string
Copyright © 2017 HashedIn Technologies Pvt. Ltd.
- Sets in Redis are normally stored as HashTables
- But if they only contain integers/longs, they are
stored in memory-efficient “intsets”
- 10x or more memory reduction if your IDs are
numeric only
Copyright © 2017 HashedIn Technologies Pvt. Ltd.
- Count Objects? Use HyperLogLog
- Check for Existence? Use Bloom Filter
You won’t get exact results, but sometimes the
memory savings are worth it.
Copyright © 2017 HashedIn Technologies Pvt. Ltd.
- Redis Strings can be used as a giant bitmap
- You can set/unset/toggle individual bits
Useful for things like access control - can this user
perform that operation?
Copyright © 2017 HashedIn Technologies Pvt. Ltd.
- Sorted Sets are very expensive memory-wise
- Tweak zset-max-ziplist-entries and
zset-max-ziplist-value if you are using lots of
smaller sorted sets
Copyright © 2017 HashedIn Technologies Pvt. Ltd.
- Sorted Sets are very expensive memory-wise
- Tweak zset-max-ziplist-entries and
zset-max-ziplist-value if you are using lots of
smaller sorted sets
Copyright © 2017 HashedIn Technologies Pvt. Ltd.
Take the guess work out of your memory analysis.
https://github.com/sripathikrishnan/redis-rdb-tools

More Related Content

PPTX
Redis Modules - Redis India Tour - 2017
PDF
Redis as database - HashedIn
DOCX
Hadoop Research
PPT
Hadoop technology
ODP
Hadoop introduction
PDF
An introduction to Big-Data processing applying hadoop
PDF
Pillars of Heterogeneous HDFS Storage
PDF
The Computer Science Behind a modern Distributed Database
Redis Modules - Redis India Tour - 2017
Redis as database - HashedIn
Hadoop Research
Hadoop technology
Hadoop introduction
An introduction to Big-Data processing applying hadoop
Pillars of Heterogeneous HDFS Storage
The Computer Science Behind a modern Distributed Database

What's hot (20)

PPTX
Introduction to Big Data and Hadoop
PPTX
Indexing with solr search server and hadoop framework
PDF
Scalable and High available Distributed File System Metadata Service Using gR...
PDF
Big Data Analytics in a Heterogeneous World - Joydeep Das of Sybase
PDF
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
PPTX
Introduction to Big Data and hadoop
PDF
DataLogix Hadoop Solution
PPTX
ImpalaToGo and Tachyon integration
PDF
An introduction To Apache Spark
PPTX
Hadoop introduction
PDF
Map reduce & HDFS with Hadoop
PPTX
JOSA TechTalks - Big Data on Hadoop
PPTX
A Basic Introduction to the Hadoop eco system - no animation
PDF
Scalable Filesystem Metadata Services with RocksDB
ODP
Introdution to Apache Hadoop
PDF
Foss evolution cos-boudnik
PDF
Hadoop sqoop
PDF
Ceph Days 2014 Paul Evans Slide Deck
PPTX
ImpalaToGo design explained
PDF
Seagate Implementation of Dense Storage Utilizing HDDs and SSDs
Introduction to Big Data and Hadoop
Indexing with solr search server and hadoop framework
Scalable and High available Distributed File System Metadata Service Using gR...
Big Data Analytics in a Heterogeneous World - Joydeep Das of Sybase
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
Introduction to Big Data and hadoop
DataLogix Hadoop Solution
ImpalaToGo and Tachyon integration
An introduction To Apache Spark
Hadoop introduction
Map reduce & HDFS with Hadoop
JOSA TechTalks - Big Data on Hadoop
A Basic Introduction to the Hadoop eco system - no animation
Scalable Filesystem Metadata Services with RocksDB
Introdution to Apache Hadoop
Foss evolution cos-boudnik
Hadoop sqoop
Ceph Days 2014 Paul Evans Slide Deck
ImpalaToGo design explained
Seagate Implementation of Dense Storage Utilizing HDDs and SSDs
Ad

Similar to Redis memory optimization sripathi, CTO hashedin (20)

PDF
Redis Workshop on Data Structures, Commands, Administration
PDF
Hadoop and object stores can we do it better
PDF
Hadoop and object stores: Can we do it better?
PDF
Talend Big Data Tutorial | Talend DI and Big Data Certification | Talend Onli...
PPTX
Aem asset optimizations & best practices
PDF
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
PPTX
HDFS & ASM
ODP
Demystifying bigdata ashish1
PDF
Elasticsearch quick Intro (English)
ODP
Redis overview
PPTX
Introduction to Redis Data Structures: Hashes
PDF
Using Pluggable Apache Spark SQL Filters to Help GridPocket Users Keep Up wit...
PPTX
Big Data Analytics -Introduction education
PPTX
Model Serving for Deep Learning
PPTX
Right time Vs real time
PPTX
NameNode Analytics - Querying HDFS Namespace in Real Time
PDF
Python Powered Data Science at Pivotal (PyData 2013)
PDF
Building an Apache Sling Rendering Farm
PDF
What is HDFS | Hadoop Distributed File System | Edureka
PDF
Techincal Talk Hbase-Ditributed,no-sql database
Redis Workshop on Data Structures, Commands, Administration
Hadoop and object stores can we do it better
Hadoop and object stores: Can we do it better?
Talend Big Data Tutorial | Talend DI and Big Data Certification | Talend Onli...
Aem asset optimizations & best practices
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
HDFS & ASM
Demystifying bigdata ashish1
Elasticsearch quick Intro (English)
Redis overview
Introduction to Redis Data Structures: Hashes
Using Pluggable Apache Spark SQL Filters to Help GridPocket Users Keep Up wit...
Big Data Analytics -Introduction education
Model Serving for Deep Learning
Right time Vs real time
NameNode Analytics - Querying HDFS Namespace in Real Time
Python Powered Data Science at Pivotal (PyData 2013)
Building an Apache Sling Rendering Farm
What is HDFS | Hadoop Distributed File System | Edureka
Techincal Talk Hbase-Ditributed,no-sql database
Ad

Recently uploaded (20)

PDF
LOW POWER CLASS AB SI POWER AMPLIFIER FOR WIRELESS MEDICAL SENSOR NETWORK
PPTX
CN_Unite_1 AI&DS ENGGERING SPPU PUNE UNIVERSITY
PDF
Design of Material Handling Equipment Lecture Note
PPTX
Feature types and data preprocessing steps
PDF
Introduction to Power System StabilityPS
PDF
August 2025 - Top 10 Read Articles in Network Security & Its Applications
PPTX
AUTOMOTIVE ENGINE MANAGEMENT (MECHATRONICS).pptx
PDF
UEFA_Embodied_Carbon_Emissions_Football_Infrastructure.pdf
PDF
August -2025_Top10 Read_Articles_ijait.pdf
PPTX
Chemical Technological Processes, Feasibility Study and Chemical Process Indu...
PDF
Java Basics-Introduction and program control
PDF
First part_B-Image Processing - 1 of 2).pdf
PPT
Chapter 1 - Introduction to Manufacturing Technology_2.ppt
PPTX
Software Engineering and software moduleing
PPTX
Sorting and Hashing in Data Structures with Algorithms, Techniques, Implement...
PPTX
CyberSecurity Mobile and Wireless Devices
PPTX
Chapter 2 -Technology and Enginerring Materials + Composites.pptx
PDF
20250617 - IR - Global Guide for HR - 51 pages.pdf
PPTX
Petroleum Refining & Petrochemicals.pptx
PPTX
PRASUNET_20240614003_231416_0000[1].pptx
LOW POWER CLASS AB SI POWER AMPLIFIER FOR WIRELESS MEDICAL SENSOR NETWORK
CN_Unite_1 AI&DS ENGGERING SPPU PUNE UNIVERSITY
Design of Material Handling Equipment Lecture Note
Feature types and data preprocessing steps
Introduction to Power System StabilityPS
August 2025 - Top 10 Read Articles in Network Security & Its Applications
AUTOMOTIVE ENGINE MANAGEMENT (MECHATRONICS).pptx
UEFA_Embodied_Carbon_Emissions_Football_Infrastructure.pdf
August -2025_Top10 Read_Articles_ijait.pdf
Chemical Technological Processes, Feasibility Study and Chemical Process Indu...
Java Basics-Introduction and program control
First part_B-Image Processing - 1 of 2).pdf
Chapter 1 - Introduction to Manufacturing Technology_2.ppt
Software Engineering and software moduleing
Sorting and Hashing in Data Structures with Algorithms, Techniques, Implement...
CyberSecurity Mobile and Wireless Devices
Chapter 2 -Technology and Enginerring Materials + Composites.pptx
20250617 - IR - Global Guide for HR - 51 pages.pdf
Petroleum Refining & Petrochemicals.pptx
PRASUNET_20240614003_231416_0000[1].pptx

Redis memory optimization sripathi, CTO hashedin

  • 1. Memory Optimization Sripathi Krishnan, CTO, HashedIn Technologies
  • 2. Copyright © 2017 HashedIn Technologies Pvt. Ltd. - Ensure the objects don’t have redundant data - Users have recommended products - If you cache users, ensure recommended_products is just a reference
  • 3. Copyright © 2017 HashedIn Technologies Pvt. Ltd. - Default serialization libraries are not efficient - Use Protobuf / Message Pack / Others - JSON must be compressed
  • 4. Copyright © 2017 HashedIn Technologies Pvt. Ltd. - Keys in redis have lots of overheads - For smaller strings, combine them into a larger hash - Tweak hash-max-ziplist-entries and hash-max-ziplist-value appropriately
  • 5. Copyright © 2017 HashedIn Technologies Pvt. Ltd. - Objects are usually saved in hashmap, but the keys are redundant if every object has the same keys - “NamedTuple”: store objects in a list instead - “C like struct”: Use Redis Bitfield command to store fixed length values in a single string
  • 6. Copyright © 2017 HashedIn Technologies Pvt. Ltd. - Sets in Redis are normally stored as HashTables - But if they only contain integers/longs, they are stored in memory-efficient “intsets” - 10x or more memory reduction if your IDs are numeric only
  • 7. Copyright © 2017 HashedIn Technologies Pvt. Ltd. - Count Objects? Use HyperLogLog - Check for Existence? Use Bloom Filter You won’t get exact results, but sometimes the memory savings are worth it.
  • 8. Copyright © 2017 HashedIn Technologies Pvt. Ltd. - Redis Strings can be used as a giant bitmap - You can set/unset/toggle individual bits Useful for things like access control - can this user perform that operation?
  • 9. Copyright © 2017 HashedIn Technologies Pvt. Ltd. - Sorted Sets are very expensive memory-wise - Tweak zset-max-ziplist-entries and zset-max-ziplist-value if you are using lots of smaller sorted sets
  • 10. Copyright © 2017 HashedIn Technologies Pvt. Ltd. - Sorted Sets are very expensive memory-wise - Tweak zset-max-ziplist-entries and zset-max-ziplist-value if you are using lots of smaller sorted sets
  • 11. Copyright © 2017 HashedIn Technologies Pvt. Ltd. Take the guess work out of your memory analysis. https://github.com/sripathikrishnan/redis-rdb-tools