How EnerKey Using InfluxDB Saves Customers Millions by Detecting Energy Usage Fluctuations Based on Weather and Geospatial Data
Download as PPTX, PDF0 likes348 views
Enerkey is a modern SaaS platform that integrates various energy management systems to improve energy efficiency, reporting, and sustainability for over 1,000 organizations. It utilizes InfluxDB for efficient data storage and processing, transitioning from an outdated MS SQL infrastructure to handle time series data effectively. The platform also incorporates machine learning to analyze energy consumption patterns and enhance operational effectiveness.
How EnerKey Using InfluxDB Saves Customers Millions by Detecting Energy Usage Fluctuations Based on Weather and Geospatial Data
- 2. Agenda • EnerKey company overview • Solution overview • Old system rewrite & InfluxDB selection process • Data collection architecture • Super Clever EnerKey – adding ML to the mix • Q&A
- 3. Wherewe are? Finland Population: 5 552 858 Area: 338 454 km2 Currency: EUR Languages: Finnish / Swedish
- 4. Save Energy Save MoneyIncrease productivity Sustainabilityand Savings Ourmission, yourvalue. Fulfilling stakeholder requirements for sustainability. Reducing energy consumption and environmental footprint. Certified ISO 50001 and ISO 14001 support.
- 5. 100 000+ 15 000+ 1st SaaS METERING POINTS PROPERTIES COMBINE SUSTAINABILITY AND ENERGY MANAGEMENT 1 000+ 80+ UP TO30% CUSTOMERS INTEGRATIONS AND INTERFACES VAAKA BUYOUT FUND AND MANAGEMENT 6M€ 60+ REVENUE 2019 PROFESSIONALS OWNED BY COST SAVINGS EnerKey facts
- 6. EnerKey in Europe Measurements active at present
- 7. WhatyougetwithEnerKey EnerKey A modern and intelligent SaaS •Fact-based savings and sustainabili ty •Support for 90+ types of •Energy •Productio n commodity •Transport fuels •Indoor air quality parameters •Waste and Emissions Best expertise and customer support Certified ISO 50001 and ISO 14001 support Seamless connectivit y AI Best expertise and customer support
- 8. •More than 80 out-of-the-shelf integrations to energy companies, building automation, measuring data systems, IoT devices, Solar PV systems, etc. •EnerKey supports seamless integrations to connected systems and services through state of the art API Connectivity If wedon’t have it, wewill build it.
- 9. • Energy company or Property Management System provider, modernize your existing energy reporting services with no additional investment and project-related risks. • Offer a superb customer experience and boost your competitiveness. • Expand your business by offering new services to generate added value for your customers. • Give your customers access to services for managing and splitting energy bills among tenants. • Virtual Energy Manager – expert services for managing energy and improving energy efficiency. Poweredby EnerKey Your logo, your brand.
- 10. References Somehighlights ofover1000 organizations that rely on EnerKey RETAIL INDUSTRY PUBLIC SECTOR POWERED BY ENERKEYLARGE PROPERTY OWNERS
- 11. References-Retail “Long co-operation with EnerKey has generated annual savings of 5 million euros.” “Energy saving is one of the key actions to combat climate change. Kesko is among the frontrunners in energy saving. We are well on track in meeting the objectives.” Matti Kalervo, Director of Corporate Responsibility, Kesko PLC
- 12. References – Powered by EnerKey “Among our corporate customers, there is a growing need to monitor, report and meter energy consumption and environmental impacts with the help of data.” “At Helen, we aim to provide our customers with the most powerful tools.” Jyrki Eurén, Head of B2B Business, Helen
- 13. Customer problemoverview • Real estate owners & managing companies have dozens of facilities distributed geologically • Different energy companies in various regions provide these facilities with water, district heating, electricity and 90+ more energy quantities • Data resides in various energy company portals Energy Management by Excel
- 14. Solutionoverview • EnerKey integrates to over 80+ different building automation systems and data sources • We collect the data via real time and scheduled integrations and harmonize the data to common energy consumption format at one hour resolution different quantities becomes comparable • InfluxDB is used to store the raw data as well as harmonized data. Metadata about facilities and buildings is stored in Azure SQL
- 15. Timeseriesproblem • Old EnerKey product used MS SQL server • As single database capacity was peaked, a manual process created next-in-sequence database • Readings1, Readings2,…,Readings6 • New EnerKey development started slightly on wrong foot • Continued use of MS SQL for timeseries data
- 16. EnerKeyproductand InfluxDBtimeline 12.6.2020 16 1995 Energiakolmio company founded 2014 Rewrite of EnerKey begins 2016 H1 New development faces perf problems with MS SQL storage 2016/10 I started working with the company as EA 2016/12 First talks with InfluxData 2017 H1 Performance and functional testing side by side other development 2017/12 Decision to buy InfluxDB Enterprise 2018 Very fast paced development Hybrid deployment with MS SQL and InfluxDB. 2019 New year First failed attempt to move all data to InfluxDB. Rollback. 2019 Q3 All data from legacy platform migrated to InfluxDB
- 17. InfluxDBdecisionfacts • PRO • High ingestion rate • High output rate • Group by time • Irregular intervals does not matter, we still get the sharp ANY supported resolution data • Natural upsert • Also has some caveats • CON • Lack of natural month • Quite easily mitigated by aggregation from days • At the time of selection, on-prem was the only feasible alternative, moving to cloud could be easier • Started in late 2016 • VM based Microsoft SQL Server Active/Passive cluster expensive and slow. Performance limits exceeded. • Open source at first, testing alongside other alternatives • Comprehensive testing during 2017 and GO decision made at end of year • Alternatives: • PostgreSQL with table manipulation • MongoDB with timeseries oriented schema • Cassandra with timeseries oriented schema • Native time handling biggest single decisive factor
- 18. Why InfluxDB Enterprise? • Business requirement for reliable storage • Some additional services includes billing based on gathered data • Data is not simple metrics business value • Support • Lots of testing and analysing before buy decision • Realized the need for first class support for new technology • Influx enterprise support has been VERY valuable • Performance case from early 2019 • Single rogue query caused 40-50% of CPU load • Enterprise support spotted this from our logs • Unbounded low limit when searching backwards for latest datapoint • Changed to exponentially widening sequential search: 1day, 2 days, 1 week, 1 month, 3 months, 12 months New data now 1 2 1 week 1 month 3 months Most likely hit
- 19. Dataacquisitionarchitecture Hangfire Scheduler Data Sources Public internet VPN tunnels Azure Service Bus Schedule Pull any format Convert to common data format Post to service bus Auto QA functions Detect faults, auto-fix Raw data storage Reading functions Raw data API Calculation functions Measurement API Reporting data storage Azure VNET - Normalize with temperatures - Aggregate to natural months DLQ • Scheduled tasks for pull functions • Listening functions for pushed data • Once “on the bus”, data is safe • Raw vs. reporting data allows manipulation without losing original values • Automatic quality checks fill in the blanks • Natural calendar aggregations performed outside InfluxDB • Normalization calculations with location & temperature allows comparison regardless of location Push any format
- 20. TICK stack • Telegraf is unfortunately ruled out mostly because we don’t have access to data sources at this level. • At EnerKey, we mostly pull the data from customer’s systems rather than push it to InfluxDB with Telegraf. • We do support also push type of integrations but in these cases the use of Telegraf has not been plausible. • Chronograf replaced with Grafana for extensive use for monitoring the platform as well as querying raw business data mixed with metadata from Azure SQL C T • InfluxDB widely used for business data storage, calculations and aggretations • InfluxDB also used for platform metrics I • Kapacitor initially planned for automatic data quality assurance, but later replaced by service bus-based solution. Not used. K
- 24. Adding intelligence • Baseline • EnerKey is a very good platform for Sustainability and Energy Management and Reporting • Data is secure, fast, and robust • Integrations in an out in place • Challenge • Competitors exist, but they’re mostly facility and real estate management systems with some Sustainability and Energy Management features • We needed to offer something that the others could not • Solution • Add Machine Learning models to pin point energy consumption profiles that are misbehaving
- 25. MachineLearningbasics • Harmonized consumption data for both main metering points and sub-metering points - Electricity consumption - Heating energy consumption • Good quality environment (weather) data - Temperature - Wind - Sun radiation • Good quality metadata - Gross area - Gross volume - Geolocation - Building year - Building type - Opening hours
- 30. Cooling energy in Grocery Stores 12.6.2020 30 • A regression line fitted to analyze the increase of energy consumption during warm days 𝑅𝑎𝑡𝑖𝑜 = 𝐴𝑣𝑒𝑟𝑎𝑔𝑒 ℎ𝑜𝑢𝑟𝑙𝑦 𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛 𝑤ℎ𝑒𝑛 𝑜𝑢𝑡𝑠𝑖𝑑𝑒 𝑡𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒 > 12 𝐴𝑣𝑒𝑟𝑎𝑔𝑒 ℎ𝑜𝑢𝑟𝑙𝑦 𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛 𝑤ℎ𝑒𝑛 𝑜𝑢𝑡𝑠𝑖𝑑𝑒 𝑡𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒 < 12 • A bar represents one facility • The color of the bar represents average electricity use per square meter • The slope represents the average increase of energy consumption when outside temperature rises one degree • A bar represents one facility • The correlation coefficient tells how good the estimate is. • A point represents one facility and it is colored red if the slope > 0.7
- 31. Coolingenergyresults • One medium sized grocery store stood out in the results compared to stores of similar size and location • The slope increase was substantiallysteeper when temperatures exceeded 20+°C • Note! Warmweather inFinland • After investigation byprofessionals on site, undersized condensing equipment was discovered and replaced by adequately sized ones • Excesscooling energy consumption did not occur any more
- 32. Thanks! Follow our story atLinkedInand our website:www.enerkey.com! Martti Kontula, CTO +358440160579 martti.kontula@enerkey.com
- 33. We look forward to bringing together our community of developers in this new format to learn, interact, and share tips and use cases. 8-9 June, 2020 Hands-On Flux Training www.influxdays.com 23-24 June, 2020 Virtual Experience