OSMC 2021 | Introduction into OpenSearch
0 likes954 views
OpenSearch is a community-driven, open source search and analytics suite derived from Elasticsearch 7.10.2 and Kibana 7.10.2. It was initiated by AWS and consists of the OpenSearch search engine and OpenSearch Dashboards visualization interface. OpenSearch aims to provide a true open source search and analytics engine following licensing changes by Elastic that removed Elasticsearch's open source status.
![© 2021 floragunn GmbH - All Rights Reserved
WHAT IS OPENSEARCH?
03.
“OpenSearch is a community-driven, open source search and analytics suite
derived from Apache 2.0 licensed Elasticsearch 7.10.2 & Kibana 7.10.2. It consists
of a search engine daemon, OpenSearch, and a visualization and user interface,
OpenSearch Dashboards. […] With OpenSearch people benefit from having an
open source product they can use, modify, extend, monetize, and resell how they
want.”
Source: https://opensearch.org/](https://image.slidesharecdn.com/introductiontoopensearch-211119125636/85/OSMC-2021-Introduction-into-OpenSearch-5-320.jpg)
![© 2021 floragunn GmbH - All Rights Reserved
LICENSE CHANGE: EFFECTS ON FOSS
Elasticsearch is not Open Source anymore
As per definition of the Open Source Initiative (OSI)
Source: https://opensourceconnections.com/blog/2021/01/15/is-elasticsearch-no-longer-open-source-software/
Source: https://opensource.org/node/1099
Elasticsearch cannot be offered as a managed service anymore
“You may not provide the products to others as a managed service”
Source: https://www.elastic.co/pricing/faq/licensing#what-are-the-key-changes-being-made-to-the-elastic-license
Contributions only possible under non-OSS licenses
“You can […] contribute […], whether that code is under the Elastic License or SSPL”
Permissive license, but not approved by Open Source Initiative as OSS license
Source: https://www.elastic.co/pricing/faq/licensing#what-are-the-key-changes-being-made-to-the-elastic-license
13.](https://image.slidesharecdn.com/introductiontoopensearch-211119125636/85/OSMC-2021-Introduction-into-OpenSearch-15-320.jpg)
![© 2021 floragunn GmbH - All Rights Reserved
ELASTIC
“So why the change? AWS and Amazon Elasticsearch Service. They have been
doing things that we think are just NOT OK”
Source: https://www.elastic.co/what-is/opensearch
“Our license change is aimed at preventing companies from taking our […]
products and providing them directly as a service without collaborating with us”
Source: https://www.elastic.co/what-is/opensearch
“We think that Amazon’s behavior is inconsistent with the norms and values that
are especially important in the open source ecosystem.”
Source: https://www.elastic.co/what-is/opensearch
“The change from the Apache License to the Elastic License v2 was a response to
the continued non-collaborative engagement AWS pursued with Elasticsearch”
Source: https://www.elastic.co/what-is/opensearch
15.](https://image.slidesharecdn.com/introductiontoopensearch-211119125636/85/OSMC-2021-Introduction-into-OpenSearch-18-320.jpg)
![© 2021 floragunn GmbH - All Rights Reserved
AWS
“Developers embrace open source software for many reasons, perhaps the most
important being the freedom to use that software where and how they wish.”
Source: https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/
“[…] we followed the recommended open source development practice of
“upstream first.” All changes to Elasticsearch were sent as upstream pull requests”
Source: https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/
“[…] to ensure open source versions […] remain available […] AWS will create and
maintain a ALv2-licensed fork of open source Elasticsearch and Kibana”
Source: https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/
“They believe that restricting their license will lock others out of offering managed
Elasticsearch services, which will let Elastic build a bigger business.”
Source: https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/
16.](https://image.slidesharecdn.com/introductiontoopensearch-211119125636/85/OSMC-2021-Introduction-into-OpenSearch-19-320.jpg)
![© 2021 floragunn GmbH - All Rights Reserved
COMMUNITY
“[…] if you choose […] FOSS, that means something, and you have the moral
obligation to uphold”
Source: https://drewdevault.com/2021/01/19/Elasticsearch-does-not-belong-to-Elastic.html
“The whole point of open source is that amazon has a right to do this”
Source: https://www.reddit.com/r/programming/comments/l0m56s/amazon_not_ok_why_we_had_to_change_elastic/
“Amazon went too far, and now the pendulum is swinging back.”
Source: https://www.reddit.com/r/programming/comments/l0m56s/amazon_not_ok_why_we_had_to_change_elastic/
“I can't say I morally disagree with this move.”
https://www.reddit.com/r/linux/comments/kxa4ri/elasticsearch_and_kibana_changing_license_from/
“It's corporate vs corporate.”
https://www.reddit.com/r/linux/comments/kxa4ri/elasticsearch_and_kibana_changing_license_from/
17.](https://image.slidesharecdn.com/introductiontoopensearch-211119125636/85/OSMC-2021-Introduction-into-OpenSearch-20-320.jpg)
OSMC 2021 | Introduction into OpenSearch
- 1. AN INTRODUCTION TO OPENSEARCH © 2021 floragunn GmbH - All Rights Reserved AND ITS IMPACT ON FOSS
- 2. © 2021 floragunn GmbH - All Rights Reserved ABOUT ME Jochen Kressin, Co-Founder of floragunn GmbH Makers of Search Guard Enterprise Security and Alerting Suite for the Elastic Stack and OpenSearch Founded 2012 Main office: Berlin, Germany Partner offices: Seattle, New York, Miami, Bordeaux, Mexico Disclaimer: I am neither employed by AWS or Elastic 01.
- 3. © 2021 floragunn GmbH - All Rights Reserved MY HISTORY WITH SEARCH AND SECURITY Lucene-based projects since 2004 Scaling and concurrency always an issue SOLR Solved many problems out of the box Elasticsearch Huge adoption rate, great ecosystem OpenSearch True FOSS search and analytics engine 02.
- 4. © 2021 floragunn GmbH - All Rights Reserved And what it means for FOSS
- 5. © 2021 floragunn GmbH - All Rights Reserved WHAT IS OPENSEARCH? 03. “OpenSearch is a community-driven, open source search and analytics suite derived from Apache 2.0 licensed Elasticsearch 7.10.2 & Kibana 7.10.2. It consists of a search engine daemon, OpenSearch, and a visualization and user interface, OpenSearch Dashboards. […] With OpenSearch people benefit from having an open source product they can use, modify, extend, monetize, and resell how they want.” Source: https://opensearch.org/
- 6. © 2021 floragunn GmbH - All Rights Reserved STATUS First announcement April 2021 OpenSearch 1.0 initial release July 2021 OpenSearch 1.1 released October 2021 ASLv2 licensed features of Elasticsearch 7.10.2 Features from Open Distro for Elasticsearch by AWS Roadmap https://github.com/orgs/opensearch-project/projects/1 04. Source: https://opensearch.org/versions/opensearch-1-0-1.html
- 7. © 2021 floragunn GmbH - All Rights Reserved OVERVIEW Elasticsearch 7.10.2 OSS / ASLv2 Com. Plugins 7.10.2 Elastic License OpenSearch Open Distro for Elasticsearch OSS / ASLv2 OSS / ASLv2 05.
- 8. © 2021 floragunn GmbH - All Rights Reserved OVERVIEW OpenSearch OpenSearch Dashboards PlugIns Security Alerting SQL Index Management KNN Anomaly Detection Async Search Performance Analyzer Replication Ingest / Data Prepper 06.
- 9. © 2021 floragunn GmbH - All Rights Reserved RESOURCES Project website https://opensearch.org/ Forum https://discuss.opendistrocommunity.dev/ Source Code https://github.com/opensearch-project/ Community Meetings https://www.meetup.com/Open-Distro-for-Elasticsearch-Meetup-Group 07.
- 10. © 2021 floragunn GmbH - All Rights Reserved FIRST QUESTIONS Does that mean OpenSearch is a fork of Elasticsearch? Yes, OpenSearch was forked from Elasticsearch OSS v7.10.2 Adds features from “Open Distro for Elasticsearch” (AWS) What license does OpenSearch use? OpenSearch is licensed under ALSv2 Does OpenSearch include any commercial features of Elasticsearch? No, only code that was originally licensed ASLv2 by Elastic Who is behind OpenSearch? OpenSearch is a community project initiated by AWS 08.
- 11. © 2021 floragunn GmbH - All Rights Reserved STRUCTURE Initiated by AWS “Amazon is the primary steward and maintainer of OpenSearch today” Source: https://aws.amazon.com/blogs/opensource/introducing-opensearch/ Designed to be a community driven project Source: https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/ Supported by other big players in the industry RedHat, SAP, Capital One, logz.io … Source: https://aws.amazon.com/blogs/opensource/introducing-opensearch/ Looking for partners and maintainers Source: https://opensearch.org/blog/community/2021/08/community-clients/ 10.
- 12. © 2021 floragunn GmbH - All Rights Reserved Why, pt 1
- 13. © 2021 floragunn GmbH - All Rights Reserved LICENSE CHANGES Elasticsearch licensing model - until January 2021 Free, ASLv2 licensed version with basic features Commercial add-ons https://www.elastic.co/blog/doubling-down-on-open License change - January 2021 Replaced ASLv2 with a dual license model: SSPL & Elastic License v1 https://www.elastic.co/blog/license-change-clarification License change - February 2021 Introduced Elastic License v2 which replaced Elastic License v1 Permissive license, but not approved by the OSI https://www.elastic.co/blog/elastic-license-v2 11.
- 14. © 2021 floragunn GmbH - All Rights Reserved OVERVIEW Elasticsearch OSS / ASLv2 Plugins Elastic License Elasticsearch + Plugins SSPL or Elastic License Source: https://www.elastic.co/blog/licensing-change Elasticsearch + Plugins SSPL or Elastic License v2 12.
- 15. © 2021 floragunn GmbH - All Rights Reserved LICENSE CHANGE: EFFECTS ON FOSS Elasticsearch is not Open Source anymore As per definition of the Open Source Initiative (OSI) Source: https://opensourceconnections.com/blog/2021/01/15/is-elasticsearch-no-longer-open-source-software/ Source: https://opensource.org/node/1099 Elasticsearch cannot be offered as a managed service anymore “You may not provide the products to others as a managed service” Source: https://www.elastic.co/pricing/faq/licensing#what-are-the-key-changes-being-made-to-the-elastic-license Contributions only possible under non-OSS licenses “You can […] contribute […], whether that code is under the Elastic License or SSPL” Permissive license, but not approved by Open Source Initiative as OSS license Source: https://www.elastic.co/pricing/faq/licensing#what-are-the-key-changes-being-made-to-the-elastic-license 13.
- 16. © 2021 floragunn GmbH - All Rights Reserved Why, pt. II The Elephant in the Room OR Two sides of the story
- 17. © 2021 floragunn GmbH - All Rights Reserved BACKGROUND Elastic published portions of their code under ASLv2 since the beginning OSS project, typical benefits from community contributions Also offered commercial, paid features AWS offered a (paid) managed service based on that code “Amazon Elasticsearch Service” (AESS), launched 2015 Source: https://aws.amazon.com/blogs/aws/new-amazon-elasticsearch-service/ Elastic began offering their own managed service “Elastic Cloud”, 2015 “Elastic Cloud Enterprise” (ECE), launched 2017 Source: https://www.elastic.co/about/history-of-elasticsearch 14.
- 18. © 2021 floragunn GmbH - All Rights Reserved ELASTIC “So why the change? AWS and Amazon Elasticsearch Service. They have been doing things that we think are just NOT OK” Source: https://www.elastic.co/what-is/opensearch “Our license change is aimed at preventing companies from taking our […] products and providing them directly as a service without collaborating with us” Source: https://www.elastic.co/what-is/opensearch “We think that Amazon’s behavior is inconsistent with the norms and values that are especially important in the open source ecosystem.” Source: https://www.elastic.co/what-is/opensearch “The change from the Apache License to the Elastic License v2 was a response to the continued non-collaborative engagement AWS pursued with Elasticsearch” Source: https://www.elastic.co/what-is/opensearch 15.
- 19. © 2021 floragunn GmbH - All Rights Reserved AWS “Developers embrace open source software for many reasons, perhaps the most important being the freedom to use that software where and how they wish.” Source: https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/ “[…] we followed the recommended open source development practice of “upstream first.” All changes to Elasticsearch were sent as upstream pull requests” Source: https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/ “[…] to ensure open source versions […] remain available […] AWS will create and maintain a ALv2-licensed fork of open source Elasticsearch and Kibana” Source: https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/ “They believe that restricting their license will lock others out of offering managed Elasticsearch services, which will let Elastic build a bigger business.” Source: https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/ 16.
- 20. © 2021 floragunn GmbH - All Rights Reserved COMMUNITY “[…] if you choose […] FOSS, that means something, and you have the moral obligation to uphold” Source: https://drewdevault.com/2021/01/19/Elasticsearch-does-not-belong-to-Elastic.html “The whole point of open source is that amazon has a right to do this” Source: https://www.reddit.com/r/programming/comments/l0m56s/amazon_not_ok_why_we_had_to_change_elastic/ “Amazon went too far, and now the pendulum is swinging back.” Source: https://www.reddit.com/r/programming/comments/l0m56s/amazon_not_ok_why_we_had_to_change_elastic/ “I can't say I morally disagree with this move.” https://www.reddit.com/r/linux/comments/kxa4ri/elasticsearch_and_kibana_changing_license_from/ “It's corporate vs corporate.” https://www.reddit.com/r/linux/comments/kxa4ri/elasticsearch_and_kibana_changing_license_from/ 17.
- 21. © 2021 floragunn GmbH - All Rights Reserved Do we need to rethink FOSS?
- 22. © 2021 floragunn GmbH - All Rights Reserved “FOSS SPIRIT” - Theory In essence Here’s my software, do whatever you want with it Including: Make money with it (if you find a way) Would be great if you contribute The goal Get developers on board to contribute (mostly for free) Drive development Make the project known in the FOSS space Increase adoption rate (because it’s free) 21.
- 23. © 2021 floragunn GmbH - All Rights Reserved “FOSS SPIRIT” - Practice Most projects remain “hobby” projects Never gain enough traction to “take off” Many projects struggle “One person projects”, developed by FTEs in their spare time Lack of funding, lack of developers, lack of time Successful projects often backed by companies or organizations Apache Software Foundation, RedHat etc. Example: hapi.js https://github.com/hapijs/hapi/issues/4111 22.
- 24. © 2021 floragunn GmbH - All Rights Reserved HAPI.JS 23.
- 25. © 2021 floragunn GmbH - All Rights Reserved MY PERSONAL OPINION Choose your weapons and license wisely Fully understand what the license allows and disallows It’s either FOSS, or it is not You can’t enjoy the advantages and avoid the disadvantages at the same time FOSS might not be the best option for your project Again, understand the advantages and disadvantages of your license choice Stick to your guns and license If you have chosen a license, stick with it What if Lucene would change its license? 25.
- 26. © 2021 floragunn GmbH - All Rights Reserved MY PERSONAL OPINION Wide adoption often only possible with a permissive license Apache web server, nginx, mySQL, K8s, Lucene etc. Managed services explicitely allowed FOSS is not a business model! FOSS is a philosophy backed by ethics, norms and values It’s not a means to generate revenue Can not be the (only) foundation of a company no FOSS no Future 26.
- 27. © 2021 floragunn GmbH - All Rights Reserved Learnings
- 28. © 2021 floragunn GmbH - All Rights Reserved JUST FORKING IS NOT ENOUGH Forking is easy, developing is challenging, maintaining is hard It’s an ecosystem, not a single product Logstash, Beats, Client Libraries etc. Keeping development over all components aligned is a challenge Testing / QA is a challenge Many different setups and usecases Many different plugins (Alerting, SQL, Security etc.) 26.
- 29. © 2021 floragunn GmbH - All Rights Reserved COMMUNITY & ENTERPRISE Community building requires effort, dedication and steadiness Community support and engagement Enterprises require trust and stability Production readiness Enterprise support including SLAs Professional services 27.
- 30. © 2021 floragunn GmbH - All Rights Reserved Q&A
- 31. SEARCH GUARD info@search-guard.com © 2021 floragunn GmbH - All Rights Reserved send us a message: 31
- 32. © 2021 floragunn GmbH - All Rights Reserved floragunn GmbH Tempelhofer Ufer 16 10963 Berlin Germany E-Mail: info@search-guard.com Web: search-guard.com Managing Directors: Claudia Kressin, Jochen Kressin Registergericht: Amtsgericht Charlottenburg Registernummer: HRB 147010 B E-Mail: info@floragunn.com Search Guard is a trademark of floragunn GmbH, registered in the U.S. and in other countries. Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. OpenSearch is a trademark of AWS floragunn GmbH is not affiliated with Elasticsearch BV. or AWS