Kotlin @ Coupang Backed - JetBrains Day seoul 2018
19 likes2,342 views
Kotlin Backend @ Coupang discusses Coupang's adoption of Kotlin for backend development. Some of the key reasons for adopting Kotlin included improving code safety, readability and testability. Example uses of Kotlin included developing common libraries, components like a Korean tokenizer, a Kafka client, an audit tool and a product creation pipeline system. Spring Data Requery was also developed as an alternative to JPA/Hibernate that provided better performance. Overall, Kotlin helped improve code quality, simplify asynchronous programming and increase development productivity at Coupang.
Kotlin @ Coupang Backed - JetBrains Day seoul 2018
- 1. Kotlin Backend @ Coupang debop@coupang.com
- 2. Who is @debop • Since 1993 • Robotics, BPM Solution, Healthcare • C/C++, Object Pascal, C#, Java, Scala, Kotlin • Use Kotlin since 2016.08 • Sr. Principle Software Engineer in Coupang (2017~) • 남의 코드 뒤집기 / 빨간펜 선생 • Common Library 제작 • Architecture 참견
- 3. Agenda • Kotlin 도입 동기 • Kotlin 도입 사례 • Kotlinx (Common Library) • 공용 Library 개발 • spring-data-requery • Lesson & Learn
- 4. Kotlin 도입 동기
- 5. What is our problem? • 개발자 수준 편차가 크다 • 조직 수준은 최하위 개발자 수준의 의해 결정된다
- 6. Our status • Lack of Safety code • Not Readable code • Lack of Testing (Unit & Integration) • Heavy oriented to JPA • Lack of experience to Async/Non-Blocking Programming
- 7. Our environments • 1일 데이터 처리량 : 천만건(2016) -> 10억건(2018) • MySQL (Aurora) : Replication delay120 sec (max) • Backend Stack in 2016 • MySQL, RabbitMQ, Hive • Spring 3, JPA
- 8. Our Challenges • Akka Cluster with Lagom Framework (CQRS) • Move to NoSQL (Cassandra, HBase) • Migrate RabbitMQ to Kafka • Adopting Scala (Pilot) • Akka, Spark …
- 9. Failure • Adopting Scala • Learning Curve (Almost Junior) • Lack of Tech Leaders • Try to develop too complex system • Adopting Kafka • Simultaneous development (duplicate same error) • Offset managements
- 10. To pursue change • 객관적 시각 유지 • History 및 현실 상황 대한 이해 (기술부채) • 현 조직의 개발 역량 객관적 평가 • 동기부여 - 필요성 설득보다는 자각할 수 있도록 자극 • 충분한 학습 시간 • 변화 경험 공유 • ASP -> C#, Java -> Scala
- 11. Prepare • Code Quality 중요성 자각 -> Code Review • Enterprise Application Commons • Patterns 교육, Library 제공 • Enterprise Architecture 학습 • Asynchronous Programming 학습 • Kotlin Language 학습 • Kotlin is language for Android ?
- 12. Adopting Kotlin • Safety Code / Readable Code • Simple grammar (Easy to learn) • Interoperability between Java and Kotlin • Extension methods • Asynchronous Programming by Coroutines
- 13. Environments 준비 • 개발 Tool 지원 - IntelliJ IDEA • Static code analysis - ktlint, detekt • Test code coverage - jacoco (sample) • Sonarcube - sonar-kotlin plugin
- 14. Kotlin Backend 도입 사례
- 15. Use cases 2017 • Common Library (2017. 6 ~) • Kotlin extension library • Components (2017.08 ~ 2017.12) • Aho-corasick, Korean Tokenizer, Kafka Client • Standalone Web Application (2017.10~~2017.12) • Audit Tool • Large scale system (2017.09~2018.01) • Catalog creation pipeline system
- 16. 1. Kotlin Extension library • Coupang Standard Framework 대체 • Kotlin Best practices 제공 • Object / IO / Utils / Cache / JDBC / Spring … • Kotlin 학습 예제 제공 (Test cases / Examples) • Based debop4k (personal project)
- 17. Kotlinx-Units • 도량형(度量衡)(Standard Unit) • Represent Meter Units • Weight, Length, Area, Volume, Storage Volume • Yard-Pound Units • Pound, Yard/Miles, Volume, Fluid Volume … • Others • Temperature, Electlic powers, Frequency, Light, Radiation, Pressure …
- 18. Our usages enum class Unit(val description: String, val unitConversion: BigDecimal) { NONE("없음", BigDecimal.ZERO), // Length MM("mm", 0.1.toBigDecimal()), CM("cm", BigDecimal.ONE), M("m", 100.toBigDecimal()), KM("km", 100_000.toBigDecimal()),
- 19. Our usages // Length (Inch) IN("in", BigDecimal.ONE), FT("ft", 12.toBigDecimal()), YD("yd", 36.toBigDecimal()), // Weight (gram) MG("mg", 0.001.toBigDecimal()), G("g", BigDecimal.ONE), KG("kg", 1000.toBigDecimal()), T("t", 1_000_000.toBigDecimal()), OZ("oz", 28.349523.toBigDecimal()), LB("lb", 453.59237.toBigDecimal()), •Define Units by Enum •Not support conversions •Not support various units •Not support operations •Not readable code
- 20. Features for units • Readable code like this • 5.kg() + 400.gram() => 5.4.kg() • 100.cm() + 2.m() => 2.1.cm() • Convert various units • 5.kg().inGram() -> 5000.gram • 2.5.Kb().inBytes() -> 2500.bytes() • Convert SI Units <-> US Units
- 21. Use Kotlin extension methods fun Double.millimeter(): Length = lengthOf(this, MILLIMETER) fun Double.centimeter(): Length = lengthOf(this, CENTIMETER) fun Double.meter(): Length = lengthOf(this) fun Double.kilometer(): Length = lengthOf(this, KILOMETER)
- 22. 1. Define Unit factors enum class LengthUnit(val unitName: String, val factor: Double) { MILLIMETER("mm", 1.0), CENTIMETER("cm", 10.0), METER("m", 1e3), KILOMETER("km", 1e6); }
- 23. Define Unit class data class Length(val value: Double = 0.0): Comparable<Length>, Serializable { operator fun plus(other: Length): Length = Length(value + other.value) operator fun minus(other: Length): Length = Length(value - other.value) operator fun times(scalar: Int): Length = Length(value * scalar) operator fun times(scalar: Long): Length = Length(value * scalar) operator fun times(scalar: Double): Length = Length(value * scalar) operator fun times(other: Length): Area = Area(value * other.value) operator fun div(scalar: Int): Length = Length(value / scalar) operator fun div(scalar: Long): Length = Length(value / scalar) operator fun div(scalar: Double): Length = Length(value / scalar) operator fun unaryMinus(): Length = Length(-value) Override Operators
- 24. 2. Korean Tokenizer • 중복상품 Merge 위한 Tokenizer 필요 (명사 위주) • Twitter 에서 개발한 open-korean-text 를 Customizing • Scala vs Kotlin 성능 비교 • Kotlin version is 1.5 ~ 3X faster with Coroutines • 효과 • Full Indexing per Day 부하 감소 : 30% • Elastic Search 질의 부하 : 80%
- 25. “제테스 MSW-3928PR 남성 비치트렁크, 95(L), 블랙블랙계열” 은전한닢 Twitter 블랙 비치 비치트렁크 계열 제테스 3928PR 블랙블랙계열 PR MSW , ( )- 트렁크 3928 95 제테스 남성
- 26. “제테스 MSW-3928PR 남성 비치트렁크, 95(L), 블랙블랙계열” 은전한닢 Twitter 블랙 비치 비치트렁크 계열 제테스 3928PR 블랙블랙계열 PR MSW , ( )- 트렁크 3928 95 제테스 남성
- 27. “제테스 MSW-3928PR 남성 비치트렁크, 95(L), 블랙블랙계열” 은전한닢 Twitter 블랙 비치 비치트렁크 계열 제테스 3928PR 블랙블랙계열 PR MSW , ( ) L - 트렁크 3928 95 제테스 남성
- 28. Tokenizer Benchmark 은전한닢 Twitter RATIO RPS 73.2 429.5 5.87 X Avg Latency 626 ms 106 ms 5.91 X Avg CPU Load 90% 55% 35 % Down Twitter Tokenizer 는 한음절을 분석하지 못하는 단점이 있다
- 29. Scala Kotlin RATIO Tokenize 668.620 ms 197.632 ms 3.38 X Phrase extractor 644.902 ms 212.500 ms 3.13 X 구문 : “동해물과 백두산이 마르고 닳도록“ Benchmark by jmh Linux Mint 18.2 Intel I7, 32GB, SSD
- 30. Why Kotlin is faster? • Scala 의 loop는 느리다 - 아주 느릴 수 있다 • eclipse-collections 사용 • 메모리 절약 • Primitive type collection 지원 • Kotlin Coroutines • 비동기 / Non-Blocking
- 31. 3. Kafka Client - Wilson • 동기 • 안정된 Kafka 사용을 위해 Offset 관리가 필수 • 각 팀별 중복된 Client 구현 및 삽질 • 효과 • Message 중복 / 유실 없음 (Latest strategy) • Retry / Circuit breaker 지원 • Metrics 를 활용한 Ack 지원 • 전사 확대 적용 중
- 32. Wilson message flows Producer ConsumerKafka Retry Circuit Breaker Metrics Retry Circuit Breaker Metrics Dead letters Sending box Redis MySQL Couchbase Dead letters Received box Last sent timestamp Kafka Offset Manager Message Managements • Metrics • Recovery / Retry • Deadletter handling
- 33. Wilson Dashboard
- 34. 4. Audit Tool • 상품 정보 Audit System • developers : 1 senior, 2 junior developer • Software stack • React • Spring Boot 1.5 on Vitamin Framework • jOOQ (향후 requery로 변환 예정) • Pilot 로 시작, 개발자들의 노력으로 정식 시스템으로 승격
- 35. 4. Audit Tool Kotlin Coroutines + Spring MVC
- 36. 5. Creation Pipeline System • 상품 정보 생성 프로세스 관리 시스템 • Features • Workflow (Heavy use Kafka) • Asynchronous / Non-blocking System • Need High throughput
- 37. Seller Retail Kafka Deduplication Refining Processing Creation Logging Message Dispatcher Creation Pipeline flow Kafka
- 38. 5. Creation Pipeline System Spring Boot 1.x Kafka 0.10.x Kotlin 1.2.x on JVM 8 Couchbase 4.x Zookeeper 3.x
- 39. %
- 40. %
- 41. Use cases 2018 • Tagging system (2018.1 ~ 2018.3) • Spring Data Requery (alternative to JPA) (2018.4 ~) • Rule engine (Thanos rule engine) (2018.6 ~) • Wakanda system (Reactive + Coroutines) (2018.9 ~) • Creation Pipeline Upgrade (2018.10 ~)
- 43. Aurora Cassandra ElasticSearch HBase JVM 8 Spring DataRequery JPA/Hibernate Virtualization kubernetes Docker Spring Framework 5.xKodein Kotlin Coroutines with Reactor / RxJava Services Kafka Redis Spring Boot 2.x Webflux with Netty / gRPC Common Backend Stack (2018)
- 45. Spring MVC + Cassandra Spring WebFlux + Cassandra Reactive 출처: Reactive Java Performance Comparison
- 46. 6. Tagging System • 상품 정보에서 원하는 특성을 추출하는 시스템 • Matching 과 Refinement 에서 따로 관리 • 범용화 필요 (Configuration, Plugin) • 기존 Java & Scala 혼용 • 장애 대응에 문제 • High Throughput
- 47. Tagging for Matching 스톤 9.0cm 여성하이힐 MZ-135 블랙 245 Brand Gender Shoes Type Color Size
- 48. 7. spring-data-requery (2018) • RequeryOperations • Wrap EntityDataStore • RequeryTransactionManager for TransactionManager • Support Spring @Transactional • Better performance than spring-data-jpa • when exists, paging, not load all entities https://www.slideshare.net/debop/alternatives-of-jpahibernate https://www.slideshare.net/debop/requery-overview https://www.slideshare.net/debop/spring-data-requery
- 49. spring-data-requery • Repository built in SQL • ByPropertyName Auto generation methods • @Query for Native SQL Query • Query By Example • Not Supported • Association Path (not specified join method) • Named parameter in @Query (just use `?`)
- 50. Define Entity - Java @Getter @Entity(name = "BasicUser", copyable = true) @Table(name = "basic_user") public abstract class AbstractBasicUser extends AuditableLongEntity { @Key @Generated protected Long id; protected String name; protected String email; protected LocalDate birthday; protected Integer age; @ForeignKey @OneToOne protected AbstractBasicLocation address; @ManyToMany(mappedBy = "members") protected Set<AbstractBasicGroup> groups; @Column(unique = true) protected UUID uuid;
- 51. Use @Query in Repository interface DeclaredQueryRepository extends RequeryRepository<BasicUser, Long> { @Query("select * from basic_user u where u.email = ?") BasicUser findByAnnotatedQuery(String email); @Query("select * from basic_user u where u.email like ?") List<BasicUser> findAllByEmailMatches(String email); @Query("select * from basic_user u limit ?") List<BasicUser> findWithLimits(int limit); @Query("select * from basic_user u where u.name=? and u.email=? limit 1") BasicUser findAllBy(String name, String email); @Query("select u.id, u.name from basic_user u where u.email=?") List<Tuple> findAllIds(String email); @Query("select * from basic_user u where u.birthday = ?") List<BasicUser> findByBirthday(LocalDate birthday); }
- 52. Query By Example BasicUser user = RandomData.randomUser(); user.setName("example"); requeryTemplate.insert(user); BasicUser exampleUser = new BasicUser(); exampleUser.setName("EXA"); ExampleMatcher matcher = matching() .withMatcher("name", startsWith().ignoreCase()) .withIgnoreNullValues(); Example<BasicUser> example = Example.of(exampleUser, matcher); Return<? extends Result<BasicUser>> query = buildQueryByExample(example); BasicUser foundUser = query.get().firstOrNull(); assertThat(foundUser).isNotNull().isEqualTo(user);
- 53. Query by Property List<User> findByFirstnameOrLastname(String firstname, String lastname); List<User> findByLastnameLikeOrderByFirstnameDesc(String lastname); List<User> findByLastnameNotLike(String lastname); List<User> findByLastnameNot(String lastname); List<User> findByManagerLastname(String name); List<User> findByColleaguesLastname(String lastname); List<User> findByLastnameNotNull(); @Query("select u.lastname from SD_User u group by u.lastname") Page<String> findByLastnameGrouped(Pageable pageable); long countByLastname(String lastname); int countUsersByFirstname(String firstname); boolean existsByLastname(String lastname); Note: Association Path is not supported Note: Association Path is not supported
- 54. Future works • Coroutines for Async JDBC Operations (working) • Support Named parameter • Support `@Param` in spring data • Requery for SQL on Hadoop • Apache Phoenix (HBase) (Done) • Apache Hive, Apache Drill …
- 55. 8. Rule Engine (2018) • Embedded • Support Language • MVEL (almost like Java spec) • KotlinScript • Javascript • Support Multi Rule combination • Rule Definition DSL (Kotlin) • Rule Editor in WEB (in construction)
- 56. Rule Engine A rule engine can be viewed as a sophisticated interpreter for if/then statements, where the statements themselves are known as rules.
- 57. Rule Definition val rule = rule { name = "myRule" description = "myDescription" priority = 3 condition { condition1.evaluate(it) } action { action1.execute(it) } action { action2.execute(it) } } val rule = RuleBuilder() .name("myRule") .description("myDescription") .priority(3) .whenever(condition) .then(action1) .then(action2) .build() @Rule(name = "weather rule", description = "if it rains then take an umbrella") class WeatherRule { @Condition fun itRains(@Fact("rain") rain: Boolean): Boolean = rain @Action fun takeAnUmbrella() { println("It rains, take an umbrella") } }
- 58. 9. Wakanda • Catalog 정보 관리 시스템의 핵심 • Read, Write Side 분리 제공 • 전사 통합에 의해 Write side만 존치 • Fully Reactive / Non-Blocking
- 60. Cassandra ElasticSearch JVM 8 Cassandra Vanilla Driver Virtualization kubernetes Docker Spring Framework 5.x Kotlin Coroutines with Reactor Services Kafka Redis Spring Boot 2.x Webflux Wakanda Software stack
- 61. Lesson & Learns
- 62. 반성해야 하는 점 • 기본기 학습 - 닥치고 코딩 (X) • Java와 차이점 및 Kotlin best practices 검토 필요 • 첫 술에 배부를 수 없다 • 실망하지 말라, Refactoring 은 필수다 • Coroutines 에 대한 학습 및 테스트 • 어차피 비동기는 어렵다. • Coroutines는 하나의 방법일 뿐이다
- 63. 안정적 도입을 위한 Tips • 충분한 학습 기회 & 실습 • Code Quality 중요성 인식 • Upsource 전사 활용 중 (문화가 중요) • 강력한 동기 부여 • Tech Leader의 지속적인 Leading & Coach • 성공 사례 만들기 (작은 것부터)
- 64. 효과 • Safety Code • Readable Code • 성능 향상 • Latency 감소, High Throughput 달성 • Asynchronous/Non-Blocking 적용 용이 • 유지보수성 향상 (장애 감소)
- 65. 언어별 Catalog Tribe 개발자 비율 (2017) 5% Kotlin 10% Scala 15% Java 70% Java Scala Kotlin Python 언어별 Catalog Tribe 개발자 비율 (2018) 17% Kotlin 40% Scala 13% Java 30% Java Scala Kotlin Python
- 66. Resources • Kotlin Resources • kotlinx.coroutines • 20 Excellent Resources for learning Kotlin • Books • Kotlin in action, Joy of Kotlin • Try Kotlin
- 67. Q&A
- 68. Thank you!