apidays LIVE New York 2021 - Designing API's: Less Data is More! by Damir Svrtan, Netflix
Download as PPT, PDF0 likes65 views
The document discusses best practices for API design, emphasizing the principle of creating a minimal API surface to avoid redundancy and unnecessary overhead. It outlines two main principles: designing from strict to loose and avoiding extra flexibility to enhance developer experience and prevent future complications. The author argues that it is easier to add features in the future than to remove them and encourages thoughtful, schema-first design approaches based on client needs.
![type Post {
title:String
text:String
reviewed:Boolean
reviewers: [Reviewer]
}
type Post {
title:String
text:String
reviewed:Boolean
reviewer: Reviewer
}](https://image.slidesharecdn.com/damirdesigning-apis-api-days-nyc-210814034443/85/apidays-LIVE-New-York-2021-Designing-API-s-Less-Data-is-More-by-Damir-Svrtan-Netflix-33-320.jpg)
![comments(postId: ID): [Comment]](https://image.slidesharecdn.com/damirdesigning-apis-api-days-nyc-210814034443/85/apidays-LIVE-New-York-2021-Designing-API-s-Less-Data-is-More-by-Damir-Svrtan-Netflix-67-320.jpg)
apidays LIVE New York 2021 - Designing API's: Less Data is More! by Damir Svrtan, Netflix
- 1. Designing APIs Damir Svrtan Less Data is More
- 2. Damir Svrtan He/him Senior Software Engineer @ Netflix Twitter: @DamirSvrtan
- 4. Zagreb 🇭🇷
- 5. Designing APIs Less Data is More
- 6. Avoiding overhead when designing APIs
- 7. Bloated overly flexible APIs
- 8. What kind of APIs? • HTTP based APIs • REST APIs, JSON APIs, GraphQL etc
- 9. API technology agnostic Applicable for all kinds of APIs
- 11. Blogging Platform: • Authors can release posts • Posts can have comments
- 12. Two principles for building APIs • Designing the minimal API surface • Designing from Strict to Loose
- 13. 1st Principal Design the minimal API surface
- 14. Bloated API surface: - redundant fields - redundant relationships - redundant input fields
- 16. Requirement: “Show the author of the blogpost”
- 17. CREATE TABLE authors ( id first_name last_name email avatar_url );
- 19. Friendly API developer: “Let’s expose the email, someone might find it useful in the future”
- 21. 6 months later: Privacy reasons: stop exposing the email
- 23. ..extreme amounts of communication..
- 25. ..extreme amounts redundant cycles..
- 27. Requirement: “Indicate whether a post was reviewed”
- 30. Friendly API developer: “Let’s expose the reviewer for future use cases”
- 32. Development costs are growing; - batch/eager loading - testing - performance impacts
- 33. type Post { title:String text:String reviewed:Boolean reviewers: [Reviewer] } type Post { title:String text:String reviewed:Boolean reviewer: Reviewer }
- 36. Delaying Decisions Adding later when we have more knowledge is better
- 37. It’s easier to add in the future than to remove
- 38. Don’t Expose Redundant Input Fields
- 39. Requirement: “Readers need to be able to create and update comments”
- 40. input CreateComment { postId: ID body: String }
- 41. The friendly API developer: “Let’s have parity between inputs”
- 42. input UpdateComment { id: ID postId: ID body: String }
- 43. The blogging platform doesn’t support comment transitions from one post to another
- 44. input UpdateComment { id: ID postId: ID body: String }
- 45. Avoid anemic data modeling
- 48. You Ain’t Gonna Need It
- 49. 2nd Principal Strict to Loose
- 50. - Avoid extra flexibility - Breaks First!
- 52. Requirement: “The users need to be able to see comments on a post”
- 54. The friendly API developer: “Our clients might want to fetch all comments in the future”
- 58. More code = More maintenance
- 59. More code = More tests
- 60. On inputs, it’s easy to go from required to optional, not the other way around
- 61. Strict to Loose gives you more control
- 62. Loose to Strict is a breaking change
- 63. Breaks First!
- 64. Requirement: “Ability to fetch comments for a post”
- 65. The friendly API developer: “Let’s just give them an array of comments, we don’t need pagination!”
- 66. The friendly API developer: “Let’s just give them an array of comments, we don’t need pagination yet!”
- 68. The friendly API developer: “We won’t have more than 5 comments per post!”
- 69. 6 months after: “There’s posts with 100’s of comments”
- 72. Work is done, let’s go home!
- 73. comments(postId: 1, first: 1000)
- 74. The Friendly API Developer didn’t add limits to pagination
- 75. Add pagination limitations from the beginning
- 76. Error Message: Requesting 1000 records exceeds the `first` limit of 100 records.
- 77. Super hard to add retroactively, but super easy to adjust in the future.
- 78. How to avoid these overheads?
- 79. Adopt Schema First Design
- 80. Talk to your clients
- 81. Be responsive to your clients requests
- 83. Platform APIs
- 84. Constraints due to release cycles
- 85. Conclusion
- 86. Redundant work slows down progress on important features
- 87. “The road to hell is paved with good intentions”
- 88. Thank you! 👋 Twitter: @DamirSvrtan
Editor's Notes
- #2: I’m super glad I could present to you all today! I’m sad that we’re not all together at a nice location where we could all meet up! But at the same time, it’s great that we could all join this conference from across the globe! Let me introduce myself.
- #4: I am based in San Francisco, and where I moved about 3 years ago. Prior to moving to San Francisco
- #5: I use to live in Zagreb, the capital city of Croatia, a small country in south eastern europe. That’s where I grew up, spent the first 28 years of my life.
- #6: Why is this topic relevant to me? I’ve been building APIs for the last 7 years, and I’ve seen a pattern where developers like to expose more data than is actually needed.
- #7: What kind of overhead?
- #8: Queries no one asked for End points no one is using Unused Functionality, extra fields, relationships That you have to maintain
- #10: Through this talk, we are going to be building a …
- #11: similar to Medium or dev.to
- #12: Fairly simple blogging platform
- #13: how not to over expose data on your APIs Or how to avoid building extra flexibility that no one asked for
- #14: Generally, I see a pattern where developers try to be speculative about what’s going to be needed in the future, so they often over build their APIs
- #15: I’m going to break this into 3 patterns of a bloated surface that I usually see.
- #20: Why wouldn’t we expose the email field right away? We’re gonna spend more time exposing it if we need it afterwards, right? So we’re saving time for the business, right?
- #22: Not making things up GDPR California privacy laws
- #23: Marking the field as deprecated - if your API schema descriptiors allows that Or if a versioned API, new API version
- #24: Emails, reaching out to clients Platform - really hard, give a few months of lead time Private API - smaller amount of clients, but a lot of communication nevertheless
- #25: Sparse field sets No sparse field sets - no idea if you’re breaking clients Sparse field sets - you better have metrics / observability
- #26: Multiple deploys It’s not just easy as removing a few lines of code
- #28: Group of reviewers Green check mark next to the post title
- #33: Sparsefield sets, clients can pick and choose fields they want to use Every time you load the post, you load the reviewers
- #35: A lot of communication, emails, slack messages..
- #36: … One thing we are not utilizing
- #38: I’ve been through this many times, where I was removing stuff that no one asked for. And I guarantee it’s way easier to add in the future than it is to remove - sometimes additions to the API can be done within an hour, while removals take a lot of time. So that’s about exposing fields on queryable resources
- #39: What kind of input fields? The payload your API accepts when mutating or changing data; in REST APIs that would be the data your API gets whenever a client initiates a POST or PUT request. Let’s look at an example
- #43: All that you need is an id and a body
- #46: Sure, the Comment does have a Post ID, but it doesn’t mean we are able to update it? No! These don’t have to match, you are supposed to build your apps acorrding to your business needs, and that business logic should be on the server, not on the client.
- #48: they might get confused What happens if I send the same value? What happens if I send a different value? And then lastly…
- #52: ready for extra flexibility in the future, but not extremely flexible now Scope out what is needed for your clients
- #55: Let’s just enable them to not send that post ID.
- #56: …What did we get with this?
- #57: We need to handle both cases, when there is a post ID, and when there’s not
- #59: Anytime you change some commenting logic, you need to reflect that in both scenarios Caching - two strategies?
- #60: Gotta test both cases
- #62: Gives the client less flexibility, but it’s not the flexibility they’ve ever asked for Gives you more flexibility in terms in changing
- #64: Second part of the second principle Defensive Programming Ensure that your API is not going to get abused Let’s go with the same example
- #66: Or a better version of this..
- #72: breaking change Introducing a new query / endpoint
- #73: Not so soon…
- #74: The client was too lazy to actually do paginated requests… What happened?
- #94: Figure out what is your domain and for whom your building your API - don’t play guessing games Don’t couple your APIs to specific implementations. Maybe today you have a relational database that is backing some of this data, tomorrow it might be some API that won’t have the same fields, and you might bring yourself more trouble than anything.
- #95: Involve them early in the process - figure out what their needs are. Don’t try to be speculative.
- #96: As you are building out a minimal schema that is not over bloated, you have to be responsive to feature requests
- #98: There might be a little bit of speculation here if you’re building a platform and trying to get ahead of competitors and try to drive user adoption But even then, I would question how much this kind of speculation is worth.
- #99: I have a coworker that worked on APIs that were deployed on premises for their clients. They would release these once every 6 months and they would have to do thousands of deployments every time they released. If they missed a field that a client might need, they would have to repeat the same redeployments, which would be very costly. This is very unique and you probably are not in this situation
- #101: You might be able to spend your time more wisely by improving API documentation, improving slow queries etc. Exposing too much data can be highly unproductive. You know, there’s a saying…
- #102: Even though we might think we are doing something good for our API clients, and being a good partner, that often is not the case.
- #103: If you want to talk more about this topic, let’s talk afterwards or reach out to me on Twitter!