FAPI / Open Banking Test Suite #fapisum - Japan/UK Open Banking and APIs Summit 2018 - July 25, 2018
2 likes294 views
This document describes a test suite for OAuth and OpenID Connect protocols. It discusses the need for conformance testing to ensure interoperability between authorization servers, clients, and protected resources. The test suite is designed to test the protocols in a multi-party manner using a structured configuration, logging, and modular execution units. It aims to test both normal and error cases to avoid a false sense of security from only testing happy paths. The architecture involves conditions, modules, plans, environments, and logging to API endpoints. The goal is to make the testing fully scriptable and transparent.
FAPI / Open Banking Test Suite #fapisum - Japan/UK Open Banking and APIs Summit 2018 - July 25, 2018
- 1. @justin__richer FAPI/OB Test Suite Justin Richer July 2018 1
- 2. @justin__richer Who am I? • Independent consultant in Boston, USA • Direct contributor to OAuth2 and OIDC • Editor of OAuth RFCs 7591, 7592, and 7662 • Software architect for Authlete and Fintechlabs • Author of OAuth2 In Action 2
- 7. @justin__richer Testing these protocols is tricky 7
- 9. @justin__richer Resource Owner Authorization Server Protected Resource Client Resource owner’s credentials Client’s credentials Authorization code Access token How do we fit the test harness in here? 9
- 10. @justin__richer End User Session at the Relying Party Identity Provider Identity Profi le APIRelying Party (Application) End User’s Credentials, Authorization of the Relying Party ID Token and Access Token Access Token and User Information 10
- 11. @justin__richer Design goals • Multi-party protocol testing • Structured configuration • Structured logging and results • Deterministic, modular execution units • Protect sensitive configuration and results data • Transparent process 11
- 12. @justin__richer We need to handle special cases • Front-channel requests that may never return • How things react to intentionally bad requests – Testing only the happy path leads to a false sense of security 12
- 13. @justin__richer What we test • UK Open Banking • FAPI • HEART • AS, Client, and RS 13
- 25. @justin__richer Everything through an API • Create, start, stop tests • Retrieve test logs • Retrieve test plan information • Fully scriptable 25
- 28. @justin__richer Condition • Simple • Reusable • Deterministic • Not built on existing OAuth/OIDC libraries – Easily isolate functionality – Better for testing for negative behaviors 28
- 29. @justin__richer Module • String a set of conditions together in order • Manage the state between condition calls • Determine how condition results map to test results – E.g., optional conditions can fail in some circumstances 29
- 30. @justin__richer Plan • Allows you to run several related modules with the same configuration • Tracks history of module run results 30
- 31. @justin__richer Environment • Holds the full current state of a test run • Modules and conditions can read and write to it • Entirely in JSON 31
- 32. @justin__richer Configuration • Anything the test module needs to run – Server locations – Secrets and keys – Certificates • Can’t be changed once test starts • Changes for different tests • Entirely in JSON 32
- 33. @justin__richer Event log • Records results as tests run • Made of many individual entries – Timestamp, source, data • Stored in MongoDB • Entirely in JSON 33
- 34. @justin__richer Image upload • Stored in the event log (as JSON) • Capture what happens in the user’s browser – Error pages – User interaction 34
- 35. @justin__richer Protecting data • Use OpenID Connect for all logins • All test instances have an “owner” • All log entries have an “owner” 35
- 36. @justin__richer Open source • Publicly available on GitLab • Code can be fully audited (no black boxes) • Enhancements from several groups to date • Contributions are welcome! 36