![Example request & response
Request Response
{
"meta": {
"beaconId":"org.example.beacon.v2",
"apiVersion": "v2.0",
"returnedSchemas": [ { "entityType": "EntryTypeA", "schema": “entry-typeA-schema-v2.0”} ],
"receivedRequestSummary": {
"apiVersion": "v2.0",
"filters": …,
"requestParameters": …,
"includeResultsetResponses": "HIT",
"requestedSchemas": [ { "entityType": "EntryTypeA", "schema": “entry-typeA-schema-v2.0”} ],
"pagination": { "skip": 0, “limit":10 },
“requestedGranularity”: “count”
}
},
"responseSummary":{
"exists": true,
"numTotalResults": 25355
},
"info": {
"someInterestingStuff": "this is the interesting stuff"
},
“beaconHandovers":[
{
"handoverType": { "id":"EFO:0004157", "label":"BAM format" },
"url":"https://api.mygenomeservice.org/Handover/9dcc48d7-fc88-11e8-9110-b0c592dbf8c0",
"note":"This handover link provides access to a summarized VCF."
}
]
{
"meta": {
"apiVersion": "v2.0",
"requestedSchemas": [{
"entityType": "EntryTypeA",
"schema": “entry-typeA-schema-v2.0”
}]
},
"query": {
"requestParameters": {
"datasets": { "datasetIds": ["DatasetXYZ", "Dataset123"] }
},
"filters": [
{
"id": "EFO:0001212",
"scope": "biosamples"
}
],
"includeResultsetResponses": "HIT",
"pagination": { "skip": 0, "limit": 10 },
"requestedGranularity": “count"
}
}](https://image.slidesharecdn.com/beaconimplementers-220223150328/85/Lighting-a-Beacon-training-for-future-implementers-9-320.jpg){kind=tracking}
![Example request
Request
{
"meta": {
"apiVersion": "v2.0",
"requestedSchemas": [{
"entityType": "EntryTypeA",
"schema": “entry-typeA-schema-v2.0”
}]
},
"query": {
"requestParameters": {
"datasets": { "datasetIds": ["DatasetXYZ", "Dataset123"] }
},
"filters": [
{
"id": "EFO:0001212",
"scope": "biosamples"
}
],
"includeResultsetResponses": "HIT",
"pagination": { "skip": 0, "limit": 10 },
"requestedGranularity": "count"
}
}](https://image.slidesharecdn.com/beaconimplementers-220223150328/85/Lighting-a-Beacon-training-for-future-implementers-10-320.jpg){kind=tracking}
![Example request & response
Request Response
{
"meta": {
"beaconId":"org.example.beacon.v2",
"apiVersion": "v2.0",
"returnedSchemas": [ { "entityType": "EntryTypeA", "schema": “entry-typeA-schema-v2.0”} ],
"returnedGranularity": "count",
"receivedRequestSummary": {
"apiVersion": "v2.0",
"filters": …,
"requestParameters": …,
"includeResultsetResponses": "HIT",
"requestedSchemas": [ { "entityType": "EntryTypeA", "schema": “entry-typeA-schema-v2.0”} ],
"pagination": { "skip": 0, “limit":10 },
“requestedGranularity”: “count”
}
},
"responseSummary":{
"exists": true,
"numTotalResults": 25355
},
"info": {
"someInterestingStuff": "this is the interesting stuff"
},
“beaconHandovers":[
{
"handoverType": { "id":"EFO:0004157", "label":"BAM format" },
"url":"https://api.mygenomeservice.org/Handover/9dcc48d7-fc88-11e8-9110-b0c592dbf8c0",
"note":"This handover link provides access to a summarized VCF."
}
]
}
{
"meta": {
"apiVersion": "v2.0",
"requestedSchemas": [{
"entityType": "EntryTypeA",
"schema": “entry-typeA-schema-v2.0”
}]
},
"query": {
"requestParameters": {
"datasets": { "datasetIds": ["DatasetXYZ", "Dataset123"] }
},
"filters": [
{
"id": "EFO:0001212",
"scope": "biosamples"
}
],
"includeResultsetResponses": "HIT",
"pagination": { "skip": 0, "limit": 10 },
"requestedGranularity": “count"
}
}](https://image.slidesharecdn.com/beaconimplementers-220223150328/85/Lighting-a-Beacon-training-for-future-implementers-11-320.jpg){kind=tracking}
![Endpoints: GET vs. POST
GET Request POST Request
{
"meta": {
"apiVersion": "2.0",
…
},
"query": {
"requestParameters": {
"datasets": { "datasetIds": ["DatasetXYZ", "Dataset123"] }
},
"filters": [
{
"id": "EFO:0001212",
"scope": "biosamples"
}
],
"pagination": { "skip": 0, "limit": 10 },
"requestedGranularity": "boolean"
}
}
?requestedSchema=beacon-individuals-v2.0&skip=0&limit=10
With query parameters: With body (application/json):](https://image.slidesharecdn.com/beaconimplementers-220223150328/85/Lighting-a-Beacon-training-for-future-implementers-18-320.jpg){kind=tracking}
Lighting a Beacon: training for (future) implementers
- 1. This project has received funding from the European Union’s Horizon 2020 research and Innovation programme under grant agreement No. 825775 This project has received funding from the Canadian Institute of Health Research under grant agreement #404896 Lighting a Beacon: Training for (future) implementers
- 2. Who is this talk for? Beacon implementors and software developers Beacon users
- 3. Contents • Beacon Framework • Beacon Model • How to start implementing a beacon from scratch PART 1 PART 2
- 4. What is Beacon? HTTP API DB Request (GET or POST + JSON) Response (JSON) Framework Model
- 5. Beacon Framework Framework Model • Specifies the structure of the wrapper of the results. • Allows the interoperability of beacons with different data, since they all understand the same protocol.
- 6. Beacon Framework: Types of beacon Framework Model Boolean Beacon Count Beacon ResultSets Beacon
- 7. Beacon Model Framework Model • Describes the entities that the beacon supports. • Describes the schemas of those entities.
- 8. Beacon Model: Entities Analyses Biosamples Cohorts Datasets Genomic Variations Individuals Runs Framework Model https://beacon-schema-2.readthedocs.io Beacon Model viewer:
- 9. Example request & response Request Response { "meta": { "beaconId":"org.example.beacon.v2", "apiVersion": "v2.0", "returnedSchemas": [ { "entityType": "EntryTypeA", "schema": “entry-typeA-schema-v2.0”} ], "receivedRequestSummary": { "apiVersion": "v2.0", "filters": …, "requestParameters": …, "includeResultsetResponses": "HIT", "requestedSchemas": [ { "entityType": "EntryTypeA", "schema": “entry-typeA-schema-v2.0”} ], "pagination": { "skip": 0, “limit":10 }, “requestedGranularity”: “count” } }, "responseSummary":{ "exists": true, "numTotalResults": 25355 }, "info": { "someInterestingStuff": "this is the interesting stuff" }, “beaconHandovers":[ { "handoverType": { "id":"EFO:0004157", "label":"BAM format" }, "url":"https://api.mygenomeservice.org/Handover/9dcc48d7-fc88-11e8-9110-b0c592dbf8c0", "note":"This handover link provides access to a summarized VCF." } ] { "meta": { "apiVersion": "v2.0", "requestedSchemas": [{ "entityType": "EntryTypeA", "schema": “entry-typeA-schema-v2.0” }] }, "query": { "requestParameters": { "datasets": { "datasetIds": ["DatasetXYZ", "Dataset123"] } }, "filters": [ { "id": "EFO:0001212", "scope": "biosamples" } ], "includeResultsetResponses": "HIT", "pagination": { "skip": 0, "limit": 10 }, "requestedGranularity": “count" } }
- 10. Example request Request { "meta": { "apiVersion": "v2.0", "requestedSchemas": [{ "entityType": "EntryTypeA", "schema": “entry-typeA-schema-v2.0” }] }, "query": { "requestParameters": { "datasets": { "datasetIds": ["DatasetXYZ", "Dataset123"] } }, "filters": [ { "id": "EFO:0001212", "scope": "biosamples" } ], "includeResultsetResponses": "HIT", "pagination": { "skip": 0, "limit": 10 }, "requestedGranularity": "count" } }
- 11. Example request & response Request Response { "meta": { "beaconId":"org.example.beacon.v2", "apiVersion": "v2.0", "returnedSchemas": [ { "entityType": "EntryTypeA", "schema": “entry-typeA-schema-v2.0”} ], "returnedGranularity": "count", "receivedRequestSummary": { "apiVersion": "v2.0", "filters": …, "requestParameters": …, "includeResultsetResponses": "HIT", "requestedSchemas": [ { "entityType": "EntryTypeA", "schema": “entry-typeA-schema-v2.0”} ], "pagination": { "skip": 0, “limit":10 }, “requestedGranularity”: “count” } }, "responseSummary":{ "exists": true, "numTotalResults": 25355 }, "info": { "someInterestingStuff": "this is the interesting stuff" }, “beaconHandovers":[ { "handoverType": { "id":"EFO:0004157", "label":"BAM format" }, "url":"https://api.mygenomeservice.org/Handover/9dcc48d7-fc88-11e8-9110-b0c592dbf8c0", "note":"This handover link provides access to a summarized VCF." } ] } { "meta": { "apiVersion": "v2.0", "requestedSchemas": [{ "entityType": "EntryTypeA", "schema": “entry-typeA-schema-v2.0” }] }, "query": { "requestParameters": { "datasets": { "datasetIds": ["DatasetXYZ", "Dataset123"] } }, "filters": [ { "id": "EFO:0001212", "scope": "biosamples" } ], "includeResultsetResponses": "HIT", "pagination": { "skip": 0, "limit": 10 }, "requestedGranularity": “count" } }
- 12. Beacon Verifier • Verify that all of your endpoints follow the framework • Optionally, you can verify that the returned results match the model of the beacon https://ga4gh-approval-service-registry-test.ega-archive.org
- 13. Questions
- 14. Break
- 15. How to implement a Beacon • Endpoints • Types of filters • Mapping the DB to the BFF • Database queries • Pagination • Responses
- 16. Endpoints ┌── common ├── configuration ├── endpoints.json ├── LICENSE ├── README.md ├── requests └── responses ┌── analyses │ ├── defaultSchema.json │ ├── endpoints.json │ └── filteringTerms.json ├── biosamples │ ├── defaultSchema.json │ ├── endpoints.json │ ├── examples │ └── filteringTerms.json ├── cohorts │ ├── defaultSchema.json │ ├── endpoints.json │ ├── examples │ └── filteringTerms.json └── datasets ├── defaultSchema.json ├── endpoints.json └── filteringTerms.json ┌── genomicVariations │ ├── defaultSchema.json │ ├── endpoints.json │ ├── examples │ ├── filteringTerms.json │ ├── requestParameters.json │ └── requestParametersComponents.json ├── individuals │ ├── defaultSchema.json │ ├── endpoints.json │ ├── examples │ └── filteringTerms.json └── runs ├── defaultSchema.json ├── endpoints.json └── filteringTerms.json beacon-framework-v2 beacon-v2-Models • Where can you find the endpoints
- 17. Endpoints Framework endpoints Model endpoints
- 18. Endpoints: GET vs. POST GET Request POST Request { "meta": { "apiVersion": "2.0", … }, "query": { "requestParameters": { "datasets": { "datasetIds": ["DatasetXYZ", "Dataset123"] } }, "filters": [ { "id": "EFO:0001212", "scope": "biosamples" } ], "pagination": { "skip": 0, "limit": 10 }, "requestedGranularity": "boolean" } } ?requestedSchema=beacon-individuals-v2.0&skip=0&limit=10 With query parameters: With body (application/json):
- 19. Endpoints: List your endpoints { "meta":{ "apiVersion": "v2.0.0", "beaconId": “org.organizationName.beaconId", "createDateTime": “2007-01-09", "returnedSchemas": …, "updateDateTime": "2021-11-16" }, "response":{ "endpointSets": { … "individual": { "endpoints": { "biosample": { "returnedEntryType": "biosample", "url": "https://examplebeacons.org/individuals/{id}/biosamples" }, "genomicVariation": { "returnedEntryType": "genomicVariation", "url": "https://examplebeacons.org/individuals/{id}/genomicVariations" } }, "entryType": "individual", "filteringTermsUrl": "https://examplebeacons.org/filteringTerms", "rootUrl": "https://examplebeacons.org/individuals", "singleEntryUrl": "https://examplebeacons.org/individuals/{id}" } } } }
- 20. Filters: Types of filters Ontology Filter
- 21. Filters: Types of filters Alphanumeric Filter
- 22. Filters: Types of filters Custom Filter
- 23. Filters: filters vs. requestParameters Should it be a filter or a request parameter? Filter Request parameter It requires its own schema It can be represented with an ontology term Anything else
- 24. Database mapping exercise • Mapping the data of your database to the correct entities of the model (and their properties) • Extending the model to include additional entities that you may provide and are not represented in the model
- 25. Database queries • The new version of the Reference Implementation uses MongoDB: • Each entity can be thought of as a collection • Allows for a flexible evolution of the schema
- 26. Pagination • We support two types of pagination: skip & limit pagination cursor-like pagination • We skip n elements and return the next m elements. • We add a hash to the response encoding the next m elements of the page
- 27. Error responses { "meta": …, "error": { "errorCode": 404, "errorMessage": "Entry not found" } } • Same meta as an OK response • Error information
- 28. Tools provided • Beacon-verifier • Reference Implementation • BFF utility Coming soon https://beacon-schema-2.readthedocs.io Beacon Model viewer: • Beacon Framework • Beacon Model
- 29. Summary • The Beacon Framework defines the structure of the response. • The Beacon Model defines the structure of the response data. • Implementing a beacon: • Start with a boolean beacon • List your available endpoints • Map the data from your DB to the models • Handle the different types of filters • Build a response following the framework
- 30. Funding This project has received funding from the Canadian Institute of Health Research under grant agreement #404896 This project has received funding from the European Union's Horizon 2020 Research and Innovation Programme under grant agreement #825775
- 31. Questions