An Open Source Platform for Social Science Research
1 like809 views
The 'Next Generation Social Sciences' program aims to enhance rigorous social science research through a fully automated platform called Dallinger, which facilitates crowdsourced experiments and validates data. Dallinger operates using directed graph models and integrates various technologies for real-time multiplayer experiments. Key contributors include Jazkarta and several team members who emphasize project management and responsive design in their development process.
An Open Source Platform for Social Science Research
- 1. An Open Source Platform for Social Science Research Sally Kleinfeldt Barcelona 2017
- 2. The Program
- 4. "The program aims to build and evaluate new methods and tools to advance rigorous, reproducible social science studies at scales necessary to develop and validate causal models of human social behaviors."
- 5. Benefits for • Public health • Economics • National security
- 6. Initial Focus Identify causal mechanisms of collective identity formation
- 7. Dallinger
- 9. Dallinger • Crowdsourced experiments • Abstracted into single function calls • Can be inserted into higher-order algorithms, for example to progressively refine experiment
- 10. Fully Automated • Recruits participants (Mechanical Turk) • Obtains informed consent • Arranges participants into a network • Runs experiement (Heroku) • Coordinates communication
- 11. Fully Automated • Records the data they produce • Pays participants • Recruits new batches of participants contingent on the structure of the experiment • Validates and manages the resulting data
- 12. How Does It Work? • Experiments are modeled as directed graphs • Experiments are like Plone add-ons being run by the Dallinger system
- 13. Games! • All teams coalesced on using a public goods game • Dallinger is the only team using a real-time multiplayer game: Grid Universe
- 14. Demo
- 17. Why Jazkarta? • Expertise in Python, Flask, PostgreSQL, SQLAlchemy,Amazon Mechanical Turk, boto, tox, pytest, Redis, Selenium, PhantomJS, JavaScript, HTML, CSS… • Expertise in project management • Expertise in a mature open source community - Plone!
- 18. Our Process • Discovery meeting fall 2017 • Developed user stories • Estimated using planning poker • Implementing in a series of iteration
- 19. Our Team • Alec Mitchell • David Glick • Matthew Wilkes • Carlos de la Guardia • Jesse Snyder
- 21. Lessons • Don’t over-engineer plugin architectures (like recruiters) • Support live editing as much as possible • Break backwards compatibility when needed • Remove references to old ways of doing things
- 22. Lessons • Ship lots of useful demos • Be diligent about code reviews • Make important approach decisions together • People involved in decisions should know user needs intimately (we miss Joel Burton)
- 23. Plone vs. Dallinger Tech Stack
- 24. Tech Stack • Web based, but Flask instead of Zope • PostgreSQL instead of ZODB • Real time websockets • Built in deployment command
- 26. Helping Dallinger Users • Documentation • Slack channel • Cookie cutter template • Extendable base templates • Javascript library • Commands for local debugging
- 28. Code Quality • Automated lint checks • Continuous integration with minimum code coverage requirement • Code review • Regression testing of an experiment (GU) against changes to core Dallinger
- 30. Fun Challenges • Scaling selenium-based bots • Getting access to track interactions with 3rd-party sites (Chrome extension) • Testing multiple participants in parallel without sharing cookies
- 31. Questions?