Open Source Statistical Software for the Statistical Office
1 like365 views
This document discusses Statistics Netherlands' use of open source software. It notes that open source software is driving modern statistics and data science. It describes how Statistics Netherlands uses communities like GitHub and Stack Overflow. It discusses motivations for using open source software, including economic benefits and supporting the community. It highlights Statistics Netherlands' curated list of over 50 open source software packages covering different areas. It provides examples of R packages and Node.js packages contributed by Statistics Netherlands. Finally, it offers tips for contributing to open source software projects.
![Mark van der Loo | Statistics Netherlands
Motivations
Wikipedians
Wikipedians enjoy a sense of accomplishment, collectivism, and
benevolence while working with exceptional freedom and ease. The
values of reputation, community, reciprocity, altruism and autonomy
are fostered by both the people and the technology[. . .].
(Kutzetsnov, 2006)
Commecial parties
[Companies] expect to benefit from their expertise in some segment
whose demand is boosted by the success of a complementary open
source program. (Learner, 2000)](https://image.slidesharecdn.com/sts043s3vanderloo-170717100244/85/Open-Source-Statistical-Software-for-the-Statistical-Office-8-320.jpg)
Open Source Statistical Software for the Statistical Office
- 1. Mark van der Loo | Statistics Netherlands Open source statistical software at the statistical office Mark van der Loo Statistics Netherlands m.vanderloo@cbs.nl
- 2. Mark van der Loo | Statistics Netherlands What is open source software? Free and Open Source Software Use Study Change Redistribute
- 3. Mark van der Loo | Statistics Netherlands FOSS is driving modern stats and data science And much, much more. . .
- 4. Mark van der Loo | Statistics Netherlands Communities (1) Social Coding with github
- 5. Mark van der Loo | Statistics Netherlands Communities (2) Q&A with stackoverflow
- 6. Mark van der Loo | Statistics Netherlands Communities (3) news & discussions on Twitter
- 7. Mark van der Loo | Statistics Netherlands Role of commercial parties, foundations And many,many more, . . .
- 8. Mark van der Loo | Statistics Netherlands Motivations Wikipedians Wikipedians enjoy a sense of accomplishment, collectivism, and benevolence while working with exceptional freedom and ease. The values of reputation, community, reciprocity, altruism and autonomy are fostered by both the people and the technology[. . .]. (Kutzetsnov, 2006) Commecial parties [Companies] expect to benefit from their expertise in some segment whose demand is boosted by the success of a complementary open source program. (Learner, 2000)
- 9. Mark van der Loo | Statistics Netherlands Motivations for Official Statistics Use Economic (its free!) New hires Supporting community Contribute Solving shared problems Many eyes make all bugs shallow Influence, reputation Built with tax payer’s money
- 10. Mark van der Loo | Statistics Netherlands FOSS for official statistics Awesome list Community effort Curated 50+ Software packages Covering 14 GSBPM areas Growing − 75 commits − 5 PR’s − 6 contributors www.awesomeofficialstatistics.org
- 11. Mark van der Loo | Statistics Netherlands When is it awesome? You may be awesome when. . . Free, open source, available for download Used in at least one statistical institute for production or, offers access to official statistics Relatively easy to install and use (for non-dev’s) Actively maintained At least one stable release
- 12. Mark van der Loo | Statistics Netherlands What’s on the Awesome List?
- 13. Mark van der Loo | Statistics Netherlands In the works. . .
- 14. Mark van der Loo | Statistics Netherlands FOSS policy at Statistics Netherlands (in short) Usage Selection and introduction follows the same procedure as for COTS (commercial off-the-shelve). R, Python, node.js Contributing When relevant to Statistics Netherlands, with positive business case. R, node.js
- 15. Mark van der Loo | Statistics Netherlands Deployment of R and Python at Statistics Netherlands Central read-only folder for executables. All users have access to the same version with curated list of libraries installed. Scripts can be prepared and integrated for non-developers with ease. Repositories (CRAN, Anaconda) on internal website, updated frequently. Old versions stay available for some time so existing applications stay working.
- 16. Mark van der Loo | Statistics Netherlands Example using R in data editing: tools and roles Database Rules, metadata, control parameters... Graphical user interface IT dpt Users IT dpt Methodology
- 17. Mark van der Loo | Statistics Netherlands Example contributions R packages − data editing: validate, simputation, errorlocate, deductive, dcmodify − data logging: lumberjack − small area estimation: hbsae − datavis: tabplot, tabplotd3, tmap, . . . node.js packages − scraping: RobotTool, S4Robo − dashboard: StatMine . . .
- 18. Mark van der Loo | Statistics Netherlands So you want to contribute? Here are some options 1. Use it (& send a thumbs-up!) 2. Advocate − Tell your friends and colleagues − Write blog posts / articles / presentations − Social media (twitter. . .) 3. File bug reports, suggestions 4. Add code to an existing project 5. Start your own project
- 19. Mark van der Loo | Statistics Netherlands FREE TIP! Don’t work alone Join your local community − meetups, news letters, hackatons Set up a community in your institute − Local wiki, user meetings, hackatons, ask-the-expert