Going To A Data Hack - Govhack 2015

Editor's Notes

• #2: https://www.govhack.org/ If you can’t read the fine print then you’re missing out on a movie reference that will most likely be lost on those under a certain age.
• #3: A Hackathon is an event where you gather a heap of resources and people, form small teams and try to deliver as fully realised solution to a set theme or problem in a short intense amount of time. Normally a hackathon is focused on delivering working software, but in the case of a data hackathon you work from a heap of datasets and try to deliver something of value, that can be working software, but often is something else. For this reason non coders can participate in a data hack easily. Another difference is a hackathon normally revolves around creating some sort of business (be that profit or non-profit) idea and validating it. Data hackathons are about understanding and realising value from data, and that value can often just be delivering better access to the information the data represents.
• #4: A push is being made to make govt data open to the public which has resulted in lots of datasets now being available But to date there has low awareness of these datasets and not much value that has been generated from these resources GovHack was formed to: Help the departments clean up their datasets and identify issues that were stopping people using them more frequently Increase public awareness of these resources Try to kick-start some for profit and non-profit ventures around these resources so the govt could start to see some return from this initiative https://www.data.gov.au/ is the home of the overall open data initiative and there are state based sites such as QLD’s https://data.qld.gov.au/ Open Knowledge Australia is the Australian arm of the global Open Knowledge network which is a non-profit that supports the open data movement all over the world. They are who runs GovHack. http://au.okfn.org/
• #5: Only a subsection of the datasets available are selected for GovHack each year. That said there was well over a hundred for GovHack 2015 (https://www.govhack.org/2015-data/). There’s huge range of data in both source and composition. Data from all levels of govt bodies Wide range of types of datasets. Some examples are: sensor readings, satellite imagery, questionnaires, letters, images, video, financial records, GPS co-ordinates, etc. Data available many different ways: APIs, web portals, simple flat files, queries
• #6: Lots of prizes are on offer. Can win more than one prize for each entry. Best Digital Transformation Hack: Open category for how government can be brought into the 21st Century via digital services. The intellectual property data bounty: Develop an easy way for non-experts to access and use the IP Government Open Data on data.gov.au to find out where, who and what IP exists in Australia.
• #7: Find the winners from 2015 here: https://www.govhack.org/2015-winners/ Remembrance: A website that retells the ANZAC story, both at the front and at home. https://hackerspace.govhack.org/content/remembrance AusTrails.org: Aggregator of trail data from community that’s run on OpenStreetMap. https://hackerspace.govhack.org/content/austrailsorg Health Buddy: Helps you find health services matched to your needs and plan trips. Govt departments can use it to see usage stats. https://hackerspace.govhack.org/content/health-buddy   Question Time: A game of policy:  2 player game where you try to guess where an MP voting record lies in relation to a portfolio area. https://hackerspace.govhack.org/content/question-time-game-policy Neuron: Mind-map of inter-connected IP data to aid discovery. Links to a purpose built IP social network. https://hackerspace.govhack.org/content/neuron-connecting-minds Gender Equality: Data visualisation of gender equality data from the  Workplace Gender Equality Act dataset using D3. https://hackerspace.govhack.org/node/542 Synergising Synergies for Sitizens: Tool in Tableau Public to work out which suburbs are best-placed for rooftop solar investment. https://hackerspace.govhack.org/content/synergising-synergies-sitizens
• #8: Do: Get familiar with the datasets and thinking about what tools you’ll use Don’t: Start building something to take in with you. You are painting yourself into a corner to start with. Before the event you should also make triple sure that whatever you are taking in as goes tech is working well and has all the updates you need. A couple of our team spent far too long getting their laptops working due to these very reasons. Whilst you shouldn’t start building something before you go in, at least have a conversation about common tools. You should get setup up with at least a common messaging service and project management tool. We used Skype and Trello, but again wasted some time getting these setup at the event. Just because you might not have decided yet what you are going to build doesn’t mean you don’t have at least a shortlist of development tools you might use. It’s a good idea to install all of these and get them working before you head in. Much easier to just not use something than it is to install and configure something for the first time at the event.
• #9: The UQ engineering block is a cathedral of wood, steel and glass. The UQ wifi is by contrast a poorly built mud hut. We struggled a lot with it over the course of the event. I ended up only being able to get it working on my phone, which I then had to tether to my laptop. Consider backup internet options you could use before you go in and try to limit how much you will rely upon the internet at the venue. Our main criteria was getting working code, and this was a mistake. This is more traditional hackathon thinking and didn’t fit well with the idea behind Govhack. I’ll talk more about this later on in the presentation.
• #10: Our best web developer was struck down by sickness on Friday night. We had originally started selecting ideas based on her considerable skills. In hindsight we didn’t readjust enough to take into account losing her from the team. We ended up with two ideas: an estimation game (based on the board game Wits & Wagers). This is an idea we’d had before the event A map based tool to help you assess the safety of a neighbourhood. This is an idea we come up with on Friday night. To decide we quickly sketched out a dev plan for each to see which was most achievable. We looked at: time we had Skill in the team what the datasets we would use were like and what prizes we’d be eligible for In the end the estimation game won out and we went to work. Node.js backend crashed and burned because we’d selected it based on the skills of the original team. Developer working on it was toast so we sent him home. Rest of the team regrouped to think about what we could salvage from the wreckage. After some research we came across Google Fusion Tables. After an hour of hacking about our remaining dev raised his hands an announced it was working. We danced a bit then called it a day as it was really late and we were really tired.
• #11: One of the very best moments at Govhack was when our dejected dev, who’d been working on the node.js backend, came through the door and we surprised him with a working website. Even though it wasn’t his fault he had felt like he’d let the team down, so he was extra happy to see we were back in the game. The whole team had been given a shot of energy from this brush with disaster and it carried us through the rest of the day. We discarded many of the game elements we’d originally planned and focused instead on delivering value from the data. In hindsight this should’ve always been the focus as it was true to the data we had. We couldn’t get the player responses to write back our Fusion Table database, even though that functionality does exist, so we used fake responses data. In hindsight it was way too optimistic to think we were going to have time to get 100’s of players through the game before submission, and not having response data earlier held up designing the results screen. Even though it was fake data we still needed to tweak it to make believable and have it produce interesting results. Even this took a little more time than we had anticipated. We started the video late as we’d been told by the organisers not to focus on it. In reality you should start planning it Friday. It’s one of your best tools to explain what you’ve built and the better you can make it the less you have to have working code.
• #12: How to play Wits & Wagers: https://www.youtube.com/watch?v=6FbGRahrAhI Our entry Hackerspace entry: http://2015.hackerspace.govhack.org/content/consensus Website (which is now broken): http://www.consensusquiz.com/ Video: https://www.youtube.com/watch?v=Q994vWVLNRM
• #13: Google Fusion tables: https://sites.google.com/site/fusiontablestalks/stories Data Driven Documents (D3.js): https://d3js.org/ D3.js is a really flexible and easy to use data visualisation tool for the web. It’s great at one thing: bespoke data vizs. It can’t be relied upon for dashboard like functionality though. Google Fusion Tables are even easier to setup and use, and many people use them to host the data from their web vizs for this reason. Now that Google have expanded their data offerings you have a wide range of free, robust, and easy to use options: Google big data tools: https://cloud.google.com/products/big-data/ Google cloud data tools: https://cloud.google.com/products/storage/ There’s also a heap of other tools that you can use to get a prototype working fast. A good place to start is the resources at the The GovHack Developer Toolkit: http://govhack-toolkit.readthedocs.io/
• #14: We had originally wanted to make a great game that used datasets in an interesting way. The engagement with those datasets, we thought, would come as a result of people enjoying the game. We had this the wrong way around. We got a much better result when we focused on how to engage people with the datasets in a way where they could understand them better, and we added game elements as a way to facilitate this process. We created another layer of data (in this case player estimations) that could help explain more fully an existing statistic. Consensus helps players see how well their own perceptions of areas of society line up with the reality. They also get to see how their perceptions relate to other players. For govt this is at tool they can use to see how the general public perceive areas of society.
• #15: Slider: We used a slider for these reasons: You don’t have to come up with answers, so you can add questions much faster You get a much more accurate measure of what people’s understanding of a statistic is. It’s not either they get the right answer or not, you get to peek inside their perceptions of a statistic to see what they think about a subject A slider feels much closer to the feeling of estimating than selecting a value or typing one in. We still have a number come up so it’s easy to see what you have selected. We debated about the slider start and end and the starting position of the selector on the range. All these things can bias a players answer; especially if they are particularly unsure of the answer. In the end we had two types of ranges: whole numbers and 0-100%. The whole number range goes super high, and if we’d had time we wanted instead to hide everything but the selector and have values come up down the bottom. This way we could have different ranges, but they would’ve been hidden from the player so as to not bias them.
• #16: Your score: Originally this was to be the most important part of the game. In the revised version it plays a much smaller role. Your score relates to how many people you were closer to the answer than.
• #17: “How close your answer was” bar: In the results screen we double up on the information about how close you were (the same info is in the histogram at the bottom), but we do so because how close you were is really important to visualise for the player so they start to get an idea of how their knowledge relates to the actual statistic. It’s a really simple visualisation with a bar which is the same width as the histogram at the bottom, but it strips away all the other data and just gives the distance to the player so they can clearly see it. To reinforce this we also added a label to qualify how close they got to the answer.
• #18: Question scrolling text: As we were running out of screen real estate, and some of the questions were quite long, we put the question into a scrolling text box. It’s a hacky solution, but it gives a “breaking news” feel to the results screen.
• #19: The histogram: We have already shown the player how close they were to the correct answer and we show that again in the histogram but now we add in all the other player answers to the mix. Here we want to the player to focus on where their answer falls in relation to all the other answers. We help them by highlighting what range their answer falls within either side of the correct answer and as a result we also highlight all those that fell outside that range. We tell the player how many players they did better than. We added a random descriptive stat to give another insight into the histogram they are looking at. We were impressed that many of the people we got to try the game after the event, even though we told them it was faked response data beforehand, ended up thinking it was real responses when they got to the results screen. That really validated for us the design decisions we had taken. If we’d have more time we would’ve liked to have had the option for players to see how their answers faired across their region, state, and nationally. We’d also have liked to have added separate categories for players to estimate questions in.
• #20: Here’s Dale and I receiving the local and state awards. Here all the winners: https://www.govhack.org/2015-winners/
• #21: I got a much better understanding of what defining a problem is and how to identify and eliminate scope. I gained a much better appreciation of how far you can get “faking it” before you have to actually get something working. How Paypal and Reddit faked their way to traction https://medium.com/platform-thinking/how-paypal-and-reddit-faked-their-way-to-traction-9411fb583205#.y8yyoli0d
• #22: You’re there to realise value from the datasets the govt has. They are valuable data. If you understand the value intrinsic to the datasets you then can much more easily match this to a data product which is valuable to someone else. Also what a dataset is like heavily influences what you need to know to be able work with it. Ruling out datasets is more important than selecting ones you’d like to work with. There is no chance you’ll not have at least a couple of dataset you can work with.
• #23: Don’t be fooled that you won’t be able to think of anything. You’ll have more than enough ideas that will be achievable. Better you spend the time finding the solution that delivers meaning value, then working out a way to quickly prototype it than searching through 100’s of datasets trying to find something that fits into what you have already built or decided to build. The real fun is finding a way to hack together a great solution and that’s where you use your creativity.
• #24: You’re not deploying production code so just decide on tools everyone can contribute with and get setup before the event. Too much time is wasted trying to pick the “perfect” tool to use that can be used coming up with a kick-ass solution. If you don’t have to have it working perfectly that opens up heaps of options for you to hack together something to show the functionality via the video. The video helps you be more creative and innovative and attempt more ambitious solutions. https://www.youtube.com/watch?v=Q994vWVLNRM