Open source intelligence analysis
- 1. Open Source Intelligence Analysis Petr Špiřík “True genius resides in the capacity for evaluation of uncertain, hazardous, and conflicting information.” Winston Churchill
- 2. About The course „You never know.“ „Truth is in the middle.“ „Can’t trust anything THEY say.“ „Don’t even try to understand.” O RLY? Petr Špiřík Enterprise security, incident response, security architecture and design. This is what I do. Cyber security, privacy, counter- surveillance and threat intelligence. This is what I like. Education and the power of knowledge. This is what I believe. CC-BY-SA • Petr Špiřík
- 3. Method in the Madness Open Source Public domain Internet centric Unstructured Unreliable Overwhelming Intelligence Analysis Define problem Collect data Analyze information Report conclusion Check with reality CC-BY-SA • Petr Špiřík
- 4. Problem definition “If one does not know to which port one is sailing, no wind is favorable.” Seneca CC-BY-SA • Petr Špiřík
- 5. Time Flow Prediction Forward looking Limited assurance Consistency is key factor Am I the target of surveillance? Explanation What lead to current situation? Which of these stories is true? Opinions and behavior forming How do vaccines cause autism? CC-BY-SA • Petr Špiřík
- 6. Tangible problem Right questions True or false Selection from menu Realistic Expected results Ability to decide and act Gather evidence Debunk lie CC-BY-SA • Petr Špiřík
- 7. Examples Good What is the root cause of Ukraine crisis? Should higher education be free of charge? Are government owned media biased? Bad Learn something about Ukraine and stuff. Kids are unhappy at our schools, this must change! Is this whole world just an illusion? CC-BY-SA • Petr Špiřík
- 8. Collection “Facts do not cease to exist because they are ignored.” Aldous Huxley CC-BY-SA • Petr Špiřík
- 9. Pick one Data Driven Holistic, mosaic, immersive Information channels required Establishes model Hypothesis Driven Problem focused Hypothesis generation required Solves one problem only CC-BY-SA • Petr Špiřík
- 10. Data Driven Sources Validation of sources credibility, accuracy, speed Source management review, update, remove Typology academic, research, news Channels Real time RSS, Twitter & TweetDeck Regular Google queries weekly, monthly Knowledge management system notepad, wiki, Evernote CC-BY-SA • Petr Špiřík
- 11. Hypothesis Driven Google (Hacking) Google operators provide powerful tool -site:bbc.co.uk (Germany OR France) AND (Russia OR Putin OR “Russian Federation”) filetype:pdf Investigation with Maltego Open source intelligence, investigation and forensic tool Community edition free of charge Requires focus and dedication Starting point and goal are absolute must CC-BY-SA • Petr Špiřík
- 12. Evidence Evaluation Weight Relative Can change based on subject of analysis 0% - not relevant 100% - critical evidence Credibility More stable Function of source selection and management 0% - aeronet.cz 100% - your mother CC-BY-SA • Petr Špiřík
- 13. How much Information you Really need? Incomplete information We make incomplete information decisions all the time We will never have complete information Consistency beats superstar intuitive guesses in the long run Beware of indecision paralysis Information Overload You can always look for more information There is critical mass of information that is “enough” Additional information provided beyond this point do not change the result significantly CC-BY-SA • Petr Špiřík
- 14. Analysis “War is 90% information.” Napoleon Bonaparte CC-BY-SA • Petr Špiřík
- 15. Mind Memory Human mind is prone to errors Tool is not important – the process is Think about thinking – some errors can’t be avoided but can be compensated Record everything Separation Do one step at a time Do not mix idea generation with analysis Do not make final judgment after first hypothesis evaluation, disregarding how strong it looks Record everything CC-BY-SA • Petr Špiřík
- 16. Situational vs. Theory Driven Analysis Situational Focus on specific situation Location, culture, company Understand the environment Seek for issues present in given context Judgment prioritizes the situation assessment and include issues identified Theory Driven Focus on issue investigated Abuse, espionage, conflict Understand the issue Seek for shared symptoms of the issues in given context Judgment prioritizes the issues and assess how these are affecting the situation CC-BY-SA • Petr Špiřík
- 17. Problem deconstruction Method Useful for decision making Define factors first Assign them weight (up to 100%) Define options Quantify options (up to 100%) Calculate the result Sample Matrix CC-BY-SA • Petr Špiřík Should I go to Erasmus? Weight Erasmus Czech Republic Cost 40 20 (8) 80 (32) Timing 10 70 (7) 30 (3) Experience gained 50 60 (30) 40 (20) Total 100 45 55
- 18. Competing Hypothesis I Hypothesis Generation Brainstorming and recording No evaluation Clear definition required Identify key differences Remove redundant or unclear hypothesis Evidence Gathering Gather and validate evidence Check evidence to each option Strong/weak Supporting/Disproving Remove irrelevant evidence CC-BY-SA • Petr Špiřík
- 19. Competing Hypothesis II Review and conclusion Identify promising hypothesis Look for invalidation Review evidence weight and credibility Review hypothesis Make tentative judgments Identify game changing evidence Example War in Europe? Yes, please. Cold war only. Everything is good Crimean crisis + ++ -- Greece vs. EU talks -- + - ISIS expansion -- - - CC-BY-SA • Petr Špiřík
- 20. Biases I Evaluation of Evidence Vividness Absence of data Thrive for consistency Unassessed evidence Confirmation bias Cause and Effect Favoring casual explanation Favoring central scheme Cause and effect Internal vs. External drivers Overestimating our importance Mirror image/Projection CC-BY-SA • Petr Špiřík
- 21. Biases II Probabilities estimates Availability rule Anchoring Verbal expressions Complex scenarios Base rate fallacy Hindsight biases “Everyone knew how this was destined to end. I am surprised you did not see it coming.” Not problem of analysis itself Problem of target audience Can be discouraging CC-BY-SA • Petr Špiřík
- 22. Reporting “If you can’t explain it simply, you don’t understand it well enough.” Albert Einstein CC-BY-SA • Petr Špiřík
- 23. Audience Formal Professional assignment Academic research Reporting up Focus on form Credibility is at stake Informal Your own use Circle of friends Informing down Don’t overdo it Shoot early, update often CC-BY-SA • Petr Špiřík
- 24. Structure Top Down Approach “Let the train crash. People want to see the train crash.” Lead with key judgment first Do not start with data Make a statement, do not ask questions. Length One sentence for key message One paragraph for executive summary One page for overview report Anything above one page – nice, but no one is going to read it. CC-BY-SA • Petr Špiřík
- 25. Content Be clear Report is finished product State the result Provide estimates Offer alternative conclusion Be consistent Create templates and use them Align with problem statement Keep the estimates consistent Highlight game changing factors CC-BY-SA • Petr Špiřík
- 26. Reality Check If you know the enemy and know yourself you need not fear the results of a hundred battles.” Sun Tzu CC-BY-SA • Petr Špiřík
- 27. Close the loop Look forward Note breaking points in advance Prepare the paths Follow up if triggered Update your system Did any evidence source changed its reliability? What was the feedback on the report? What tasks were waste of time? Learn, adapt, improve. CC-BY-SA • Petr Špiřík
- 28. Tips & Tricks DO Trust in your analysis Aim for constant improvement Train. Intelligence analysis is a skill Make this count Do not Become overconfident Expect to read the future Lose focus on problem Raise unrealistic expectations CC-BY-SA • Petr Špiřík
- 29. Key Judgments “Hope is not a strategy. Fear is not an option. Luck is not a factor.” James Cameron CC-BY-SA • Petr Špiřík
- 30. Thanks! Petr Špiřík petr.spirik@gmail.com www.slideshare.net/zapp0/ Resources Richards J. Heuer: Psychology of Intelligence Analysis Michael Bazzell: Open Source Intelligence Techniques Daniel Kahneman: Thinking, Fast and Slow CC-BY-SA • Petr Špiřík