Abuser Stories: Thinking Like the Bad Guy to Reduce Software Vulnerabilities

Editor's Notes

• #3: Education Government Finance Healthcare What else?
• #7: At the end of November, the massive hotel chain Marriott announced that as many as 500 million travelers who made a reservation at a Starwood hotel since 2014 had their data compromised. The hack originated at Starwood's reservation system; Marriott acquired that hotel group in September 2016, but the intrusion went undetected until September 8 of this year. Marriott says it blocked attacker access by September 10, but it took until November 19 for the company to fully understand the scale of the breach. Reports have increasingly indicated state-sponsored Chinese hackers were behind the attack, though this attribution has not been officially confirmed. The stolen data would be an espionage bonanza for government hackers, though. About 170 million impacted Marriott customers only had their names and basic information like address or email address stolen, but about 327 million people lost much more. Marriott says that this larger group had different combinations of name, address, phone number, email address, date of birth, gender, trip and reservation information, passport number, and Starwood Preferred Guest account information stolen. The Marriott incident is one of the largest data breaches in history.
• #8: At the end of September, Facebook disclosed a data breach in which attackers gained access to 30 million accounts by stealing "user authorization tokens," essentially access badges that get generated after a user successfully logs in. Sites use authorization token schemes so users don't need to sign in multiple times as they move around a platform. In Facebook's case, the attackers coordinated exploitation of three different bugs in the social network's "View As" feature to grab user tokens, gain access to Facebook accounts, and exfiltrate a significant and diverse trove of user data. The vulnerabilities existed in Facebook's platform since July 2017, but the company only detected suspicious activity related to them on September 14 of this year. Eventually, Facebook discovered the flaws and the attack on September 25. Here's how to check whether your Facebook account data was compromised in the breach. The company is investigating with the FBI, and hasn't said who may have been behind the hack. The incident is Facebook's first known data breach—impressive given that the platform has existed for well over a decade. But between the company's increasingly dismal track record on third-party access limits and a recent incident in which a bug exposed 6.8 million users' photos to third-party developers, it's hard to feel like things are going as well as they could on the user privacy and data management front.
• #9: In March, a ransomware attack locked down the City of Atlanta's digital systems, destabilizing municipal operations. The recovery took months, not to mention millions of dollars. The notorious SamSam criminal hacking group targeted the city and asked for about $50,000-worth of bitcoin. The ransomware attack affected five of Atlanta's 13 government departments, and undermined services like the Atlanta Police Department's records system, infrastructure maintenance requests, and court networks. Atlanta residents also couldn't pay their water bills for days. At the end of November, the Department of Justice indicted two Iranian men for allegedly carrying out SamSam attacks.
• #10: In the lead up to the Pyeongchang Olympics, Russian hackers launched a number of related cyberattacks as retaliation for the country's doping ban from the games. Then, before the opening ceremony of the Olympics in February, they orchestrated a hack that crippled the event's IT infrastructure, knocking out Wi-Fi, the Olympics website, and network devices in the process. Hackers used a worm dubbed Olympic Destroyer to wreak havoc as event technicians raced to restore service. Then in June, the same hackers reemerged—this time in preliminary spear phishing attacks against labs that research biological and chemical threats in France, Germany, Switzerland, Russia, and Ukraine. Specifically, the targeted lab investigating the poisoning of former Russian double agent Sergei Skripal. Those attacks did not turn destructive—although no telling if they might have had security researchers not spotted them first.
• #11: At the beginning of September, British Airways revealed a data breach that impacted information from 380,000 reservations made between August 21 and September 5 of this year. The company said that names, addresses, email addresses, and sensitive payment card details were all stolen in the breach. Hackers from the well-known criminal group Magecart pulled off the attack by specifically evaluating the airline's digital systems and tailoring a plan for installing malicious skimming code in its payment data entry forms. That way, any time someone entered information to make a reservation, all the data would silently go to Magecart. Cathay Pacific also announced an even larger data breach perpetrated in March that impacted 9.4 million travelers. The airline first disclosed the breach at the end of October. It then added in November that the intrusion had been even more intense than it originally said, and that it took three months to fend the hackers off. Cathay has been widely criticized for its delayed disclosure and lack of transparency about the incident. Data stolen in the breach included passenger names, dates of birth, addresses, telephone numbers, email addresses, nationalities, passport numbers, frequent flier membership numbers, and other ID numbers. Airlines can be a particularly valuable target for hackers, because they hold both personal and financial data, as well as travel data and passport numbers.
• #20: What security relevant acceptance criteria is missing? User logs in with his credentials. k
• #21: What’s missing? How can this feature be exploited? What security relevant acceptance criteria is missing? The user authenticates himself Talk at your tables for 5 mins about how this feature can be exploited, what security relevant acceptance criteria should be added and then share.
• #23: One way to look at security: typical security requirement How excited would you be about implementing this feature? Functional features tend to overshadow these types of requirments!
• #24: Here’s another way…. Identify how attackers may abuse the system and jeopardize stakeholder assets. Help organizations see their products in the same way attackers do. Describe how users can misuse a system with malicious intent. Every time a new requirement, feature is created, someone should spend time thinking about how that feature might be unintentionally or intentionally abused.
• #25: Help organizations see their products in the same way attackers do. Describe how users can misuse a system with malicious intent. Every time a new requirement, feature is created, someone should spend time thinking about how that feature might be unintentionally or intentionally abused. Stories are logically equivalent, except from a planning standpoint Estimating value, cost and effort significantly more difficult for the abuser story
• #26: Brainstorm some abuser stories for this feature. TIMEBOX = 5 mins
• #27: What’s missing? How can this feature be exploited? What security relevant acceptance criteria is missing? The user authenticates himself Talk at your tables for 5 mins about how this feature can be exploited, what security relevant acceptance criteria should be added and then share.
• #30: No absolute guarantee that no exploitable vulnerability remains.
• #31: What would be some acceptance criteria for this story?
• #32: Add refutation criteria to your abuser stories! Timebox = 5 mins What would be some refutation criteria for this story? User cannot login without 2 forms of identification User cannot see credit card information
• #34: User stories carry business value. Abuser Stories bring an expected cost = loss due to a successful attack or probability of an attack
• #35: Examples: If a design assumes that connections from the Web server to the database server are always valid an attacker will try to make the Web server send inappropriate requests to access valuable data. If the software design assumes that the client never modifies its Web browser cookies before they are sent back to the requesting server (in an attempt to preserve some state), attackers will intentionally cause problems by modifying the cookies. Every sprint should optimize net value!
• #36: Abuser stories value and rank can be affected by…. A technological breakthrough may make an attack easier and therefore more likely. Assets may become more attractive targets. Adversaries may become better funded. Similar systems may since have been secured, making the system being developed the weakest in its class. Countermeasures taken in previous sprints may increase the risk of an abuser story because it has become the easiest way to attach the system.
• #40: User stories are written by customers. Customers should also be involved in writing abuser stories, as they are attuned to the business assets which need protection. However, to achieve a good threat coverage quickly it is essential to draw on the expertise of the development team because many hands make light work and because developers distinctive areas of expertise tend to make them sensitive to certain types of threats sooner than non-technical authors. Some of the systems assets are, by definition, of a technical nature. In the example of the gambling web site, it is likely that customers will quickly come up with threats to various accounts. For example, they may point out that accounts holding user’s gains must be protected from attack. Threats to the randomness of the gambling process, on the other hand are more readily identified by a developers. So abuser stories depart from traditional agile requirements engineering to the extent that they are no exclusives written by customers, but jointly with the development team. They reinforce the agile principle of involving all team members in a broad spectrum of activities. No one I deemed to have a monopoly on a given area of expertise.
• #41: Assets are a good starting point for writing abuser stories. Anything of value to the customer which is potentially accessible through the system, should be considered a target. An asset may have intrinsic value, such as a money in a bank account, or it may derive its value from its role in revenue generation, such as a random process at a gambling site. The latter are harder to identify, but will tend to show up with examining who the attackers are, their motivation, resources and expertise.
• #42: The nature of an attack is largely determined by the kind of adversary. I therefore pays to reflect on who protential abusers may be. Pertitnetn factors include the resources they command, their skills, motivation and risk aversion. Predators co-evolve with their prey and hence sensitivity to the species that inhabit the customer’s ecosystem is required. The history of the customer’s industry is typically a good guide to the motivation and even the attack techniques. Skills and resources are, in a certain sense, interchangeable as a resourceful adversary can hire skillful mercenaries. Organized crime is a resourceful adversary. So are intelligence agencies and terrorists. However, their motivations are different and they will go after different targets, use different techniques and have a distinctive risk assessment. Attackers are unlikely to invest many resources unless they have a clear motive. At the other end of the spectrum lie low-investment acts of vandalism. Threats from low-skilled system users may have devastating consequences. Secret gamblers using the example gambling site may rather deny using the site than settle their debts. Customer staff are a rich source of inspiration for potential attackers. The majority of fraud cases occur with inside help.
• #43: Add refutation criteria to your abuser stories! Timebox = 5 mins What would be some refutation criteria for this story? User cannot login without 2 forms of identification User cannot see credit card information