Our first remote play to learn sessions with threat modeling games

Agile Stationery conducted two Play to Learn sessions on Threat Modeling card games. This article will report how we ran them, and what we learned.

Session Format

We ran our first sessions on OWASP Cornucopia with Grant Ongers a cyber security expert and gamification fan. Our session on Elevation of Privilege was facilitated by its inventor Adam Shostack. Both figures are senior cyber security experts who play an active role in cyber security institutions at the heart of the industry.

We invited paying customers to the sessions to experience remote gameplay based on our own proposed format for remote working which we developed during the summer of 2020. The proposal calls for a physical card deck for each player and a randomly assigned hand created from our Croupier tool. This tool replaces the practice of shuffling and dealing cards in a round-robin, as you would do in person.

We played with a gamesmaster (Grant, or Adam) and a score-keeper, in addition to the players. Sometimes the gamesmaster acted as referee, using their cybersecurity expertise as a guide, and sometimes one of the players was given the role of "product owner" and it was their responsibility to award points for threats-identified.

Resources

The OWASP Cornucopia players used the rebranded deck featuring Mercury and a cornucopia on the front. The EoP players used the deck with a Goblin King rising above the the King and Sysadmin on a pillar of green light. (As we've said before, we like our deck covers to tell a story).

The event organisers used Croupier to generate hands and sent the assigned cards by email. With the exception of one stand-in player, all of the players had pulled their assigned hands from the deck and had the cards ready at the start of the session. The stand-in was able to find their cards in a pre-sorted deck in a couple of minutes and this was not disruptive.

System Diagrams

One group were introduced to the Juiceshop system architecture during the call. The others had received a written description of a sample system beforehand and drew a diagram in Miro during the call. The Juiceshop group made some further notes on the shop's features inside Zoom, but the diagram used was static throughout the call. The second group got their diagram drawn, from scratch, in about 20 minutes, and continued to evolve it throughout the session.

In both cases we saw interesting in-depth conversations about how threats might apply to the diagram. Where the diagram was easily editable, we saw an interesting dynamic where players were incentivised to expand the diagram in order to win the hand.

In both cases, while the diagram was on the screen, it was visible full size and it was easy to see all the available detail. This was useful as conversations often revolved around details on the diagram and the gamesmaster was able to point to details with the mouse.

Playing the Game

There were some interesting observations about the gameplay.

The players did not miss having a shared gaming surface. This actually surprised some people as normally you would place cards in the middle of the table as they were played, so that everyone can see. It took a little repetition, but we found it was simple to keep track of what cards were played. The gamesmaster and note taker between them would keep a written record to act as a reassurance and to award points. Some players pulled out each card as the round progressed and made a pile of them to refer to privately.

On our Zoom calls, players never queried the content of the gamesmaster's record and there was good engagement.

Each group spent ample time talking about the game and system and only played one or two rounds. One group found 7 threats at the end of their round (one per player) and the second group found 15. It was interesting to note how one card lead to two valid threats. Both the players discussing that card got points for their threats and this collaboration decided the round and the overall winner of the game.

We also saw players getting competitive and trying to show an opponents threat to be invalid. This needed a good amount of detail in the system diagram and the conversation resulted in the diagram being updated. Your opinion of competitive attitudes might vary, but system documentation was improving during the game!

Often, diagrams become stale because there is little incentive to update them or criticise them. Both the competitive atmosphere and the playful critique lead delegates to improve the state of the diagram despite having no pride invested in the codebase. Something to watch is whether this will be enhanced or diminished in real-life settings where players include the programmers and architects that created the target system.

Croupier improvements

We got a bit of feedback about Croupier and how we had set up the game. We had included the Cornucopia wildcards and aces which can require a high skill-level to play, and might sometimes be out of reach for some players. We also included lower value Cornucopia cards like the "2"s which seemed to map to threats that are low priority which some teams would choose to ignore.

We'll be making changes to allow low value, and ace cards to be removed. Wildcards in Cornucopia are a suit and can be removed already, but we will remove wildcards by default in the next version.

We also found that we quite liked the OWASP Cornucopia score sheet and will look at making that available as an output from Croupier, or as a physical product.

Conclusion

We were delighted to find that the use of physical cards on a Zoom call felt very natural. Players were engaged and got competitive. At times, competition led to collaboration on the editable system diagram, at other times apparently altrusitic collaboration lead to an extra threat being identified. This collaboration actually won that player his game.

What this means is that we have an effective model for mixed-media delivery of threat modeling sessions that maximises engagement with threat modeling content, and delivers effective threat modeling.

If you would like to participate in our next play to learn sessions, sign-up here:

Play to learn sessions