multimodal multitouch game

Graduate Student Mentor:Bill Hamilton

The team will develop a fun and engaging game using 4 iPhones and an iPad. Each player will be given an iPhone as a private screen. This private screen can be used to show player-centric private information, such as his/her hand of cards. A shared display, consisting of a multitouch projection surface is used for public data and group interactions. This might be where the public cards and the pot are located, e.g. in a texas hold'em style poker game. Students are encouraged to find (or create) a fun and interesting board and/or card game to implement.

Students will investigate methods of bridging the two modalities (iPhone private display & projection surface public display) into a coherent interface ecosystem. Issues involving information transfer from one device to another, while maintaining a consistent game state will arise. Students should also consider the physical and information gap that exists between the two devices, and how that will influence the experience. A review of the prior work will aid in finding a good solution to these problems. You should also consider the role of interpersonal interactions within the context of the game itself. Does playing with the iPhones draw players attention away from other players? How will the digital implementation of the game alter and/or enhance the experience. Some information in games is not a direct part of the game mechanic, but is implicit in the interactions it inspires. Understanding the different levels of communication inherent in the game that you choose is key.

The team will study relevant literature on Computer Supported Collaborative Work (CSCW) specifically regarding private and public spaces for human-computer interaction and visualization. The readings will make the students aware of research issues involved in multimodal interaction of teams. They will develop and analyze various forms of interaction within the context of their game. The game developed will leverage the novel features of the provided hardware and software to explore new interactions between players.

Possible card games include poker, bridge, spades, etc., but battle card games are also a possibility. There are also many interesting board games that might be considered including: Pandemic, Cosmic Encounter, and Fairy Tale

bonus objectives

  • incorporation of iPhone sensors into game - your game could incorporate things like the built-in accelerometer, compass, etc. The interaction must go beyond a trivial use of the sensors, and really add to the gameplay to qualify for extra credit. Keep in mind the more obscure sensors such as the proximity sensors.
  • augmented reality integration - Using the iPhone AR toolkit, incorporate augmented reality in your game. This could include fiducial markers on each iPhone, or other tangible markers incorporated into the game world and used as a part of gameplay.
required readings
technical resources
project resources
  • 4x iPhone 3GS
  • iPad WiFi
  • 8-core Mac Pro /w Snow Leopard
project-specific deliverables

Throughout your project you are encouraged to play a variety of games, both physical and digital. This activity will expose you to a variety of game mechanics and experiences, it will give you new ideas about games and play.

Game Analysis

Write a critical breakdown and analysis of two card or board (depending on what you plan to implement) games that are considered for adaptation. Every team member should have experience with the games; preferably, they should have been played together.

Visual aids (diagrams, flow charts, screenshots, photographs, etc.) will be essential. The breakdown should address the following in a narrative (not outline) format:

  1. an overview of the games' rules, including the choices the player(s) make, outcomes of decisions (game mechanics)
  2. notable subsystems / patterns (feedback loops, for example; constituent rules)
  3. social interaction / rules (implicit rules)
  4. representations: information available to the player and how it is encoded in the game (game board, external representations (scoreboard?), head-up display)
  5. how rules are encoded in the system (as Juul notes, the laws of physics encode many of the rules of sports)
Lightweight Prototype (addendum)
  • Your lightweight prototype must include game mechanics; it is insufficient to present only interface design.
Project Plan (addendum)
  • In your project plan, reflect on your game analysis and discuss it. Your project plan should address the following questions:
    1. How did what you learned from the game analysis inform your game design?
    2. What game design elements (from the literature) are you employing? Justify your decisions.