X-pell It: A Mixed-Reality Game for Kids' Computational Thinking

Technologies: C#, Unity 3D

Type: course project

Hardware specifications

X-pell it was developed in Unity C#, using projection-based MR (wall-mounted or TV) along with two HTC Vive trackers. Each tracker is attached to the child’s hand in a “hand wrap” fashion, and is visually projected as a “pin” icon on the screen.

Software specifications

Summary. As a high-level goal, the game was designed as a collaborative MR experience to promote the learning of coding concepts (especially loops and conditionals) to children.

Brief note. The game draws inspiration from automata theory. An automaton is an abstract model of a machine made of a finite set of states. It runs when it is given a sequence of inputs in discrete steps. At each moment during a run of the automaton, the automaton is in one of its states. When the automaton receives a new input, it transitions to another state based on a transition function that takes both the current state and the input as parameters. The automaton reads inputs one after another, and transitions from state to state according to the transition function, until all inputs are read completely.

Game play. X-pell it is a spelling game in which two children collaboratively play together. The game objective is to correctly spell a word (e.g. “water”, “thought”) displayed on the screen. To achieve this, players take turns popping balloons in order to sequentially give inputs to the automaton-like game machine. Whenever a player provides an input, the game takes both the current state and player input as parameters to transition to another state; by doing so, the transition outputs a letter to be appended to the spelled word. The transition from state to state continues as the players collaboratively work on providing the correct sequence of transitions (that is, the correct word spelling). If a wrong letter is spelled, players can undo the most recent transition and return to the previous state. The game ends when the desired word is spelled completely.

While playing the game, children learn and act on programming concepts (sequences, loops, conditionals, logical operations). They run computation logic by deliberating on courses of action for spelling the word, such as “IF I pop the red balloon, the letter ‘a’ will be added to the spelling, ELSE IF I pop the green balloon, the letter ‘b’ will be added”.

Player input. The player slowly hovers a balloon to select it, or quickly hits a balloon to pop it. The player can also hover over the undo button.

Types of transitions. The game uses a variety of transition types to visually communicate algorithmic concepts (e.g. logical operators, sequences, loops, conditionals).