Prototyping Tool · iOS · 2016

Form Enhanced

A patch library and container app that took Form, the node-based prototyping tool for iOS, out to hardware, music, games engines and the client's hands.

A phone held in one hand running a generative pattern, with the Form patch graph that drives it laid out either side

RoleDesign & Tooling

Year2016

Built onForm · Node-based

ReachHardware · Music · Games

Overview

Form was built to test gestures and interactions on a real phone. Form Enhanced kept that simple, node-based system and added the things client work kept asking for: patches for the hardware and software around the phone, and a way to send the prototype to someone who doesn't own Form.

Build a prototype, ship it through TestFlight or wrap it in a container app. The client taps through the proposed UI on their own device, then records the screen with their commentary and sends the feedback straight back.

Build

Wire the prototype in Form. The enhanced patches sit in the library beside the originals, each with a documented card.

Send

Ship it via TestFlight, or build a branded container app so the client opens a real app, not a viewer.

Feedback

The Screen Recorder patch captures screen and microphone, so a client's walkthrough and commentary come back as one clip.

The Screen Recorder patch card beside a phone showing a recorded prototype ready to play back — Screen Recorder: a screen and microphone recorder, with playback and sharing built in.

The same prototype open on a laptop, tablet and phone — Container apps ran the prototype as a standalone install on the client's devices.

The patch library

Every enhanced patch followed Form's own rules: one job, typed ports, and a card in the library explaining what it does. This is what it reached.

iOS Device patch: name, model, battery, IP address, latitude, longitude, proximity, motion and camera feed, with vibrate, flash, status bar and orientation inputs

Device Location and per-limb device patches, with the Vitruvian Man motif

Button Kit patch wired between a touch interaction and a view

Filters Pack patch, three filter packs with sixteen effects, wired into a view

GamePadController patch: connected, trigger, shoulder buttons, buttons, d-pad and sticks

Thalmic Labs Myo patch: acceleration, pitch, roll, yaw and pose from the gesture armband

Sphero patch: position, pitch, roll, yaw and quaternion values from the rolling robot, with colour inputs

Sphero patch card beside a Philips Hue bulb

Ableton patch linking Live 7+ over LiveOSC, with an array input and output

Serato patch: crossfader, needle velocity, song position, BPM, artist and album from Serato DJ

Beardyman audio playback patch with a set of named samples

Unity patch with send, received and name ports, next to a Network patch listing Unity and UE4

UE4 patch with sender, receiver and port number

Device, UI kit, filters, gamepad, Myo, Sphero, Hue, Ableton, Serato, sampled audio, Unity and Unreal, each as a patch.

Talking to Unity and Unreal

The Unity and UE4 patches let a live mobile prototype send and receive state from a running game engine over the network. A phone's motion, touch and location could drive a scene; the scene could drive the phone's UI back. For games studios it meant testing a companion UI against the actual game, not a mockup of it.

A Form graph: touch interaction and scroll gesture feeding an iPhone patch, its location, motion and orientation sent on to Unity — Touch, gesture and device sensors wired through to a Unity sender.

The engine side: a Unity scene receiving the phone's data live.

EA Games, Warner Bros, Konami, Nordeus, Ubisoft and Sega

Filters and shaders

The Filters Pack bundled sixteen image and camera effects behind one patch, and a set of GLSL shader patches ran generative visuals on the phone's GPU, driven by the same signals as everything else in the graph.