Close

System Component 2: Crowdsourced Segmenting and Labeling

A project log for Facade: Tactile Interfaces to Appliances

Facade is a crowdsourced fabrication pipeline to automatically generate tactile interfaces to appliances for blind people.

anhong-guoAnhong Guo 10/09/2016 at 22:160 Comments

Facade uses a two-step workflow to label the area of the image that contains the interface and then label the individual visual elements. Crowd workers are first asked to rate the image quality, and segment the interface region. To assist with later attachment, we asked crowd workers to segment the interface region aligned with the physical boundaries of the appliance interface, so that blind people can feel that boundary and align the overlay themselves at attachment time.

Then, they are instructed to draw bounding boxes around all of the individual buttons within the interface area, and provide a text annotation for each element (such as labeling buttons as `baked potato', `start/pause'). Similar to RegionSpeak and VizLens, Facade has multiple workers label in parallel to ensure high quality of the labels, and complete labeling all of the buttons within a very short time.

Discussions