The feature that Apple took 26 years to get right

OSX Mojave comes with a little-publicised feature called Stacks. Apple patented the design for this in 1992, first implemented it in a half-hearted way in 2003, and then it took another 15 years before they launched it in the way it was originally envisaged.

Apple is a famously secretive organization. But in 1992 and 1993 they published research articles on the protoype of a feature that would take until 2003 to appear in OSX. This wasn’t any prototype though, it was a semi-working implementation of an enormously sophisticated user interface that was subjected to rigorous user testing. And they didn’t publish somewhere obscure. The work was published and presented at CHI, then, and still today, the biggest and most important academic interaction design conference in the world.

For us, now, this seems amazing. Secretive Apple literally standing on a stage and describing in detail their highly innovative designs and the research approach that led to them. But in the 90s Apple had a whole team of people devoted to doing cutting edge user-interface work and telling the world about it. The authors of this paper, Richard Mander, Gitta Salomon and Yin Yin Wong are still active interaction designers though all have left Apple.

A “pile” metaphor for supporting casual organization of information

Mander, Salomon and Wong’s paper is called “A “pile” metaphor for supporting casual organization of information”. The problem presented in the paper is that of organizing documents using a graphical user interface.

Todays’ direct manipulation computer interfaces, such as the Macintosh desktop interface, offer limited means of handling information. Users can manually place files within folders, organized in a rigid hierarchy. Users are responsible for appropriately filing all items; the system offers little assistance in this often tedious task. Recent enhancements, such as “aliases”, allow users to overcome a frequent problem, namely that an item belongs in more than one folder. However, the folder as the sole container type presents an impoverished set of possibilities.

To find out about physical document filing, Mander, Salomon and Wong conducted 13 interviews with people at Apple. They took documents with them (they don’t say what the documents were) and asked their participants to show how they’d file them. They were also interested in automated assistants so they asked some of their participants who had assistants, secretaries I guess, how they worked with them. (They were asking about automated assistants. In case you missed it, besides the whole Piles/Stacks thing, the thread of interaction design research that led to Siri goes back to at least 1992.)

From the user interviews they developed a “design sketch” in “Macromind Director“. Today this seems like pretty standard practice — showing clickable prototypes — but this was 1992 and Director isn’t Sketch. I also think the paper elides that they must have had less refined prototypes before they fired up Director. The “sketches” seem quite well worked out with different representations for user-created piles of documents and algorithmically generated ones. There’s also a cute animation for adding a document to a pile. Mander and colleagues also present a user-interface for scripting a pile, essentially a search interface, a mouse gesture for disassembling a pile and another mouse gesture for browsing a pile while maintaining the pile in place.

Mander_Piles_Fig_2
Figure 2 from Mander et al.

The sketches described in the paper were animated but not interactive. A subsequent paper describes a graphical prototype of some of the more sophisticated sorting algorithms and user interface elements. In both papers, the “piles” UI element is vastly more sophisticated than the implementation available in OSX today.

The “sketches” were tested with ten people, five men and five women, in “non technical positions” at Apple. In general the users understood the interface and even liked it. It seems like the participants were relatively sophisticated users given some of their comments:

Most users asked for features generally available in desktop systems, but which were not present in the testing prototypes. For example, they wanted to be able to add a selected group of items to a pile, name piles, apply ordering schemes based on date, size, name, and kind, and control where a document was placed within a pile.

As this was an academic paper, it concludes with a “further research is needed” statement.

Unstated is the fact that Apple patented this UI approach around the time the paper was published. It took another ten years before the pile metaphor appeared in an Apple operating system. Even then the tech press called the commercial implementation out for being impoverished compared to the research vision.

This paper matters today as more than a 25 year old artifact of a very different Apple. It’s evidence that much of what we do in creating novel, useful or sympathetic interactions has a long history, one that reaches back several technological generations but one that is also recognizable today as good practice.

Bonus Unexpected Insight

This seems like a lot of incoming information streams for 1992:

At the end of the test, we asked users how they would use piles, and how the system might assist them. In general, users were receptive to the idea of having the system help them with their routine tasks, such as sort-ing incoming mail. Most users reported having between two and five mail systems, fax, and voice mail. (my bold added)

Formal reference

Mander, R., Salomon, G., & Wong, Y. Y. (1992). A “pile” metaphor for supporting casual organization of information. In CHI ’92: Proceedings of the SIGCHI conference on human factors in computing systems (pp. 627-634). ACM. doi:10.1145/142750.14305

Canonical URL

http://dl.acm.org/citation.cfm?id=143055 (paywalled)

Publicly available source

http://graphics.cs.columbia.edu/courses/csw4170/resources/p627-mander.pdf

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s