Programming with Examples to Develop Data-Intensive User Interfaces


7/15/2016 IDE PX IEEE Computer 49(7)

This page provides summary and additional information for the journal article that appeared in the IEEE Computer magazine, special issue on 21st Century User Interfaces.


The programming-with-examples (PwE) workflow lets developers create interactive applications with the help of example data. It takes a general programming environment and adds dedicated user interfaces for visualizing and managing the data. This lets both programmers and users understand applications and configure them to meet their needs.

(Programming by Example vs Programming with Examples)

Figure 1. Two workflows for developing data-intensive interactive systems. (a) Programming by example (PbE). (b) Programming with examples (PwE). Unlike PbE systems, PwE systems allow the representation of example data and enable explicit programming. IDE: integrated development environment.

What is (and what is not) "Programming with Examples"?

It is...

  • It is the workflow of programmers that makes use of example data during the development of applications.
  • It is often required for the development of modern interactive systems that have become more and more data-intensive.
  • Example target applications include IoT, robot control, gesture recognition, image processing, and animation, and others that require intensive parameter tuning such as machine learning.
  • Programmers have often developed their own tools for the purpose such as tools for visualizing recorded sensor data. We argue that such features should be built into IDEs to provide fluid programming experience.

It is NOT...

  • It is different from typical PbE systems that encapsulate and hide the implementation of programs from the user. It requires explicit programming. (See Figure 1 above.)
  • It is slightly different from the traditional definition of "Programming with Example" that appears in Brad Myers' survey paper on PbE systems [CHI '86]. It indicated the omission of inferences in PbE systems and did not discuss data representations.
  • It is different from Joel Brandt's "Example-Centric Programming" that helps programmers to benefit from existing example code on the web. Examples in his work denote the use of APIs (the logic of the programs) and examples in PwE denote data used in the programs.

Which IDEs support "Programming with Examples"?

We have designed IDEs that support the PwE workflow. Some of them are implemented as WIDEs and you can try them out! At this moment, those with Live Tuning interfaces are publicly available. See TextAlive at and f3.js at

Picode [CHI '13] supports handling robot/human posture information with the code editor that shows inline photos representing the posture data.

DejaVu [UIST '12] supports creating interacitve camera-based apps with the timeline and canvas interfaces for built-in record-and-replay and re-evaluation feature.

VisionSketch [GI '13] implements graphical annotation-based code completion where methods are suggested based on sketched shapes on input images.

TextAlive [CHI '15] supports authoring Kinetic Typography videos of lyrics text synchronized with songs equipped with Live Tuning interfaces for end-users.

There are of course excellent prior work and other ongoing work that supports the PwE workflow.

Programming with Examples in conventional IDEs

Conventional CUI and GUI applications do not involve continuous and complex data handling. As a result, their development environments typically support simple text-based or symbol-based representations of example data.

Programming with Examples in IDEs for data-intensive applications

To develop data-intensive applications, more intuitive (typically graphical) representations of example data are required other than text-based representations.

Please contact us to add your project here.

What are the design implications for future IDEs?

While we would strongly recommend you to read the article (p.40-41) for the detailed discussion, here are some lessons learned in case you are in a hurry.

Revision history

Upload the initial revision.