DejaVu: Integrated Support for Developing Interactive Camera-Based Programs

The increasing popularity of interactive camera-based programs highlights the inadequacies of conventional IDEs in developing these programs given their distinctive attributes and workflows. We present DejaVu, an IDE enhancement that eases the development of these programs by enabling programmers to visually and continuously monitor program data in consistency with the frame-based pipeline of computer-vision programs; and to easily record, review, and reprocess temporal data to iteratively improve the processing of non-reproducible camera input. DejaVu was positively received by three experienced programmers of interactive camera-based programs in our preliminary user trial.

This research was done during my internship at Microsoft Research Asia HCI Group. My mentors were Xiang Cao and Sean McDirmid.

Canvas

• Visualizing various variable contents by drag-and-drop
• Continuously showing camera input, variable contents, window output
• Sketching capability

Timeline

• Auto-recording, replaying (input streams) & refreshing (variables) to handle non-reproducible input
• Contents synchronized with canvas