Pause-and-play: Automatically linking screencast video tutorials with applications

Suporn Pongnumkul, Mira Dontcheva, Wilmot Li, Jue Wang, Lubomir Bourdev, Shai Avidan, Michael F. Cohen

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Video tutorials provide a convenient means for novices to learn new software applications. Unfortunately, staying in sync with a video while trying to use the target application at the same time requires users to repeatedly switch from the application to the video to pause or scrub backwards to replay missed steps. We present Pause-and-Play, a system that helps users work along with existing video tutorials. Pause-and-Play detects important events in the video and links them with corresponding events in the target application as the user tries to replicate the depicted procedure. This linking allows our system to automatically pause and play the video to stay in sync with the user. Pause-and-Play also supports convenient video navigation controls that are accessible from within the target application and allow the user to easily replay portions of the video without switching focus out of the application. Finally, since our system uses computer vision to detect events in existing videos and leverages application scripting APIs to obtain real time usage traces, our approach is largely independent of the specific target application and does not require access or modifications to application source code. We have implemented Pause-and-Play for two target applications, Google SketchUp and Adobe Photoshop, and we report on a user study that shows our system improves the user experience of working with video tutorials.

Original languageEnglish
Title of host publicationUIST'11 - Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology
Pages135-144
Number of pages10
DOIs
StatePublished - 2011
Externally publishedYes
Event24th Annual ACM Symposium on User Interface Software and Technology, UIST'11 - Santa Barbara, CA, United States
Duration: 16 Oct 201119 Oct 2011

Publication series

NameUIST'11 - Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology

Conference

Conference24th Annual ACM Symposium on User Interface Software and Technology, UIST'11
Country/TerritoryUnited States
CitySanta Barbara, CA
Period16/10/1119/10/11

Keywords

  • Instructions
  • Screencast
  • Video tutorial

Fingerprint

Dive into the research topics of 'Pause-and-play: Automatically linking screencast video tutorials with applications'. Together they form a unique fingerprint.

Cite this