Science & Technology, Canada (Commonwealth Union) – In today’s digital age, the way we learn and acquire new skills has been transformed by the advent of “how-to” videos. These short, instructional videos have become a popular medium for sharing knowledge and empowering individuals to learn a wide range of skills, from cooking and crafts to home repairs and tech hacks.
How-to videos excel at providing visual and practical learning experiences. Unlike traditional written instructions or textbooks, these videos offer a step-by-step visual guide, making it easier to understand complex processes or techniques. Viewers can see the actions performed in real-time, observe the correct hand movements, and gain a deeper understanding of the subject matter. The combination of visual demonstration and auditory instructions enhances the learning experience, catering to different learning styles and ensuring greater comprehension and retention of the information presented. They can also save time as what try to understand by reading instructions can be presented to us in a video in a few seconds providing greater understanding.
Computer scientists from the University of Toronto (U of T) may take the production of how-to videos to the next level with the new robotic technology.
The research team has produced Stargazer, an interactive camera robot capable of assisting university instructors as well as other content creators make engaging tutorial videos displaying physical skills.
For individuals with no cameraperson, Stargazer is capable of capturing dynamic instructional videos while resolving restrictions while using static cameras.
Lead researcher Jiannan Li, who is a PhD candidate in the U of T, department of computer science in the Faculty of Arts & Science indicated that the robot is for assisting humans, but not replacing them.
He also indicated that the instructors were there to teach and the actions of the robot is to assist when shooting the heavy-lifting activities.
The details of the Stargazer were provided in a published paper presented in 2023 at the Association for Computing Machinery Conference on Human Factors in Computing Systems, which is a global conference in human-computer engagements.
The team for Li consisted of co-authors who were fellow members of U of T’s Dynamic Graphics Project (dgp) lab: postdoctoral researcher Mauricio Sousa, Ph.D. students Karthik Mahadevan and Bryan Wang, Professor Ravin Balakrishnan, and Associate Professor Tovi Grossman; together with Associate Professor Anthony Tang (cross-appointed with the Faculty of Information); recent Faculty of Information graduates from the university Paula Akemi Aoyaui and Nicole Yu; as well as 3rd-year computer engineering student Angela Yang.
The Stargazer applies a single camera on a robot arm, with 7 independent motors with the ability to move along with the video subject by itself as it monitors regions of interest. The camera behaviors of the system are adjustable based on subtle cues the instructors provide, like body motions, gestures as well as speech identified by the sensors of the prototypes.
Researchers indicated that the voice of the instructor gets recorded with a wireless microphone which is then transferred to a speech-recognition software. The copied text, together with a custom prompt, is transmitted to the GPT-3 program, a wide language model labeling the instructor’s intention for the camera.
The camera control instructions are cues naturally utilized by instructors for the guidance of attention of their audience and to avoid disrupting the instruction delivery, according to researchers.
While researchers presented Stargazer as a choice for those with no access to professional film crews, they pointed out that the robotic camera prototype depends on a costly robot arm together with a suite of external sensors.
“I think there’s a real market for robotic filming equipment, even at the consumer level. Stargazer is expanding that realm, but looking farther ahead with a bit more autonomy and a little bit more interaction. So realistically, it could be available to consumers,” said Li.
