Remote practical instruction using web browsers

Abstract

This  paper  introduces  a  novel  approach  to  remote  coaching,   specificallytargeting the body movements of learners participating remotely. The proposedsystem  employs  a  smartphone  camera  to  capture  the  learner’s  body  andrepresent it as a 3D avatar. The instructor can then offer guidance and instructionby manipulating the 3D avatar’s shape, which is displayed on a web browser.The main challenge faced by the system is to enable the sharing and editingof 3D objects among users.  Since the HTML5 drag-and-drop feature is inade-quate for transforming virtual objects consisting of multiple interconnected rigidbodies,  the  system  tracks  the  pivot  point  of  the  manipulated  rigid  body.   Itassigns attributes such as pivot points and action points to each object, extendingbeyond their 2D screen coordinates.  To implement the system, an interactiveweb  application  framework  following  the  model-view-view-model  (MVVM)architecture  is  utilized,  incorporating  Vue.js,  Three.js,  and  Google  Firebase.The  prototype  system  takes  advantage  of  the  data  binding  capability  of  theframework  and  successfully  operates  within  the  3D  space  of  a  web  browser.Experimental  results  demonstrate  that  it  can  effectively  share  transformationinformation with an average delay of 300 ms.