Recognition of Design Fixation via Body Language Using Computer Vision

The main objective of this study is to recognize design fixation accurately and effectively. First, we conducted an experiment to record the videos of design process and design sketches from 12 designers for 15 minutes. Then, we executed a video analysis of body language in designers, correlating body language to the presence of design fixation, as judged by a panel of six experts. We found that three body language types were significantly correlated to fixation. A two-step hybrid recognition model of design fixation based on body language was proposed. The first-step recognition model of body language using transfer learning based on a pretrained VGG-16 convolutional neural network was constructed. The average recognition rate achieved by the VGG-16 model was 92.03%. Then, the frames of recognized body language were used as input vectors to the second-step fixation classification model based on support vector machine (SVM). The average recognition rate for the fixation state achieved by the SVM model was 79.11%. The impact of the work could be that the fixation can be detected not only by the sketch outcomes but also by monitoring the movements, expressions, and gestures of designers, as it is happening by monitoring the movements, expressions, and gestures of designers.

Citation

Mathematical Problems in Engineering, volume 2021

Publisher

Hindawi

URI

http://hdl.handle.net/10034/625781

Description

From Hindawi via Jisc Publications Router
History: received 2020-12-21, publication-year 2021, rev-recd 2021-06-19, accepted 2021-08-12, archival-date 2021-08-27, pub-print 2021-08-27
Publication status: Published
Funder: National Natural Science Foundation of China; doi: http://dx.doi.org/10.13039/501100001809; Grant(s): 51905175
Funder: Shanghai Pujiang Talent Program; Grant(s): 2019PJC021
Funder: Ministry of Education of the People's Republic of China; doi: http://dx.doi.org/10.13039/501100002338; Grant(s): 202002SZ05
Funder: Shanghai Soft Science Key Project; Grant(s): 21692196800