Please use this identifier to cite or link to this item: http://nuir.lib.nu.ac.th/dspace/handle/123456789/2491
Title: Bhutanese Sign Language Hand-shaped Alphabets and Digits Detection and Recognition
-
Authors: KARMA WANGCHUK
Karma Wangchuk
Panomkhawn Riyamongkol
พนมขวัญ ริยะมงคล
Naresuan University. Faculty of Engineering
Keywords: Bhutanese Sign Language
BSL Dataset
Convolutional Neural Network
Visual Geometry Group
Image augmentation
Issue Date: 2020
Publisher: Naresuan University
Abstract: The communication problem between the deaf and the public is an emerging concern for both parents and the government of Bhutan. The parents are not able to understand their children. The deaf students are not able to communicate with the general public. Therefore, deaf school and government is urging people to learn Bhutanese Sign Language (BSL) but learning Sign Language (SL) is not easy. However, Computer Vision and machine learning applications have been solving communication gaps. It has been easy to learn and understand SL with the help of signs’ translation apps. The basics of all sign languages are alphabets and numbers. The purpose of this study is to develop a suitable machine learning model to detect and recognize the BSL alphabets and digits using BSL hand-shaped alphanumeric datasets. In this study, the first BSL hand-shaped alphanumeric dataset was created with different augmentation techniques. Different SL models were evaluated with the dataset. However, the Convolutional Neural Network (CNN) based architecture outperformed them. Using six layers of CNN with the batch normalization and different dropout ratios, 20000 digits dataset, and 30000 alphabets dataset obtained better results compared to LeNet-5, SVM, KNN, and logistic regression. Furthermore, ResNet with 43 convolutional layers obtained the best training and validation accuracy of 100% and 98.38% respectively on 60,000 alphanumeric datasets. This research is the first of its kind to study the possibility of machine learning integration with the BSL to detect and recognize hand-shaped alphabets and digits. It was found that machine learning models can be deployed to develop Computer Vision applications to make BSL learning easier and accessible to the general public. Further studies are needed to create a video-based dataset and study BSL dynamic gesture recognition for word translation.
-
Description: Master of Engineering (M.Eng.)
วิศวกรรมศาสตรมหาบัณฑิต (วศ.ม.)
URI: http://nuir.lib.nu.ac.th/dspace/handle/123456789/2491
Appears in Collections:คณะวิศวกรรมศาสตร์

Files in This Item:
File Description SizeFormat 
62061764.pdf5.2 MBAdobe PDFView/Open


Items in NU Digital Repository are protected by copyright, with all rights reserved, unless otherwise indicated.