• Development of a technology that uses a high-speed camera-based fingertip sensor to accurately estimate the timing and position of finger tapping actions in real-time.　
• Construction of a system that combines CNN-based fingertip detection and DIC-based velocity estimation to detect the movements of multiple fingers at 500fps. 　
• Verification of the high accuracy and speed of finger tapping actions during interaction with a virtual keyboard.

  In this study, we propose a new concept based on software-based fingertip velocimetry utilizing high-frame-rate video processing. This technology detects the high-frequency component that arises when a fingertip makes contact with something, thereby estimating when and where the operator has tapped. By employing GPU for high-speed parallel processing, this software-based fingertip sensor can accurately and quickly estimate the velocities of multiple fingers. Digital Image Correlation (DIC) for sub-pixel precision velocity estimation is combined with CNN-based object detection operating at intervals of dozens of frames to robustly update the ROI regions of the fingertips for frame-by-frame DIC operations. We developed a real-time multi-finger tapping detection system that executes DIC operations at 500 fps for 720×540 resolution images and performs CNN-based fingertip detection at 30 fps. 

  The system's effectiveness in real-time finger tapping detection, including interactions with a virtual keyboard involving ten-finger keyboard input, demonstrates its potential. 

  Further research will explore image segmentation to eliminate background interference and improve velocity estimation accuracy, as well as applications in IoT and intelligent robotics control.