Abstract: Traditional teleoperation systems have many shortcomings in real-time performance, complexity, and robustness when executing unstructured tasks, urgently requiring the construction of a low-cost, high-precision, and natural-smooth human-machine interaction control scheme.Based on this, a visual-based teleoperation system for robotic dexterous hands based on motion retargeting is designed, mainly including three modules:hand key point detection, hand pose retargeting, and robot action generation.Firstly, a visual human hand 3D pose estimation algorithm is used for high-precision detection of hand key points.Secondly, an optimization objective function for retargeting is designed by combining the angle and distance errors of vectors, and a dynamic scale scaling function and a piecewise weight function are introduced for hand pose retargeting.Finally, the robotic action generation module is adopted to achieve natural mapping from human hand poses to robotic dexterous hand movements.Through experiments and performance evaluations on multiple typical tasks, it is shown that the system has high precision, good real-time performance, and stability, demonstrating great application potential.