iFighter
II Turbo is a Computer Vision project
I worked on with partner Niall Parker. The
system uses a standard consumer-grade webcam
to detect martial arts poses. It then
translates those poses into input commands
for games such as Street Fighter.
The system uses MediaPipe Pose to acquire
the player skeletal position. The skeletal
data points are then fed into a neural
network classifier to detect specific poses
correlating to punches, kicks, throws,
directional movement, and special moves. The
system works with fast movements, and has
imperceptible input latency on par with
Super Nintendo controller hardware (Around
30 milliseconds).
The goal of this project was to show that
modern computer vision software can be used
to facilitate effective game control without
the need for expensive hardware like depth
sensing cameras.
