Used Again Machine Learning In My Workout, This Time Building a Personal TrAIner

In aprevious story, I tried to resolve how to find if an exercise is executed in a correct form, using OpenPose to gather exercise data and Dynamic Time Warping to compare the exercise execution with a correct one as a baseline.

This time I added the following other components to greatly enhance the overall experience:

• The awesome Face Recognition library ( https://github.com/ageitgey/face_recognition ), used to perform face recognition and load an appropriate workout
• The awesome Speech Recognition ( https://github.com/Uberi/speech_recognition ), used to process user’s answer
• The awesome Google Google Text-to-Speech ( https://github.com/pndurette/gTTS ), used to talk and guide the user

This is a graphical representation of the workflow

I’m still using the “dockerized” version of OpenPose ( https://github.com/garyfeng/docker-openpose) — installing locally is quite long — and tweaking some parameters is possible to achieve good fps too.

The experience is quite straightforward:

• You enter in the exercise area and the Face Recognition component match your face, extracting a profile (just a name for now)
• The trainer asks if you’re ready to start and waits for an answer (a “yes” at least — this part is scripted but, of course, can be extended with more complex solutions)
• The trainer loads your associated “wod” — workout of the day — with the exercises to perform (just one for now)
• You perform exercise repetitions and sets while the OpenPose component records the body data
• The trainer compares the gathered data with the correct baseline, evaluating the dynamic time warping value for every part of the body and finally giving a median value.
• If the value is below a certain threshold, the exercise is considered correct and the trainer gives this feedback otherwise it warns (a generic warning for now, but should be possible to have specific hints based on which body parts were incorrect)
• Repeat until all the repetitions are done

But let’s see in action — sorry for the video editing, definitely I’m not good doing it :)

I worked hard to have a decent experience but, of course, there is a lot more to do.

For example, now the exercise estimation is timeboxed but it should be possible to train the system to understand when an exercise is started and finished, discarding other frames for the evaluation.

Moreover, this is just a unique exercise type (pullup) and with a baseline correct exercise specifically done with my body measurements, but it should be possible to perform different exercises with a generic baseline.

A more precise feedback on what was specifically wrong during execution is another interesting point to investigate.

One achieved this, it could be used for any kind of exercise (fitness, yoga, etc) and with objects too (using custom tracking object algorithms), in a standalone mode, or as an assistant to a real trainer.

This is another example showing what is possible to achieve in hours, combining different tools and libraries, something that a few years ago would have been unthinkable and that never ceases to amaze me!

I hope you enjoyed reading this (but take a look at the above video too) and thank you!