In this series of liveProjects, you’ll use the ReactJS JavaScript framework and the TensorFlow.js PoseNet model to create an exercise mobile app that estimates and tracks human poses. Each project in this series delves into a different and standalone aspect of constructing an AI-enhanced mobile app.
These projects are designed for learning purposes and are not complete, production-ready applications or solutions.
here's what's included
Project 1 Build a Web App
In this liveProject you’ll create a React web application and then integrate TensorFlow.js machine learning functionality into the app. React is highly prized by developers for its ease of building simple and intuitive frontends. Once you’ve set up your development environment and built your React-based UI, you’ll train your TensorFlow.js model with synthetic data.
Project 2 Deploy a PoseNet Model
In this liveProject, you’ll set up and tune a TensorFlow.js PoseNet model in a React web app. PoseNet allows real-time human pose estimation in your browser, all through the magic of TensorFlow.js. You’ll enable a WebGL backend, set up a system where PoseNet can utilize a webcam stream for its data input, and establish a canvas that can draw on top of an estimated pose.
Project 3 Code the UI with Material
In this liveProject you’ll use ReactJS to build an entire user interface for a machine learning-powered exercise app. Your app records its users while they exercise, and so needs to have a short delay between them pressing a button and the start of recording. You’ll use Material-UI for React to implement layout, form elements, buttons, and user messages, and set up the UI logic for your delayed training data capture.
Project 4 Train a Model
In this liveProject you’ll use human pose data and TensorFlow.JS’s PoseNet to build and train a machine learning model that can recognize workout exercises. This model will record and recognize the workout session of a user, to be logged for future comparison. You’ll need to prepare data structures for the TensorFlow.js Dataset API, and execute one-hot-encoding with a simple mapping function. You’ll also define model architecture using the TensorFlow.js sequential API, and train your model using the fitDataset method. You’ll finish up by saving your trained model in local browser storage to allow for model reuse.
Project 5 Classify a Workout
In this liveProject, you’ll test the functionality of a TensforFlow.js based fitness assistant and assess its capabilities of pose estimation for workout sessions. You’ll move data from a PoseNet model to a TensorFlow.js inference function that can recognize a workout type and add delays to prevent unnecessary logging duplication. You’ll then utilize TensorFlow.js’s prediction functionalities to get prediction results for your recognized workouts, pick top scores, and return the recognized workout type. Finally, you’ll use Material UI and React.JS to display a complete workout history to the user.