Four-Project Series

Machine Learning on Embedded Systems you own this product

prerequisites
intermediate Python, including variables, loops, and object-oriented programming • basic machine learning training and testing • basic edge computer system assembly and connection • basic device systems and integration • basic preprocessing • intermediate TensorFlow
skills learned
machine learning on embedded boards • training machine learning algorithms on embedded systems • using embedded boards to deploy trained machine learning models for real-time inference
Kanishka Tyagi & Raghavendra Sriram
4 weeks · 6-8 hours per week average · INTERMEDIATE

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases

lite $19.99 per month

  • access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more


You’re a data scientist at EKKo Inc., a machine learning consultancy. Your task is to help create an embedded system that will enable people who are deaf or hard of hearing to participate in online meetings and events with their mobile devices. You’ll set up an environment for running ML programs on an embedded system and process data from the existing American Sign Language (ASL) dataset. You’ll build, train, and validate a logistic regression model that’s capable of classifying the letters of the ASL image dataset. To help the system recognize the hand gestures that represent letters in ASL, you’ll use TensorFlow to create and configure a convolutional neural network (CNN) model.

To improve the model’s performance, you’ll add regularization and fine-tune the learning rate (LR). Using the Keras Tuner, you’ll optimize the model further by fine-tuning its hyperparameters, as well as using the TensorFlow Model Optimization Toolkit (MoT) to enable the model to be quantization-aware. You’ll take the final step in achieving EKKo’s goal for this project by connecting the quantized model with a live video stream and training it on the embedded system. When you’re done with this series of liveProjects, you’ll have a fully functional optimized CNN you can run on an embedded system that successfully detects ASL in real time.

These projects are designed for learning purposes and are not complete, production-ready applications or solutions.

Excellent course. Probably the only one of its kind on the market.

Srdjan Santic, Data Engineer, HTEC Group

Manning author Kanishka Tyagi shares what he likes about the Manning liveProject platform.

here's what's included

Project 1 Train a Logistic Regression

You’re a data scientist at EKKo Inc., a machine learning consultancy that has won a contract to create an embedded system that can be implemented on embedded systems. This system will enable people who are deaf or hard of hearing to participate in online meetings and events. After setting up an environment for running ML programs on an embedded system, you’ll process data from the existing American Sign Language (ASL) dataset and build, train, validate, and analyze the performance of a logistic regression model. By the end of this liveProject, you’ll be able to install, configure, and deploy a basic logistic regression algorithm that’s capable of classifying the letters of the ASL image dataset.

Project 2 Train a CNN

You’re a data scientist at EKKo Inc., a machine learning consultancy that’s working on an embedded system to help deaf or hard of hearing people participate in online meetings and events on their mobile devices. To help the system recognize the hand gestures that represent the letters in American Sign Language (ASL), your task is to classify them using a convolutional neural network (CNN), an algorithm widely used for image processing applications.

You’ll write a Python script that preprocesses the ASL dataset, ensuring the model can interpret it. Using TensorFlow, a popular choice for such tasks, you’ll create and configure the CNN model. You’ll train the model and improve its performance by adding regularization to avoid overfitting, fine-tuning the learning rate (LR) to increase training speed, and introducing callbacks to monitor the training process. When you’re done, you’ll have firsthand experience using TensorFlow tools to configure various CNN hyperparameters, train a CNN onto an embedded board, and generate predictions from the CNN.

Project 3 Train on Microcontroller Devices

EKKo Inc., a machine learning consultancy, is working on an embedded system to enable deaf or hard of hearing people to participate in online meetings and events on their mobile devices. Your task as a data scientist is to optimize this system. Using the model optimization toolkit for TensorFlow, you’ll train a CNN model with an existing American Sign Language (ASL) dataset and convert it to TFLite format to reduce the code footprint. You’ll optimize it further using quantization, drastically reducing size and CPU consumption while maintaining model accuracy. Using batch normalization to decrease the number of training cycles, you’ll significantly speed up the CNN model’s training process. Lastly, you’ll integrate the quantization changes you’ve made by compiling and training the CNN model. When you’re finished, you’ll have a fully functional quantized CNN that can be run successfully on an embedded device.

Project 4 Run a Real-Time ML Model

EKKo Inc., the machine learning consultancy you work for, has been building an embedded system to enable deaf or hard of hearing people to participate in online meetings and events on their mobile devices. As a data scientist, your task is to complete the system by enabling it to detect American Sign Language (ASL) in real time. You’ll define a CNN model, train it using the existing ASL dataset, and evaluate the model’s accuracy. To optimize training speed and quality, you’ll fine-tune the model’s hyperparameters using the Keras Tuner. You’ll enable the model to be quantization-aware with the TensorFlow Model Optimization Toolkit (MoT), optimizing size and CPU consumption while maintaining model accuracy. To complete the project, you’ll connect the quantized model with a live video stream and train it on the embedded system. When you’re done, you’ll have a fully functional quantized CNN you can run on an embedded system that successfully detects and transcribes ASL in real time.

book resources

When you start each of the projects in this series, you'll get full access to the following book for 90 days.

choose your plan

team

monthly
annual
$49.99
$399.99
only $33.33 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • Machine Learning on Embedded Systems project for free

Working through the challenges can be very edifying.

Giri Swaminathan, Independent Consultant, Indept

project authors

Kanishka Tyagi

Dr. Kanishka Tyagi received his bachelor's degree in electrical engineering in 2008 from Pantnagar, India. Later he worked as a research associate at the Department of Electrical Engineering, Indian Institute of Technology, Kanpur, with Dr. P.K.Kalra. He received his master’s and doctoral degree with Dr. Michael Manry in the Department of Electrical Engineering at The University of Texas at Arlington in 2012 and 2017. Currently, he works as a senior machine learning scientist at Aptiv advance research center, California. Prior to Aptiv, he worked at Siemens research, and interned in machine learning groups at The MathWorks and Google Research. He has worked as a visiting researcher at Ajou University and Seoul National University. He received the 2007 and 2011 IEEE CIS Outstanding Student Paper Travel Grant Award and 2013 IEEE CIS Walter Karplus Summer Research Grant award. Dr. Tyagi is an IEEE senior member and member of various IEEE-CIS committees. He currently serves as an associate editor for IEEE Transaction on Neural Network and Learning Systems. Dr. Tyagi has published over 30 papers and filed 17 U.S. patents and trade secrets.

Raghavendra Sriram

Raghavendra Sriram completed his bachelor’s degree in engineering at the Department of Electrical and Electronics Engineering at Canara Engineering College in Mangalore in 2012. Currently, he’s a senior development engineer for OBD systems working at Paccar, Inc. in Mt. Vernon, WA. His work focuses on developing innovative solutions to diagnostic capabilities for diesel engine misfire detection and calibration efforts using machine learning and optimization techniques. Previously, he developed and designed diagnostic algorithms for after-treatment systems. Before joining the industry, he worked as a researcher under the guidance of Dr. Frank L. Lewis in the electrical department at the University of Texas at Arlington, focusing mainly on developing and implementing intelligent control algorithms on various robotics platforms. He has extensive experience with multiple rapid prototyping systems and tools, has been awarded several university department scholarships for research, and contributed to several journals and research groups.

Prerequisites

The liveProject is for intermediate Python programmers who know the basics of data science. To begin these liveProjects you’ll need to be familiar with the following:

TOOLS
  • Raspberry Pi/NVIDIA Jetson
  • Ubuntu Desktop 18.04 +
  • Intermediate Python
  • Basic Jupyter Notebook
  • Basic object-oriented programming
  • Intermediate NumPy
  • Intermediate TensorFlow
  • Conda/pip virtual environment setup skills
TECHNIQUES
  • Understand logistic regressions and classification
  • Model assessment and regularization
  • Basic computer vision
  • Basic linear algebra

features

Self-paced
You choose the schedule and decide how much time to invest as you build your project.
Project roadmap
Each project is divided into several achievable steps.
Get Help
While within the liveProject platform, get help from other participants and our expert mentors.
Compare with others
For each step, compare your deliverable to the solutions by the author and other participants.
book resources
Get full access to select books for 90 days. Permanent access to excerpts from Manning products are also included, as well as references to other resources.