September, 2024 - Vol. 2
Welcome to Vol. 2 of our GGR ML newsletter!
We offer a review of the embedded AI + ML space, reporting on both semiconductor IP and software with a focus on deeply embedded low-power edge-based devices.
Our monthly newsletter will offer the following.
• A review of one of our own low-power ML projects
• News in Brief
• Interesting Products
• Getting started in Embedded AI and ML
• Educational resources
• Upcoming Events
In our next newsletter, we will take a look at the ARM U-55 uNPU (Micro Neural Processing Unit). It is gaining wide usage with ARM licensees in low-power edge devices. We will highlight one implementation from Alif Semiconductor and show how a Tensorflow Lite model is mapped onto the uNPU hardware to improve inference performance.
News in Brief
The TinyML Conference was held in Burlingame, CA in April with a growing list of exhibitors. Large Language Models (LLMs) are not the first thing you associate with low-power edge-based ML but they are having an impact in the TinyML world as well.
Here’s information on one presentation showing a 3B parameter LLM running on a Raspberry PI 5 with 8GB of memory. The video shows how to get the LLM running on your own Raspberry PI, so, have fun trying it out.
Syntiant is also positioning its’ extreme low power NPU as an LLM companion.
Semiconductor News
Interesting Products
A lot of our ML work involves edge-based cameras. Here are some interesting products that have caught our attention in the last month.
High Resolution Depth Sensor with CSI camera interface
AMS Osram offers an IR (Infra Red) capable, high-resolution depth camera with a MIPI CSI interface.
The camera is designed to be paired with a VCSEL (Vertical Cavity Surface Laser) that produces a dot projection pattern similar to what the iPhone uses for depth perception.
Their Raspbery Pi Evalutation Kit includes the camera but no VCSEL so we are in the early stages of adding VCSEL dot projection. The picture below shows the VCSEL powered and emitting IR (at 940nm).
IR is not visible to the human eye but using a camera with “false coloring” and the IR filter disabled the VCSEL dot projection pattern can be seen!
Event-based cameras
Event-based cameras offer extremely high frame rates (10k+ fps) by sending only pixel deltas. IE instead of sending a full frame on every update only pixels that change are sent with the full image assembled on the receiving end. Here’s a video showing the Oculi camera in action.
One use case we have been exploring is high-speed microfluidics monitoring in Blood Analyzers. We will provide more information in a future newsletter.
Getting Started in Embedded AI / ML
Just getting started in the deeply embedded AI + ML world? We will regularly update our recommended short courses and reading material.
Here are some new books we have been reading
TinyML Cookbook, Gian Marco Iodice
The book offers a practical introduction to TinyML with a focus on hardware platforms like Raspberry Pi Pico and Arduino Nano that are more accessible to those just getting started in the embedded hardware and software world.
Hands-on examples the reader can work through include a simple weather station as well as audio and imaging examples that are all developed using Tensorflow and Edge Impulse’s end-to-end web-based framework.
Machine Learning Yearning, Andrew Ng.
The goal of this e-book is to provide a practical guide for setting up and executing an ML project. The chapters are short and focused.
The e-books start with recommendations on partitioning data into training, test, and dev sets and advice on picking a metric to focus on as you optimize your model.
The e-book includes good practical information on debugging your ML pipeline, error analysis, and the challenges of using synthesized data.
Ng’s e-book is good to read prior to jumping into Chip Huyen’s more comprehensive Designing Machine Learning Systems (see below).
Pinned Recommendations
Andrew Ng’s Coursera course is a great entry point to a more formal understanding of ML models and minimization functions and ultimately, an understanding of the architectures behind the plethora of silicon-based hardware accelerators hitting the market.
AI + ML projects have unique workflows that include elements from traditional Software Development (eg. CI/CD – Continuous Integration/ Continuous Development) along with large data set management and run-to-run tracking of model accuracy.
Chip Huyen’s book “Designing Machine Learning Systems” is required reading for anyone setting up Machine Learning pipelines.
NPU Hardware Development Platforms
Getting hands-on with ML workflows and actual hardware is always the best way to go.
Here is a short list of some reference design boards with NPUs. These are a good starting point for prototyping or personal development. We have included an approximate price to get started.
Alif Semiconductor Ensemble Family - Dual Core CortexM55 + Dual Ethos U55.
Alif’s device is now in mass production. As of this writing the board can be purchased from Arrow for $249.00.
The platform is designed for low-power ML apps and has a well-developed SDK with good examples that you can run immediately as well as leverage for your own application. In contrast to the Raspberry PI and Arduino platforms highlighted in “TinyML Cookbook” (see above) the Alif platform will allow you to work with the ARM NPU, an extremely low-power ML accelerator.
We have been developing Bare Metal (no Operating System) models for this board for a number of months. We will share some of this work, utilizing the low-power CSI camera interface on the board, in a future newsletter.
NXP i.MX93 with the ARM U65 NPU
The ARM U65 core is an enhanced version of ARM’s U55 NPU used in the Alif platform. Reference design boards for the i.MX93 are available from a number of suppliers including NXP.
Variscite has a nice implementation that we have recently started working with. More info here.
Variscite supplies the i.MX93 as a SoM – System on Module. SoMs have become more common in the embedded world as DDRx speeds have passed the gigabit threshold. Having a prequalified SOM removes considerable design risk as they can be plugged into a baseboard that typically has custom I/O interfaces.
In Variscite’s case, the off-the-shelf baseboard they offer is common to all their SOM implementations. If you are doing a lot of embedded development this can prove very cost effective: purchase one base board and as needed plug in the much less expensive SOM boards for your R+D work. We recently paid $100 for the i.MX93 SOM.
Himax Technologies, Inc.'s At $15.99 the WiseEye2 (WE2) offers the most economical way to check out the Ethos U55 NPU. It includes support for Tensorflow and PyTorch.
More information here.
Links to Recent and Upcoming Events
Events of interest to the embedded low power ML community.
TinyML Milan June 24-26 Milan
TinyML holds annual conferences in both the US and Europe. https://www.tinyml.org/event/emea-2024/
Design & Reuse -- D&R IP-SoC in Silicon Valley (April)
The Design & Reuse Conference offers insights into emerging IP including Neural Processing Units (NPUs) and complementary low power IP. The ams OSRAM low power CSI-2 camera core was highlighted at the conference. More information here.
Embedded Vision Conference – Santa Clara (May)
More information here.
Комментарии