TinyML を試してみる
takeharak
社内で TinyML の話題が上がったので、身の回りにあるもので環境を構築して Hello World する
* ハンズオンのみ Machine Learning, TesnsorFlow Lite 等詳しい内容には触れません
今回の環境
TL;DR
１. TensorFlow 環境構築
// .devcontainer/devcontainer.json
+ // For format details, see https://aka.ms/devcontainer.json. For config options, see the
+ // README at: https://github.com/devcontainers/templates/tree/main/src/debian
+ {
+ "name": "tflite-micro",
+ // Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
+ "image": "mcr.microsoft.com/devcontainers/base:bullseye",
+
+ // Features to add to the dev container. More info: https://containers.dev/features.
+ "features": {
+ "ghcr.io/devcontainers/features/git:1": {},
+ "ghcr.io/devcontainers/features/python:1": {
+ "version": "3.11.8",
+ "installJupyterlab": true
+ }
+ },
+
+ // Use 'forwardPorts' to make a list of ports inside the container available locally.
+ // "forwardPorts": [],
+
+ // Use 'postCreateCommand' to run commands after the container is created.
+ // "postCreateCommand": "pip3 install --user -r requirements.txt",
+
+ // Configure tool-specific properties.
+ "customizations": {
+ "vscode": {
+ "extensions": [
+ "ms-toolsai.jupyter"
+ ]
+ }
+ }
+
+ // Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
+ // "remoteUser": "root"
+ }
+
２．Jupyter Notebook ファイルを入手する
* この記事執筆時点では train_hello_world_model.ipynb は tensorflow/tflite-micro #1822 で削除されているため過去のコミットから復元する
git init
Initialized empty Git repository in ../tflite-micro/.git/
git remote add origin git@github.com:tensorflow/tflite-micro.git
git fetch origin 1df426d984288894367f6aa3a914eb998494e3bd
Warning: Permanently added the ECDSA host key for IP address 'XXX.XXX.XXX.XXX' to the list of known hosts.
From github.com:tensorflow/tflite-micro
* branch 1df426d984288894367f6aa3a914eb998494e3bd -> FETCH_HEAD
git restore --source FETCH_HEAD -- tensorflow/lite/micro/examples/hello_world/train/train_hello_world_model.ipynb
ARM マシンに対応した tensorflow パッケージ (AWS の tensorflow-cpu-aws) は 2.10 以降のため最新の安定バージョンを指定する
// tensorflow/lite/micro/examples/hello_world/train/train_hello_world_model.ipynb
...
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "cr1VLfotanf6",
"outputId": "510567d6-300e-40e2-f5b8-c3520a3f3a8b"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: tensorflow==2.4.0rc0 in /usr/local/lib/python3.6/dist-packages (2.4.0rc0)\n",
"Requirement already satisfied: termcolor~=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.4.0rc0) (1.1.0)\n",
"Requirement already satisfied: gast==0.3.3 in /usr/local/lib/python3.6/dist-packages (from tensorflow==2.4.0rc0) (0.3.3)\n",
...
]
}
],
"source": [
- "! pip install tensorflow==2.4.0"
+ "! pip install tensorflow==2.15.0"
]
},
...
３．TensorFlow モデルの構築と TensorFlow Lite モデルへの変換
Run All で全て実行する
* 詳しい処理内容を知りたい人は順番に沿って実行
cat ./tensorflow/lite/micro/examples/hello_world/train/models/model.cc
unsigned char g_model[] = {
0x1c, 0x00, 0x00, 0x00, 0x54, 0x46, 0x4c, 0x33, 0x14, 0x00, 0x20, 0x00,
0x1c, 0x00, 0x18, 0x00, 0x14, 0x00, 0x10, 0x00, 0x0c, 0x00, 0x00, 0x00,
0x08, 0x00, 0x04, 0x00, 0x14, 0x00, 0x00, 0x00, 0x1c, 0x00, 0x00, 0x00,
0x98, 0x00, 0x00, 0x00, 0xc8, 0x00, 0x00, 0x00, 0x1c, 0x03, 0x00, 0x00,
0x2c, 0x03, 0x00, 0x00, 0x30, 0x09, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00,
0x01, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x60, 0xf7, 0xff, 0xff,
0x10, 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x28, 0x00, 0x00, 0x00,
0x44, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x73, 0x65, 0x72, 0x76,
0x65, 0x00, 0x00, 0x00, 0x0f, 0x00, 0x00, 0x00, 0x73, 0x65, 0x72, 0x76,
0x69, 0x6e, 0x67, 0x5f, 0x64, 0x65, 0x66, 0x61, 0x75, 0x6c, 0x74, 0x00,
0x01, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0xbc, 0xff, 0xff, 0xff,
0x09, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00,
0x64, 0x65, 0x6e, 0x73, 0x65, 0x5f, 0x34, 0x00, 0x01, 0x00, 0x00, 0x00,
0x04, 0x00, 0x00, 0x00, 0x76, 0xfd, 0xff, 0xff, 0x04, 0x00, 0x00, 0x00,
0x0d, 0x00, 0x00, 0x00, 0x64, 0x65, 0x6e, 0x73, 0x65, 0x5f, 0x32, 0x5f,
0x69, 0x6e, 0x70, 0x75, 0x74, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
0x0c, 0x00, 0x00, 0x00, 0x08, 0x00, 0x0c, 0x00, 0x08, 0x00, 0x04, 0x00,
0x08, 0x00, 0x00, 0x00, 0x0b, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
0x13, 0x00, 0x00, 0x00, 0x6d, 0x69, 0x6e, 0x5f, 0x72, 0x75, 0x6e, 0x74,
0x69, 0x6d, 0x65, 0x5f, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x00,
0x0c, 0x00, 0x00, 0x00, 0x50, 0x02, 0x00, 0x00, 0x48, 0x02, 0x00, 0x00,
0x34, 0x02, 0x00, 0x00, 0xdc, 0x01, 0x00, 0x00, 0x8c, 0x01, 0x00, 0x00,
0x6c, 0x01, 0x00, 0x00, 0x5c, 0x00, 0x00, 0x00, 0x3c, 0x00, 0x00, 0x00,
0x34, 0x00, 0x00, 0x00, 0x2c, 0x00, 0x00, 0x00, 0x24, 0x00, 0x00, 0x00,
...
0x0c, 0x00, 0x00, 0x00, 0x09, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x09
};
unsigned int g_model_len = 2488;
４．Microcontroller 環境構築
// .devcontainer/devcontainer.json
+ // For format details, see https://aka.ms/devcontainer.json. For config options, see the
+ // README at: https://github.com/devcontainers/templates/tree/main/src/debian
+ {
+ "name": "myproject",
+ // Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
+ "image": "mcr.microsoft.com/devcontainers/base:bullseye",
+ "features": {
+ "ghcr.io/devcontainers/features/git:1": {}
+ },
+
+ "workspaceFolder": "/workspaces/myproject",
+ "workspaceMount": "source=${localWorkspaceFolder},target=/workspaces/myproject,type=bind,consistency=cached",
+ "mounts": [
+ "source=/dev,target=/dev,type=bind,consistency=consistent"
+ ],
+
+ // Use 'forwardPorts' to make a list of ports inside the container available locally.
+ // "forwardPorts": [],
+
+ // Use 'postCreateCommand' to run commands after the container is created.
+ "postCreateCommand": ".devcontainer/postcreate.sh",
+
+ // Configure tool-specific properties.
+ "customizations": {
+ "vscode": {
+ "extensions": [
+ "sony-spresense.spresense-vscode-ide"
+ ]
+ }
+ },
+
+ "remoteEnv": {
+ "EDITOR": "code --wait"
+ }
+
+ // Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
+ // "remoteUser": "root"
+ }
+
# .devcontainer/postcreate.sh
+ #!/bin/sh
+ # vim:sw=4:ts=4:et
+ set -e
+
+ git config --global --add safe.directory $(pwd)
+
+ sudo usermod -a -G dialout $(whoami)
+
+ cd /tmp
+ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/sonydevworld/spresense/master/install-tools.sh)"
+
+ cat << EOF | sudo tee -a /etc/profile > /dev/null
+ source ~/spresenseenv/setup
+ EOF
+
+ sudo chown $(whoami) /workspaces
+ cd /workspaces
+
+ if [ ! -d "/workspaces/spresense" ]; then
+ git clone --depth 1 --recursive https://github.com/sonydevworld/spresense.git
+ fi
+
５. IDE 設定
コマンドパレット > Spresense: Workspace setup wizard
コマンドパレット > Spresense: Serial port selection
６. ビルド
source tools/build-env.sh
Warning: Spresense user application directory is not set.
Please run
$ spr-set-approot <application home directory>
=======================================
SDK_VERSION = SDK3.2.0
NUTTX_VERSION = 12.3.0
SPRESENSE_SDK = /workspaces/spresense
SPRESENSE_HOME =
GCC_VERSION = arm-none-eabi-gcc (GNU Arm Embedded Toolchain 10.3-2021.10) 10.3.1 20210824 (release)
HOST = Linux aarch64
=======================================
tools/config.py examples/hello
make -j
Create version.h
make[2]: Nothing to be done for `context'.
make[2]: Nothing to be done for `context'.
make[2]: Nothing to be done for `context'.
make[2]: Nothing to be done for `context'.
make[2]: Nothing to be done for `context'.
...
IN: /workspaces/spresense/sdk/apps/libapps.a -> staging/libapps.a
make[3]: `libboard.a' is up to date.
LD: nuttx
Generating: nuttx.spk
tools/cxd56/mkspk -c2 nuttx nuttx nuttx.spk;
File nuttx.spk is successfully created.
Done.
７． Spresense を接続
８． 書き込み
tools/flash.sh -c /dev/cu.usbserial-210 -b 115200 nuttx.spk
>> Install files ...
install -b 115200
Install nuttx.spk
|0%-----------------------------50%------------------------------100%|
######################################################################
159632 bytes loaded.
Package validation is OK.
Saving package to "nuttx"
updater# sync
updater# Restarting the board ...
reboot
９． 確認
