嵌入式TensorFlow Lite C++部署推理

将上述代码用 TensorFlow Lite C++ 实现主要分为两部分：

1. 转换模型为 TensorFlow Lite 格式。
2. 使用 TensorFlow Lite C++ 加载并推理。

---

1. 将模型转换为 TensorFlow Lite 格式

在 Python 中将训练好的 Keras 模型转换为 .tflite 格式：

import tensorflow as tf

# Load your trained model
model_path = "model.h5"  # Update to your actual path
model = tf.keras.models.load_model(model_path)

# Convert to TensorFlow Lite
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()

# Save the TFLite model
tflite_model_path = "model.tflite"
with open(tflite_model_path, "wb") as f:
    f.write(tflite_model)

print(f"TensorFlow Lite model saved at {tflite_model_path}")

---

2. 使用 TensorFlow Lite C++ 加载模型并推理

2.1 TensorFlow Lite 环境准备

1. 下载 TensorFlow 源码：

   git clone https://github.com/tensorflow/tensorflow.git

2. 编译 TensorFlow Lite 静态库：

3. https://www.tensorflow.org/lite/guide/build_cmake?hl=zh-cn

   cd tensorflow
   mkdir build
   cd build
   cmake ../tensorflow/lite -DTFLITE_ENABLE_GPU=OFF -DCMAKE_BUILD_TYPE=Release
   make -j$(nproc)

   这将生成 TensorFlow Lite 的静态库和相关头文件，确保你能找到以下文件：

   • 静态库：libtensorflow-lite.a
   • 头文件：tensorflow/lite/*.h

---

2.2 编写推理代码

以下是基于 C++ 使用 TensorFlow Lite 加载并推理的代码：

#include <iostream>
#include <fstream>
#include <vector>
#include <tensorflow/lite/interpreter.h>
#include <tensorflow/lite/kernels/register.h>
#include <tensorflow/lite/model.h>
#include <tensorflow/lite/c/common.h>

// Helper function to read the model file
std::vector<char> ReadFile(const std::string& file_path) {
    std::ifstream file(file_path, std::ios::binary | std::ios::ate);
    if (!file) {
        throw std::runtime_error("Failed to open file: " + file_path);
    }
    std::streamsize size = file.tellg();
    file.seekg(0, std::ios::beg);

    std::vector<char> buffer(size);
    if (!file.read(buffer.data(), size)) {
        throw std::runtime_error("Failed to read file: " + file_path);
    }
    return buffer;
}

// Preprocess the input image (normalize pixel values)
std::vector<float> PreprocessImage(const std::vector<uint8_t>& input_image) {
    std::vector<float> normalized_image;
    normalized_image.reserve(input_image.size());
    for (uint8_t pixel : input_image) {
        normalized_image.push_back(pixel / 255.0f); // Normalize to [0, 1]
    }
    return normalized_image;
}

int main() {
    const std::string model_path = "model.tflite";
    const int input_size = 28 * 28; // MNIST image size

    // Load the TensorFlow Lite model
    std::vector<char> model_data = ReadFile(model_path);
    auto model = tflite::FlatBufferModel::BuildFromBuffer(model_data.data(), model_data.size());
    if (!model) {
        std::cerr << "Failed to load TFLite model." << std::endl;
        return 1;
    }

    // Create the interpreter
    tflite::ops::builtin::BuiltinOpResolver resolver;
    tflite::InterpreterBuilder builder(*model, resolver);
    std::unique_ptr<tflite::Interpreter> interpreter;
    if (builder(&interpreter) != kTfLiteOk) {
        std::cerr << "Failed to create interpreter." << std::endl;
        return 1;
    }

    // Allocate tensors
    if (interpreter->AllocateTensors() != kTfLiteOk) {
        std::cerr << "Failed to allocate tensors." << std::endl;
        return 1;
    }

    // Prepare input data
    float* input_data = interpreter->typed_input_tensor<float>(0);
    std::vector<uint8_t> dummy_image(input_size, 128);  // Example: all pixel values = 128
    std::vector<float> normalized_image = PreprocessImage(dummy_image);

    // Copy data to input tensor
    std::copy(normalized_image.begin(), normalized_image.end(), input_data);

    // Run inference
    if (interpreter->Invoke() != kTfLiteOk) {
        std::cerr << "Failed to invoke interpreter." << std::endl;
        return 1;
    }

    // Get output tensor and print results
    float* output_data = interpreter->typed_output_tensor<float>(0);
    int output_size = interpreter->output_tensor(0)->dims->data[1];

    int predicted_class = std::distance(output_data, std::max_element(output_data, output_data + output_size));
    std::cout << "Predicted class: " << predicted_class << std::endl;

    return 0;
}

---

2.3 编译推理代码

编译上述代码时，确保包含 TensorFlow Lite 的头文件路径和库路径。

示例编译命令：

g++ -std=c++17 -I/path/to/tensorflow/lite/ -L/path/to/tensorflow/lite/ -ltensorflow-lite \
    tflite_inference.cpp -o tflite_inference

运行推理代码：

./tflite_inference

---

总结

1. 使用 Python 将 TensorFlow 模型转换为 .tflite 格式。
2. 准备 TensorFlow Lite 的 C++ 环境，编译相关库。
3. 编写 C++ 推理代码加载 .tflite 模型，进行推理。
4. 编译运行代码，验证推理结果。

这种方法适用于嵌入式设备或性能敏感的环境。

嵌入式部署官方推荐 tflite-runtime，此方案接口为python。

https://blog.csdn.net/Y798711330/article/details/138140275

深度学习模型的流式部署 - Taylor的文章 - 知乎
https://zhuanlan.zhihu.com/p/508457006