I attended a seminar today describing the use of AI models on edge devices with AI-tech Systems founder, Rohit Sharma. He described the applications of AI on some surprisingly compact edge systems in a wide variety of domains. It’s surprising because we typically think of the development and application of ML models in large, high-capacity computing environments. The difference in computing resources between the development environment and edge computing devices seems insurmountable. While model development is still primarily in the large academic and coorporate setting, the models themselves are almost exclusively open source. This presents the opportunity to take developed, effective models and optimize them for a wide variety of application.

Current Model -> Edge Pathway:

  1. Setup
  2. Aquire Data
  3. Train Model
  4. Analyze Model
  5. Develop Application
  6. Compile App
  7. Flash to Device
  8. Monitor Device

cAInvas aims to streamline steps 2-8, meaning developers really just need to bring data and a device and will be able to develop an embedded ML application entirely on the platform.

Current dev process is difficult as it requires a wide variety of specializations in both technique and scale. Rohit’s cAInvas model aims to streamline this process by providing an end-to-end platform. The service pipeline allows users to bring a model or pick an approved, precompiled model and have it cross compiled to ready to use libraries or executables suitable for use on edge devices. Models can be compiled for a wide variety of edge device boards. Model notebooks can be directly imported from Github repos and worked with in the platforms notebook editor and compiling framework.

The precompiled models include some really useful info on edge device hardware requirements to run a given deployed library or executable. The edge compiler, open source deepSea, cross compiles from Python to C++ model/library very quickly. Current targeted verticals are Industrial IoT, Smart City, and Healthcare devices. Even applicable on low-power, disconnected edge devices!–apparently the first service to compile for disconnected devices, which addresses a huge concern for the use of embedded ML devices where privacy is paramount. This service looks like it could really revolutionize the development to deployment pipeline.

Supported ML domains include Vision AI, NLP, Mobile, Biometrics, Sensor AI, and Industrial application, all of which are experiencing rapid market expansion. Requiring traditional centralized ML processing is insufficient to address this expansion due to the massive data bandwidth required to convey the growing data stream. The solution is to process the data at the edge and only convey essential data centrally via minimal connectivity. This is a sort of antithesis of to the currently dominant cloud ML services provided by larger companies. The question is, why hand off all that data and spend the bandwidth when models can be deployed at the edge?

Very cool presentation! It’s an exciting opportunity for research, development, and enterprise application. Rohit finished by demonstrating the workflow from model to deployment for an ASL to speech handheld edge device application that uses accelerometer and gyroscopic data to convert American Sign Language signs to speech output in real time, on device. Amazing. Model is developed, trained, evaluated, and validated from data sets using Pytorch on the service platform. He then compiles the model to both an Ubuntu application and C++ executable. The executable immediately works as is, producing a probability output from sample, vectorized input.

I highly recommend checking this project out!