Hardware accelerated Function-as-a-Service (FaaS) enables cloud developers to deploy inference functionalities  on Intel® IoT edge devices with accelerators (CPU, Integrated GPU, Intel® FPGA, and Intel® Movidius™ technology). These functions provide a great developer experience and seamless migration of visual analytics from cloud to edge in a secure manner using a containerized environment. Hardware-accelerated FaaS provides the best-in-class performance by accessing optimized deep learning libraries on Intel® IoT edge devices with accelerators.
This tutorial demonstrates how to:
- Set up the Intel® edge device with Clear Linux* OS
- Install the OpenVINO™ toolkit and Amazon Web Services* (AWS*) Greengrass* software stacks
- Use AWS Greengrass* and AWS Lambda* to deploy the FaaS samples from the cloud
Refer to the following topics:
- Operating System: Clear Linux OS latest release
- Hardware: Intel® core platforms (This tutorial supports inference on CPU only.)
The AWS Greengrass samples are located at Edge-Analytics-FaaS. This tutorial uses the 1.0 version of the source code.
Clear Linux OS provides the following AWS Greengrass samples:
This AWS Greengrass sample classifies a video stream using classification networks such as AlexNet and GoogLeNet and publishes top-10 results on AWS* IoT Cloud every second.
This AWS Greengrass sample detects objects in a video stream and classifies them using single-shot multi-box detection (SSD) networks such as SSD Squeezenet, SSD Mobilenet, and SSD300. This sample publishes detection outputs such as class label, class confidence, and bounding box coordinates on AWS IoT Cloud every second.
Create user accounts
After Clear Linux OS is installed, create two user accounts. Create an administrative user in Clear Linux OS and create a user account for the Greengrass services to use ( see Greengrass user below).
Create a new user and set a password for that user. Enter the following commands as root:
useradd <userid> passwd <userid>
Next, enable the sudo command for your new <userid>. Add <userid> to the wheel group:
usermod -G wheel -a <userid>
Create a /etc/fstab file.
By default, Clear Linux OS does not create an /etc/fstab file. You must create this file before the Greengrass service runs.
Add required bundles
Use the swupd software updater utility to add the prerequisite bundles for the OpenVINO software stack:
swupd bundle-add os-clr-on-clr desktop-autostart computer-vision-basic
Learn more about how to swupd.
The computer-vision-basic bundle installs the OpenVINO™ toolkit, and the sample models optimized for Intel® edge platforms.
Convert deep learning models
Locate sample models
There are two types of provided models that can be used in conjunction with AWS Greengrass for this tutorial: classification or object detection.
To complete this tutorial using an image classification model, download the BVLC AlexNet model files bvlc_alexnet.caffemodel and deploy.prototxt to the default model_location at /usr/share/openvino/models. Any custom pre-trained classification models can be used with the classification sample.
For object detection, the sample models optimized for Intel® edge platforms are included with the computer-vision-basic bundle installation at /usr/share/openvino/models. These models are provided as an example; however, you may also use a custom SSD model with the Greengrass object detection sample.
Run model optimizer
Follow these instructions for converting deep learning models to Intermediate Representation using Model Optimizer. To optimize either of the sample models described above, run one of the following commands.
For classification using BVLC AlexNet model:
python3 mo.py --framework caffe --input_model <model_location>/bvlc_alexnet.caffemodel --input_proto <model_location>/deploy.prototxt --data_type <data_type> --output_dir <output_dir> --input_shape [1,3,227,227]
For object detection using SqueezeNetSSD-5Class model:
python3 mo.py --framework caffe --input_model <model_location>/'SqueezeNet 5-Class detection'/SqueezeNetSSD-5Class.caffemodel --input_proto <model_location>/'SqueezeNet 5-Class detection'/SqueezeNetSSD-5Class.prototxt --data_type <data_type> --output_dir <output_dir>
In these examples:
- <model_location> is /usr/share/openvino/models.
- <data_type> is FP32 or FP16, depending on target device.
- <output_dir> is the directory where the Intermediate Representation (IR) is stored. IR contains .xml format corresponding to the network structure and .bin format corresponding to weights. This .xml file should be passed to <PARAM_MODEL_XML>.
- In the BVLC AlexNet model, the prototxt defines the input shape with batch size 10 by default. In order to use any other batch size, the entire input shape must be provided as an argument to the model optimizer. For example, to use batch size 1, you must provide: --input_shape [1,3,227,227]
For each Intel® edge platform, you must create a new AWS Greengrass group and install AWS Greengrass core software to establish the connection between cloud and edge.
To create an AWS Greengrass group, follow the instructions in Configure AWS IoT Greengrass on AWS IoT.
To install and configure AWS Greengrass core on edge platform, follow the instructions in Start AWS Greengrass on the Core Device. In step 8(b), download the x86_64 Ubuntu* configuration of the AWS Greengrass core software.
You do not need to run the cgroupfs-mount.sh script in step #6 of Module 1 of the AWS Greengrass developer guide because this is enabled already in Clear Linux OS.
Be sure to download both the security resources and the AWS Greengrass core software.
Security certificates are linked to your AWS account.
Complete steps 1-4 of the AWS Greengrass tutorial at Create and Package a Lambda Function.
This creates the tarball needed to create the AWS Greengrass environment on the edge device.
In step 5, replace greengrassHelloWorld.py with the classification or object detection Greengrass sample from Edge-Analytics-Faas:
Zip the selected Greengrass sample with the extracted Greengrass SDK folders from the previous step into greengrass_sample_python_lambda.zip.
The zip should contain:
- greengrass classification or object detection sample
zip -r greengrass_lambda.zip greengrasssdk greengrass_object_detection_sample_ssd.py
Return to the AWS documentation section called Create and Package a Lambda Function and complete the procedure.
In step 9(a) of the AWS documentation, while uploading the zip file, make sure to name the handler to one of the following, depending on the AWS Greengrass sample you are using:
After creating the Greengrass group and the Lambda function, start configuring the Lambda function for AWS Greengrass.
Follow steps 1-8 in Configure the Lambda Function for AWS IoT Greengrass in the AWS documentation.
In addition to the details mentioned in step 8, change the Memory limit to 2048 MB to accommodate large input video streams.
Add the following environment variables as key-value pairs when editing the Lambda configuration and click on update:
Table 1. Environment variables: Lambda configuration Key Value PARAM_MODEL_XML <MODEL_DIR>/<IR.xml>, where <MODEL_DIR> is user specified and contains IR.xml, the Intermediate Representation file from Intel® Model Optimizer. For this tutorial, <MODEL_DIR> should be set to ‘/usr/share/openvino/models’ or one of its subdirectories. PARAM_INPUT_SOURCE
- <DATA_DIR>/input.webm to be specified by user. Holds both input and
- output data. For webcam, set PARAM_INPUT_SOURCE to ‘/dev/video0’
PARAM_DEVICE “CPU” PARAM_CPU_EXTENSION_PATH /usr/lib64/libcpu_extension.so PARAM_OUTPUT_DIRECTORY <DATA_DIR> to be specified by user. Holds both input and output data PARAM_NUM_TOP_RESULTS User specified for classification sample. (e.g. 1 for top-1 result, 5 for top-5 results)
Add subscription to subscribe, or publish messages from AWS Greengrass Lambda function by completing the procedure in Configure the Lambda Function for AWS IoT Greengrass.
The optional topic filter field is the topic mentioned inside the Lambda function. In this tutorial, sample topics include the following: openvino/ssd or openvino/classification
Add local resources
Refer to the AWS documentation for details about local resources and access privileges.
The following table describes the local resources needed for the CPU:
|Name||Resource type||Local path||Access|
|ModelDir||Volume||<MODEL_DIR> to be specified by user||Read-Only|
|DataDir||Volume||<DATA_DIR> to be specified by user. Holds both input and output data.||Read and Write|
Refer to the AWS documentation for instructions on how to deploy the lambda function to AWS Greengrass core device. Select Deployments on the group page and follow the instructions.
There are four options available for output consumption. These options are used to report, stream, upload, or store inference output at an interval defined by the variable reporting_interval in the AWS Greengrass samples.
IoT cloud output:
This option is enabled by default in the AWS Greengrass samples using the enable_iot_cloud_output variable. You can use it to verify the lambda running on the edge device. It enables publishing messages to IoT cloud using the subscription topic specified in the lambda. (For example, topics may include openvino/classification for classification and openvino/ssd for object detection samples.) For classification, top-1 result with class label are published to IoT cloud. For SSD object detection, detection results such as bounding box coordinates of objects, class label, and class confidence are published.
Follow the instructions here to view the output on IoT cloud.
This option enables inference output to be streamed from the edge device to cloud using Kinesis  streams when enable_kinesis_output is set to True. The edge devices act as data producers and continually push processed data to the cloud. You must set up and specify Kinesis stream name, Kinesis shard, and AWS region in the AWS Greengrass samples.
Cloud storage using AWS S3 bucket:
When the enable_s3_jpeg_output variable is set to True, it enables uploading and storing processed frames (in jpeg format) in an AWS S3 bucket. You must set up and specify the S3 bucket name in the AWS Greengrass samples to store the JPEG images. The images are named using the timestamp and uploaded to S3.
When the enable_s3_jpeg_output variable is set to True, it enables storing processed frames (in jpeg format) on the edge device. The images are named using the timestamp and stored in a directory specified by PARAM_OUTPUT_DIRECTORY.