OpenCV on VxWorks enables intelligence at the edge

When a wind turbine collects sensor data about its surrounding environment, or an off-shore oil rig logs its readings into its records, the data can be analyzed for decision-making factors: Are the sudden changes in engine noise an indication of wear?  Are there any flying objects headed towards danger zones? Are the engine settings optimal for the current weather pattern?

The vast amount of data to be processed to reach such decisions would flood the data pipes, and the delay in round-trip communication can mean that the control command is obsolete by the time it reaches its intended target.  By pushing intelligence to the edge, allowing edge devices to analyze the data and filter outgoing information, a networked system can relieve the load on its network and generate tighter control loops.

VxWorks, the safe and secure real-time operating system found in many controller and edge devices, enables data analysis through its support of the OpenCV libraries.  OpenCV is an open source library specifically geared towards computer vision.  On VxWorks, the libraries are based on C++.

OpenCV comes with several computer vision modules.  The object detection module provides a particularly visual demonstration of OpenCV on VxWorks, so I’ll use this module as an example.  This module uses a Haar Feature-based Cascade Classifier that identifies objects through a series of edge and line patterns.  The coolest aspect of Cascade Classifier is that I can feed the module with sample images and train the module to identify target objects.  The result of the training is a trained cascade XML file that can be used across various platforms.

Figure 1.  OpenCV cascade training and object detection process.

Sample face recognition trained cascade XML files are readily available to test this functionality.  OpenCV on VxWorks easily identifies various faces picked up by the USB video camera.  A fun project was to collect images of a hand, and run through the OpenCV Cascade Training Tool to train the module to identify an open palm.  Even more interesting were the suspicious looks I got from fellow co-workers when I circulated with a camera asking to take pictures of their hands.  For the user-interactive activities of the training process, I used the OpenCV cascade training tools on a Linux desktop to produce the trained cascade XML file for the open palm.

At runtime, the VxWorks application opens the XML file and feeds it to the OpenCV module to identify hands.  OpenCV version 3.3 includes a detection-based tracker, which uses a combination of a detection algorithm combined with a more efficient tracking algorithm.  As seen in the sample code below, the trained cascade XML file is used to create a detector and a tracker, and combined into a detection-based tracker with default parameters.  The detector’s methods process() and getObjects() give the coordinates of the detected objects.

vector<Rect> Hands;  cv::Ptr<cv::CascadeClassifier> Cascade;  cv::Ptr<DetectionBasedTracker::IDetector> MainDetector;  cv::Ptr<DetectionBasedTracker::IDetector> Tracker;  std::string cascadeHandFilename = "cascade-palm.xml";   /* Create detectors by reading the trained cascade XML file. */  Cascade = makePtr<cv::CascadeClassifier>(cascadeHandFilename);  MainDetector = makePtr<CascadeHandDetectorAdapter>( Cascade);  Cascade = makePtr<cv::CascadeClassifier>(cascadeHandFilename);  Tracker = makePtr<CascadeHandDetectorAdapter>( Cascade);   DetectionBasedTracker::Parameters params;  Detector(MainDetector, Tracker, params);   /* After obtaining the grayscale image in GrayFrame,      execute the object detection algorithm. */  Detector.process(GrayFrame);  Detector.getObjects(Hands);   /* Hands is now a vector of Rect objects indicating the locations     of the detected hands. */

If you have an installation of VxWorks, you can see the full source code in your VxWorks Source Build (VSB) project under 3pp/OPENCV/opencv-3.3.1/vxworks_examples.

Figure 2:  Using OpenCV library, VxWorks identifies the palm of a hand from USB camera input.

While OpenCV has a heavy focus on computer vision, it also contains some matrix libraries and Fourrier Transform libraries that can boost performance in generic applications with heavy computation.  These libraries take advantage of multi-processor capabilities.

Wind River continues to invest in VxWorks platforms to enable safe and secure devices that integrate with Internet of Things (IoT) technologies like OpenCV.  You can find out more information about VxWorks here.