AForge.NET is a C# framework designed for developers and researchers in the fields of computer vision and artificial intelligence, which include image processing, neural networks, genetic algorithms, machine learning, robotics, etc. This article gives a quick introduction to this powerful framework.
Computer vision (CV) is the capability of a machine to get and analyse visual information. It comprises videos and photos, and consists of pictures from varied thermal cameras, sensors, and other different sources. CV is a branch of computer science, and is termed as a process that involves capturing, processing and analysing real world images to allow machines to extract meaningful, contextual information from the physical world. Nowadays, computer vision is the foundation and a key means of testing and exploiting deep learning models and using other artificial intelligence techniques for implementing useful and practical applications.
Since 2012, the concepts of ‘context, attention and intention’ are developing computer vision into a new sub-discipline of AI. Computer vision helps robots, autonomous vehicles, and everything from smart agriculture to drones to become more productive. The market for computer vision is developing nearly as fast as the capacities are increasing. It is estimated that the global computer vision market can reach US$ 26.2 billion by 2025. Computer vision adds a natural way for users to interact with the digital and physical world around them. It is also enabling new ways for devices to interact with users and the environment around them.
Currently, lots of commercial and open source tools are available for developers and industry R&D professionals to design applications based on computer vision. In this article, we primarily focus on the AForge.NET library.
An introduction to AForge.NET library
AForge.NET is a powerful AI and computer vision library designed by Andrew Kirillov for the .NET framework. The framework, at present, consists of many parts—image processing, neural networks, evolution algorithms, fuzzy logic, machine learning, robotics, etc.
The framework comprises library sets and some sample applications that are listed below.
- AForge.Imaging: An image processing routines and filters library
- AForge.Vision: A computer vision library
- AForge.Video: A video processing library
- AForge.Neuro: A neural networks computation library
- AForge.Genetic: An evolutionary biology programming library
- AForge.Fuzzy: A fuzzy computations library
- AForge.Robotics: A robotics kits library
- AForge.MachineLearning: A machine learning library
Let’s briefly discuss each of these libraries.
AForge.Imaging
The image processing library contains a set of image processing filters and tools designed to address many different tasks of computer vision and image analysis/processing.
The library consists of the following filters:
- Colour filters (grayscale, sepia, invert, rotate channels, channel extraction, channel replacing, channel filtering, colour filtering, Euclidean colour filtering, RGB channels, linear correction, etc)
- HSL filters (linear correction, brightness, contrast, saturation, hue modifier and HSL filtering)
- YCbCr filters (linear correction, YCbCr filtering and channel extraction/replacement)
- Binarisation filters (threshold, threshold with carry, ordered dithering, Bayer dithering, Floyd-Steinberg, Burkes, Jarvis-Judice-Ninke, Sierra, Stevenson-Arce and Stucki dithering methods)
- Adaptive binarisation (simple image statistics)
- Mathematical morphology filters (erosion, dilatation, opening, closing, hit and miss, thinning and thickening)
- Convolution filters (mean, blur, sharpen, edges, Gaussian blur, sharpening based on Gaussian kernel, etc)
- Two source filters (merge, intersect, add, subtract, difference, move towards and morph)
- Edge detectors (homogeneity, difference, Sobel and canny)
- Gamma correction, median filter
- Conservative smoothing, jitter, oil painting, pixellate and simple skeletonisation
- Blob counter and connected components labelling filter
- Texture generators (clouds, marble, wood, labyrinth and textile), texturer, textured filter and texture merge filter
- Resize and rotation (nearest neighbour, bilinear and bicubic)
- Frequency filtering with FFT
- Image statistics
- Flat Field Illumination correction, simple skeletonisation, shrink, canvas crop/fill/move, mirroring, Bayer filter and mask/masked filter
- Fourier transformation (low-pass and high-pass filters)
AForge.Vision
This library consists of different motion detection and motion processing algorithms. The former are primarily targeted to detect motion in video frames by recording the amount of detected motion, the motion frame-binary image, and also the regions where motion is detected. Motion processing algorithms are designed to perform post-processing of detected motion—to highlight motion regions, count moving objects and track objects.
Motion detection algorithms: The AForge.NET library comprises various motion detection algorithms like the two-frame difference motion detector, the simple background modelling motion detector, and the custom frame difference motion detector.
Motion processing algorithms: The AForge.NET library consists of various motion processing algorithms—motion area highlighting, motion border highlighting, grid motion area processing, and blob counting objects processing.
AForge.Math
This library comprises various mathematics related algorithms like:
- 3D pose estimation using POSIT and Coplanar POSIT algorithms
- Simple shape checker
- Graham convex hull
AForge.Video
This library comprises various classes to process all sorts of video data. It has the following components:
- Access to JPEG and MJPEG streams – IP cameras
- Access to USB Web cameras – captures video from the DirectShow Interface
- Reading/writing of AVI files
- Reading/writing video files using the FFmpeg library
- Microsoft Kinect sensor support
- XIMEA camera support
AForge.Robotics
The following robotics kits/boards are supported by this library:
- Qwerk robotics board
- Lego Mindstorm RCX robotics kit
- Lego Mindstorm NXT robotics kit
- Surveyor SRV-1 Blackfin robot
- Surveyor Stereo Vision Systems robotics board
AForge.Neuro
This library consists of neural network architectures along with varied learning algorithms, such as:
- Multi-layer feed forward networks
- Distance networks
- Simple perceptron’s learning, Delta rule learning, back propagation learning, Kohonen SOM learning,
- evolutionary learning based on genetic algorithms
- Threshold, Sigmoid, Bipolar Sigmoid based activation functions
AForge.Genetic
This library makes use of various genetic algorithms, genetic programming and gene expression areas like:
- GA chromosomes (binary, short array and double array), GP tree-based chromosome and GEP chromosome
- Selection algorithms (elite, roulette wheel and rank)
- Common fitness functions (1/2D function optimisation, symbolic regression and time series prediction)
- Population class to handle chromosomes
AForge.Fuzzy
This library has been primarily designed to perform fuzzy computations by using basic fuzzy sets and linguistic variables with a complete inference system.
AForge.MachineLearning
This library supports various machine learning based algorithms like:
- QLearning and Sarsa learning
- Epsilon Greedy, Boltzmann, Roulette wheel and Tabu Search exploration
The AForge.NET framework currently consists of the following two tools:
- Image processing prototypers
- Debugging visualisers
Image processing prototypers
These were primarily designed to process image processing and computer vision algorithms faster, focusing on algorithms only. They provide a generic user interface to help developers test algorithms on image data sets and inspect the results.
Debugging visualisers
The AForge.NET framework consists of a few visualiser components, which are used for debugging image processing and computer vision algorithms. The following are the visualisers available currently:
- System.Drawing.Image
- System.Drawing.Imaging.BitmapData
- AForge.Imaging.UnmanagedImage
- Aforge.Math.Histogram
