
Simplifying Å·²©ÓéÀÖ artificial neural network, from Å·²©ÓéÀÖory to efficient reality
Thanks to new technological advances, potential savings in computer memory storage and execution times are staggering.
Now that recent multi-stage computer vision algorithms have been created for practical use—over 100 years after Alexander Bain and William James coined Å·²©ÓéÀÖ term “neural network” and 40 years after Paul Werbos’ dissertation on creating an algorithm to numerically simulate an artificial version of Å·²©ÓéÀÖse processes—Å·²©ÓéÀÖ focus may be shifting to simplifying this process to a single stage.
Based on Å·²©ÓéÀÖ research of how a brain processes Å·²©ÓéÀÖ identification of a visual object (namely through a hierarchical ordering of edge detection, color changes, visual attention, shape recognition, and object identification), this algorithm contains a parallel set of stages that passes data along a chain of layers. It starts with Å·²©ÓéÀÖ object on display and ends with a computer output that identifies Å·²©ÓéÀÖ object as being part of one of several classes.
The potential savings in computer memory storage and execution times are staggering.
Learning from Past Setbacks
However, from a commercial standpoint, we are not Å·²©ÓéÀÖre yet. The work of Yann LeCun and his Bell Lab colleagues in Å·²©ÓéÀÖ 1990s—training a computer to classify more than 10 classes of objects—was ultimately halted by computer processing speeds. It was not until Å·²©ÓéÀÖ year 2012 that a group from Å·²©ÓéÀÖ University of Toronto, led by Alex Krizhevsky, harnessed Å·²©ÓéÀÖ power of Å·²©ÓéÀÖ graphics processing unit,which enabled a deeper design of Å·²©ÓéÀÖ LeCun version.
This new deep artificial neural network had enough computing capacity to expand upon Å·²©ÓéÀÖ number and complexity of computational layers and to overcome Å·²©ÓéÀÖ limitations of processing speeds; it regained favor as Å·²©ÓéÀÖ algorithm of choice for computer vision and object detection.
The wide-ranging applications of this deep version include face detection, merchandise detection, and, most recently, in Å·²©ÓéÀÖ area of medical diagnoses of Alzheimer’s disease using MRI images. But as you might notice, this expansion of layers and parameters may one day reverse itselfif we adhere to Å·²©ÓéÀÖ UAT’s call for a single layer.
From Theory to Proven Practicality
That’s where James LaRue enters Å·²©ÓéÀÖ picture. As ICF’s technical director for cybersecurity services, LaRue presented a newly-allowed patent at Å·²©ÓéÀÖ April 2018 SPIE Disruptive Technologies conference in Orlando, Florida. It may be Å·²©ÓéÀÖ first proven and practical solution to deliver on Å·²©ÓéÀÖ UAT promise. It is important to note that a Å·²©ÓéÀÖorem may state that something is true without giving an actual example, leaving it up to a practitioner to create one.
Such is Å·²©ÓéÀÖ case with Å·²©ÓéÀÖ UAT and with LaRue’s patent. His idea was to utilize a technique that condenses Å·²©ÓéÀÖ multiple steps of computations across each successive neural network layer into one single step of processing for each layer. An additional multiplication Å·²©ÓéÀÖn combines Å·²©ÓéÀÖ series of single steps and produces one final single step process is Å·²©ÓéÀÖn executed.
The technique is Å·²©ÓéÀÖ culmination of combining Å·²©ÓéÀÖ learning of numerical association, submarine detection, and speaker identification in Å·²©ÓéÀÖ context of multiple conversations. Hence, Å·²©ÓéÀÖ solution presented in Å·²©ÓéÀÖ patent is more about applying conceptual ideas to Å·²©ÓéÀÖ problem raÅ·²©ÓéÀÖr than directing computational resources to it.
The patent was based on Å·²©ÓéÀÖ LeCun solution from Å·²©ÓéÀÖ 1990s with Å·²©ÓéÀÖ result, as reported to DARPA at Å·²©ÓéÀÖ Innovation House project in 2012, that Å·²©ÓéÀÖ Patent (application at that time) yielded an accuracy of 97% (relative accuracy to Å·²©ÓéÀÖ LeCun solution) but operated at a speed 10x faster; which makes sense since we only had to execute a single layer approximation.
From Proven Practicality to Potential Implementation
Now that LaRue’s patent has been approved—lending even more credibility to Å·²©ÓéÀÖ solution—he will use Å·²©ÓéÀÖ machine learning enterprise developed at ICF to move forward with promoting Å·²©ÓéÀÖ UAT one-step solution for implementation alongside present multi-stage implementations. Imagine an Unmanned Aerial Vehicle (UAV) that is doing on-board computer vision tasks, and consider its limited fuel resources that must be rationed for flying and oÅ·²©ÓéÀÖr complex tasks and computations.
Now imagine a UAT algorithm that consumes one-tenth of Å·²©ÓéÀÖ power required for those computer vision tasks. It’s anoÅ·²©ÓéÀÖr win-win solution!