Key Differences Between Transfer Learning and Incremental Learning
BrainChip offers insight into two widely accepted forms of deep learning
Aliso Viejo, Calif. – August 24, 2021 – The massive computing resources required to train neural networks for AI/ML tasks has driven interest in two forms of learning presumed to be more efficient: transfer learning and incremental learning. Experts at BrainChip Holdings Ltd (ASX: BRN), (OTCQX: BRCHF), a leading provider of ultra-low power high performance artificial intelligence technology, offered the following insight and considerations for their use in edge AI/IoT environments.
In transfer learning, applicable knowledge established in a previously trained AI model is “imported” and used as the basis of a new model. After taking this shortcut of using a pretrained model, such as an open-source image or NLP dataset, new objects can be added to customize the result for the particular scenario.
The primary downfall of this system is accuracy. Fine-tuning the pretrained model requires large amounts of task-specific data to add new weights or data points. As it requires working with layers in the pretrained model to get to where it has value for creating the new model, it may also require more specialized, machine-learning savvy skills, tools, and service vendors.
When used for edge AI applications, transfer learning involves sending data to the cloud for retraining, incurring privacy and security risks. Once a new model is trained, any time there is new information to learn, the entire training process needs to be repeated. This is a frequent challenge in edge AI, where devices must constantly adapt to changes in the field.
“First and foremost is the issue of there being an available model that you can make work for your application, which is not likely for anything but very basic AI, and then you need enough samples to retrain it properly,” said Anil Mankar, Co-founder, Chief Development Officer at BrainChip. “Since this requires going to the cloud for retraining and then back down to the device, transfer learning is still a very complex and costly process, though it’s a nice option when and where it’s possible to use it.”
Incremental learning is another form that is often used to reduce the resources used to train models because of its efficiency and ability to accommodate new and changed data inputs. An edge device that can perform incremental learning within the device itself, rather than send data to the cloud, can learn continuously.
Incremental or “one-shot” learning can begin with a very small set of samples, and grow its knowledge as more data is absorbed. The ability to evolve based on more data also results in higher accuracy. When retraining is done on the device’s hardware, instead of cloud retraining, the data and application remains private and secure.
“Most of the time, AI projects don’t have large enough data sets in the beginning, and don’t have access to cloud computing for retraining, so they keep paying their vendor whenever anything changes,” said Mankar. “We generally recommend incremental learning because it addresses most of the shortcomings of transfer learning and requires dramatically lower computing costs.”
BrainChip’s Akida brings artificial intelligence to the edge in a way that existing technologies are not capable. The solution is high-performance, small, ultra-low power and enables a wide array of edge capabilities. The Akida (NSoC) and intellectual property, can be used in applications including Smart Home, Smart Health, Smart City and Smart Transportation. These applications include but are not limited to home automation and remote controls, industrial IoT, robotics, security cameras, sensors, unmanned aircraft, autonomous vehicles, medical instruments, object detection, sound detection, odor and taste detection, gesture control and cybersecurity.
About BrainChip Holdings Ltd (ASX: BRN, OTCQX: BRCHF)
BrainChip is a global technology company that is producing a groundbreaking neuromorphic processor that brings artificial intelligence to the edge in a way that is beyond the capabilities of other products. The chip is high performance, small, ultra-low power and enables a wide array of edge capabilities that include on-chip training, learning and inference. The event-based neural network processor is inspired by the spiking nature of the human brain and is implemented in an industry standard digital process. By mimicking brain processing BrainChip has pioneered a processing architecture, called Akida™, which is both scalable and flexible to address the requirements in edge devices. At the edge, sensor inputs are analyzed at the point of acquisition rather than through transmission via the cloud to a data center. Akida is designed to provide a complete ultra-low power and fast AI Edge Network for vision, audio, olfactory and smart transducer applications. The reduction in system latency provides faster response and a more power efficient system that can reduce the large carbon footprint of data centers.
Additional information is available at https://www.brainchipinc.com
Follow BrainChip on Twitter: https://www.twitter.com/BrainChip_inc
Follow BrainChip on LinkedIn: https://www.linkedin.com/company/7792006
