Transfer learning in layman’s terms

Often, machine learning happens on an isolated basis. Algorithms are trained task by task, dataset by dataset. Traditionally, then, machines have been unable to apply any past learned knowledge to new challenges.

This is where a technique called ‘transfer learning’ comes in. Transfer learning allows machines to repurpose their past training when working on new tasks and behaviours.

But what does that mean, exactly? Here, we explain transfer learning in layman’s terms – without all the complex dives into the inner workings of AI.

What is transfer learning?

When humans learn, they don’t only use the knowledge for one task. Rather, they can apply and leverage their experience to solve new problems and support future learning.

Take, for example, someone that has learned to play classical guitar. They will have the fundamental knowledge of how to play the guitar, which they can transfer and apply when learning to play rock guitar. This accelerates their learning, leading to faster success.

Transfer learning seeks to replicate this ability to apply learned knowledge to new problems.

For instance, a machine has learned to identify pictures of dogs. Traditionally, teaching the machine a new but similar ability means starting from scratch. (I.e. recognising cats.) With transfer learning, the machine can take some of what it’s learned when identifying dogs into account and apply it to the new task.

Using transfer learning

In transfer learning, you have a source model trained on a specific dataset. This contains the knowledge a machine gained when it learned to complete a task.

Then, there’s the new task (or target) that you want the machine learning algorithm to address. The second task must relate to the first in some way for transfer learning to be useful.

Next, you need to determine what to transfer. This means identifying the data and knowledge that’s common between the source model and the new task. This could be a set of parameters, certain instances or feature-representations. Or, it might be knowledge from data that relates to other datapoints.

This transferred knowledge, combined with new training data, allows machines to build on past success.

Why use transfer learning?

Progress feeds more progress. Using previously trained models as a jump start to learning new things means that you’re getting more out of each task the machine learns. You’re optimising the data, time and effort involved in machine learning.

Transfer learning means you’re not starting from scratch – thereby speeding up training time. It can fill the gap left if you don’t have enough training data for the new task.  Plus, it potentially leads to better, higher-quality results and output. After all, the machine has a bigger bank of domain knowledge to inform its output.

Beyond the observable benefits, perfecting transfer learning techniques could bring us closer to artificial general intelligence (AGI). It upskills machine learning even while making it more scalable, greasing the wheel for the ‘strong’ AI currently beyond our reach.

The caveat

Before using transfer learning, it’s important to analyse whether it will provide benefit. Sometimes, transferring already learned knowledge brings no benefit to the new task. This is particularly true when the new ability you want the machine to learn isn’t related to the source task.

There’s also such a thing as negative transfer. For example, driving on right-sided vehicles can initially throw you off when adapting to left-sided vehicles.

So, it’s important to be careful about when and what to transfer. (Rather than transfer learning for the sake of transfer learning.)

Building more intelligent systems

Learning is an inherent ability of many creatures in the world. Machine learning is our attempt to replicate this ability in computers. And with transfer learning, we recreate a human’s ability to apply prior knowledge to new problems. In so doing, we continue our journey towards more intelligent systems.

Useful links

What is machine learning? A beginner’s guide

Types of AI: distinguishing between weak, strong, and super AI

An easy overview of reinforcement learning in AI