Now we'll see how to learn to learn in the concept space using deep meta learning. First, how do we perform meta learning? We sample a batch of related tasks and some k data points in each task and train our meta learner. Instead of just training using our vanilla meta learning technique, we can combine the power of deep learning with meta learning. So, when we sample a batch of tasks and some k data points in each task, we learn the representations of each of the k data points using deep neural networks and then we'll perform meta learning on those representations.
Our framework consists of three components:
- Concept generator
- Concept discriminator
- Meta learner
The role of the concept generator is to extract the feature representations of each of the data points in our dataset, capturing its high-level concept, and the role of the concept discriminator is to recognize and classify the concepts generated by the concept generator, while the meta learner learns on the concepts generated by the concept generator. All of the previous components—that is, the concept generator, concept discriminator, and meta learner—learn together. So, we improve the vanilla meta learning by integrating meta learning with deep learning. Our concept generator evolves with new incoming data so we can view our framework as a lifelong learning system.
But what's really going on here? Look at the following diagram; as you can see, we sample a set of tasks and feed them to our concept generator, which learns the concepts—that is, embeddings—and then feeds those concepts to the meta learner, which learns on these concepts and sends the loss back to the concept generator. Meanwhile, we also feed some external dataset to the concept generator, which learns the concepts for those inputs and sends those concepts to the concept discriminator. The concept discriminator predicts the labels for those concepts, calculates the loss, and sends the loss back to the concept generator. By doing so, we enhance our concept generator's ability to generalize concepts:
But still, why are we doing this? Instead of performing meta learning on a raw dataset, we perform meta learning in the concept space. How do we learn these concepts? These concepts are generated by the concept generator by learning the embeddings of the input. So, we train the concept generator and meta learner on various related tasks; along with this, we improve the concept generator through the concept discriminator by feeding an external dataset to the concept generator so that it can learn the concepts better. This joint training process allows our concept generator to learn various concepts and perform better on related tasks; we feed the external dataset only to enhance the performance of our concept generator, which learns continuously when we feed a new set of inputs. So, it's a lifelong learning system.