Bigger is Better?
Huge models and transformers change how we use AI.
But they shouldn’t change how we study intelligence both human,and artificial.
// Generalization is a bare necessity of life. No animal (humans included) can thrive without it. While partly inspired by our cognitive system, today’s AI is still task-biased.
Hierarchical representations and efficient computations are the matchmakers of nature and nurture.
With that in mind we cannot see a way for AI to achieve such abilities by choosing between rule-based and end-to-end learning. Instead, we strive to find the unified approach that combines their strengths.
// We explore the use of inductive biases within deep learning architectures to continue the evolution of AI from models that learn how to solve a specific task (“learning”), to a model-hub that learns how to represent the world through language (“acquisition”).
To achieve this, we develop fundamental building blocks, each with its own inductive bias, that together form a hub - a platform that helps to manipulate the blocks and the relations between them, allowing the freedom and flexibility to model and form almost anything.