Learning with Unstructured Data

Lecture 0: Introduction

Prof. Claudia Soares
claudia.soares@fct.unl.pt

Today

• Course outline
• Introduction to deep learning
• Fundamentals of machine learning

Outline

• Lecture 1.1: Introduction
• Lecture 1.2: Fundamentals of machine learning
• Lecture 2.1: Multi-layer perceptron
• Lecture 2.2: Training neural networks
• Lecture 3.1: Computer Vision
• Lecture 3.2: Convolutional neural networks
• Lecture 4: Attention and transformer networks
• Lecture 5: Graph Neural Networks
• Lecture 6: Natural Language Processing

Main Reference

My mission

By the end of this course, you will have acquired a solid and detailed understanding of the field of deep learning.

You will have learned how to design deep neural networks for a wide range of advanced probabilistic inference tasks and how to train them.

These models seen in the course apply to a wide variety of artificial intelligence problems, with plenty of applications in engineering and science.

Why learning?

???

.italic[How do you do that?]

class: middle

.center[ .width-70[]

Sheepdog or mop? ]

.footnote[Credits: Karen Zack, 2016.]

class: middle

.center[ .width-70[]

Chihuahua or muffin? ]

.footnote[Credits: Karen Zack. 2016.]

class: middle

The (human) brain is so good at interpreting visual information that the gap between raw data and its semantic interpretation is difficult to assess intuitively:

.center[

This is a mushroom. ]

class: middle, center

.width-70[]

This is a mushroom.

class: middle, center

.width-30[] + .width-30[] + .width-30[]

This is a mushroom.

class: middle, center

.width-80[]

This is a mushroom.

class: middle, center

Writing a computer program that sees?

class: middle

.center.width-60[]

count: false class: black-slide

.center.width-60[]

count: false class: black-slide, middle

.center.width-80[]

count: false class: black-slide, middle

.center.width-80[]

class: middle

Extracting semantic information requires models of high complexity, which cannot be designed by hand.

However, one can write a program that learns the task of extracting semantic information.

class: middle, black-slide

.center.width-70[]

The common approach used in practice consists of: - defining a parametric model with high capacity, - optimizing its parameters, by “making it work” on the training data.

class: middle, center, black-slide

class: middle, black-slide

.center[ ]

class: middle

Applications and successes

class: middle, center, black-slide

Object detection, pose estimation, segmentation (2019)

class: middle, center, black-slide

Reinforcement learning (Mnih et al, 2014)

exclude: true class: middle, center, black-slide

Strategy games (Deepmind, 2016-2018)

class: middle, center, black-slide

Autonomous cars (NVIDIA, 2016)

class: middle, center, black-slide

Autonomous cars (Waymo, 2022)

exclude: true class: middle, black-slide

.center[

Physics simulation (Sanchez-Gonzalez et al, 2020)

]

class: middle, black-slide, center

AI for Science (Deepmind, AlphaFold, 2020)

class: middle, center, black-slide

Speech synthesis and question answering (Google, 2018)

class: middle, center, black-slide

Image generation and AI art (OpenAI, 2022)

class: middle, center, black-slide

Write computer code (OpenAI, 2021)

class: middle, center, black-slide

.center.width-100[]

Answer all your questions (OpenAI, 2022)

exclude: true class: middle, center, black-slide

Dali Lives (2019)

class: middle, center

.width-70[]

.italic[“ACM named .bold[Yoshua Bengio], .bold[Geoffrey Hinton], and .bold[Yann LeCun] recipients of the .bold[2018 ACM A.M. Turing Award] for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing.”]

Why does it work now?

.center.grid[ .kol-1-2[ Algorithms (old and new)

.width-90[]] .center.kol-1-2[ More data

.width-50[]]]

.center.grid[ .kol-1-2[ Software
.width-90[]] .kol-1-2[ Faster compute engines

.width-50[]]]

???

The success of deep learning is multi-factorial…

class: middle

Building on the shoulders of giants

Five decades of research in machine learning provided - a taxonomy of ML concepts (classification, generative models, clustering, kernels, linear embeddings, etc.), - a sound statistical formalization (Bayesian estimation, PAC), - a clear picture of fundamental issues (bias/variance dilemma, VC dimension, generalization bounds, etc.), - a good understanding of optimization issues, - efficient large-scale algorithms.

.footnote[Credits: Francois Fleuret, EE559 Deep Learning, EPFL.]

class: middle

Deep learning

From a practical perspective, deep learning - lessens the need for a deep mathematical grasp, - makes the design of large learning architectures a system/software development task, - allows to leverage modern hardware (clusters of GPUs), - does not plateau when using more data, - makes large trained networks a commodity.

.footnote[Credits: Francois Fleuret, EE559 Deep Learning, EPFL.]

class: middle

.center.circle.width-30[]

.italic[For the last forty years we have programmed computers; for the next forty years we will train them.]

.pull-right[Chris Bishop, 2020.]

class: end-slide, center count: false

The end.