Claudia Soares

Claudia Soares

I use optimization, physics, and probability theory to develop robust, interpretable, and trustworthy learning methods to be applied in the real world.

% Lecture 0: Introduction % Author CS % Date March

Learning with Unstructured Data

Lecture 0: Introduction



Prof. Claudia Soares
claudia.soares@fct.unl.pt

Today

  • Course outline
  • Introduction to deep learning
  • Fundamentals of machine learning

Outline

  • Lecture 1.1: Introduction
  • Lecture 1.2: Fundamentals of machine learning
  • Lecture 2.1: Multi-layer perceptron
  • Lecture 2.2: Training neural networks
  • Lecture 3.1: Computer Vision
  • Lecture 3.2: Convolutional neural networks
  • Lecture 4: Attention and transformer networks
  • Lecture 5: Graph Neural Networks
  • Lecture 6: Natural Language Processing

Main Reference

:::::::::::::: {.columns} ::: {.column width=”40%”}

  • Good balance of theory and practice
  • Covers almost all topics; Will be added more references, as adequate
  • Online book dl2.ai

::: ::: {.column width=”60%”}

::: ::::::::::::::

My mission

By the end of this course, you will have acquired a solid and detailed understanding of the field of deep learning.

You will have learned how to design deep neural networks for a wide range of advanced probabilistic inference tasks and how to train them.

These models seen in the course apply to a wide variety of artificial intelligence problems, with plenty of applications in engineering and science.

Why learning?

What do you see?

???

.italic[How do you do that?]

class: middle

.center[ .width-70[]

Sheepdog or mop? ]

.footnote[Credits: Karen Zack, 2016.]

class: middle

.center[ .width-70[]

Chihuahua or muffin? ]

.footnote[Credits: Karen Zack. 2016.]

class: middle

The (human) brain is so good at interpreting visual information that the gap between raw data and its semantic interpretation is difficult to assess intuitively:


.center[

This is a mushroom. ]

class: middle, center

.width-70[]

This is a mushroom.

class: middle, center

.width-30[] + .width-30[] + .width-30[]

This is a mushroom.

class: middle, center

.width-80[]

This is a mushroom.

class: middle, center

Writing a computer program that sees?

class: middle

.center.width-60[]

count: false class: black-slide

.center.width-60[]

count: false class: black-slide, middle

.center.width-80[]

count: false class: black-slide, middle

.center.width-80[]

class: middle

Extracting semantic information requires models of high complexity, which cannot be designed by hand.

However, one can write a program that learns the task of extracting semantic information.

class: middle, black-slide

.center.width-70[]

The common approach used in practice consists of:

  • defining a parametric model with high capacity,
  • optimizing its parameters, by “making it work” on the training data.

class: middle, center, black-slide

class: middle, black-slide

.center[

]

class: middle

Applications and successes

class: middle, center, black-slide

Object detection, pose estimation, segmentation (2019)

class: middle, center, black-slide

Reinforcement learning (Mnih et al, 2014)

exclude: true class: middle, center, black-slide

Strategy games (Deepmind, 2016-2018)

class: middle, center, black-slide

Autonomous cars (NVIDIA, 2016)

class: middle, center, black-slide

Autonomous cars (Waymo, 2022)

exclude: true class: middle, black-slide

.center[

Physics simulation (Sanchez-Gonzalez et al, 2020)

]

class: middle, black-slide, center

AI for Science (Deepmind, AlphaFold, 2020)

class: middle, center, black-slide

Speech synthesis and question answering (Google, 2018)

class: middle, center, black-slide

Image generation and AI art (OpenAI, 2022)

class: middle, center, black-slide

Write computer code (OpenAI, 2021)

class: middle, center, black-slide

.center.width-100[]

Answer all your questions (OpenAI, 2022)

exclude: true class: middle, center, black-slide

Dali Lives (2019)

class: middle, center

.width-70[]

.italic[“ACM named .bold[Yoshua Bengio], .bold[Geoffrey Hinton], and .bold[Yann LeCun] recipients of the .bold[2018 ACM A.M. Turing Award] for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing.”]

Why does it work now?

.center.grid[ .kol-1-2[ Algorithms (old and new)

.width-90[] ] .center.kol-1-2[ More data

.width-50[] ] ]

.center.grid[ .kol-1-2[ Software
.width-90[] ] .kol-1-2[ Faster compute engines

.width-50[] ] ]

???

The success of deep learning is multi-factorial…

class: middle

Building on the shoulders of giants

Five decades of research in machine learning provided

  • a taxonomy of ML concepts (classification, generative models, clustering, kernels, linear embeddings, etc.),
  • a sound statistical formalization (Bayesian estimation, PAC),
  • a clear picture of fundamental issues (bias/variance dilemma, VC dimension, generalization bounds, etc.),
  • a good understanding of optimization issues,
  • efficient large-scale algorithms.

.footnote[Credits: Francois Fleuret, EE559 Deep Learning, EPFL.]

class: middle

Deep learning

From a practical perspective, deep learning

  • lessens the need for a deep mathematical grasp,
  • makes the design of large learning architectures a system/software development task,
  • allows to leverage modern hardware (clusters of GPUs),
  • does not plateau when using more data,
  • makes large trained networks a commodity.

.footnote[Credits: Francois Fleuret, EE559 Deep Learning, EPFL.]

class: middle

.center.circle.width-30[]

.italic[For the last forty years we have programmed computers; for the next forty years we will train them.]

.pull-right[Chris Bishop, 2020.]

class: end-slide, center count: false

The end.