Chapter 1: Introduction

introduction

• deep learning solves the central problem in representation learning
  • find the right representation for data, e.g. polar vs rect coordinates

another perspective

• each layer in multilayer network represents
  • multi-steps in a computer program
  • each layer of representation is a state in a sequence of executions
  • depth equiv. no instructions
  • not all information in a layer’s activation encodes factors of variation that explain the input! Representation also store state information that helps to execute a program

depth of a model

1. computational depth : number of sequential instructions
2. depth of graph representing concepts PGM

• dlbook