In this lecture we continue our discussion of Convolutional Neural Networks.

Lecture 06 CNNs (slides modified from Hugo Larochelle’s course notes)

**Reference: **(* = you are responsible for this material)

- *Chapter 9 of the Deep Learning textbook (continued from last time), Sections 9.10 and 9.11 are optional.
- Slides include Hiroshi Kuwajima’s Memo on Backpropagation in Convolutional Neural Networks.
- Theano guide and paper on convolution arithmetic by Vincent Dumoulin and Francesco Visin.
- WaveNet Blog presenting dilated convolutions animation and samples.
- Blog on Deconvolution and Checkerboard Artifacts by Augustus Odena, Vincent Dumoulin and Chris Olah.

Advertisements

When we were looking at the ByteNet (slide 27), I asked the following: Isn’t the structure of dilated convolutions, when applied to time-series, similar to using a more classical statistical model with auto-correlation? each auto-correlation corresponding to a different frequency, in this case, more or less equivalent to a particular dilation level.

The anser given by Aaron was around these lines (I didn’t make a very detailed job of taking notes of the answer): The Intuition is the same… you are looking at different frequencies at the same time. However, deep learning for time series does not work well if the data is noisy. It works where the series have some kind of structure like voice, speech, music, or language. It probably is not well suited for “noisy” signals, like those seen in economy. It certainly is a completely different model.

Note: This comment is only to leave trace in the blog of something that I asked in class. I frequently asked questions, but was very shy to come to the blog and post them.

LikeLike