Go to  Advanced Search

Recklessly approximate sparse coding

Show full item record

Files in this item

Files Size Format Description   View
ubc_2013_spring_denil_misha.pdf 407.0Kb Adobe Portable Document Format   View/Open
Title: Recklessly approximate sparse coding
Author: Denil, Misha
Degree: Master of Science - MSc
Program: Computer Science
Copyright Date: 2012
Issue Date: 2012-12-06
Publisher University of British Columbia
Abstract: Introduction of the so called “K-means” or “triangle” features in Coates, Lee and Ng, 2011 caused significant discussion in the deep learning community. These simple features are able to achieve state of the art performance on standard image classification benchmarks, outperforming much more sophisticated methods including deep belief networks, convolutional nets, factored RBMs, mcRBMs, convolutional RBMs, sparse autoencoders and several others. Moreover, these features are extremely simple and easy to compute. Several intuitive arguments have been put forward to describe this remarkable performance, yet no mathematical justification has been offered. In Coates and Ng, 2011, the authors improve on the triangle features with “soft threshold” features, adding a hyperparameter to tune performance, and compare these features to sparse coding. Both soft thresholding and sparse coding are found to often yield similar classification results, though soft threshold features are much faster to compute. The main result of this thesis is to show that the soft threshold features are realized as a single step of proximal gradient descent on a non-negative sparse coding objective. This result is important because it provides an explanation for the success of the soft threshold features and shows that even very approximate solutions to the sparse coding problem are sufficient to build effective classifiers.
Affiliation: Science, Faculty of
URI: http://hdl.handle.net/2429/43662
Scholarly Level: Graduate

This item appears in the following Collection(s)

Show full item record

UBC Library
1961 East Mall
Vancouver, B.C.
Canada V6T 1Z1
Tel: 604-822-6375
Fax: 604-822-3893