Support Distribution Machines (2012)

Barnabas Poczos, Liang Xiong, Dougal J. Sutherland, Jeff Schneider

Abstract

Most machine learning algorithms, such as classification or regression, treat
the individual data point as the object of interest.  Here we consider
extending machine learning algorithms to operate on groups of data
points.  We suggest treating a group of data points as a set of i.i.d. samples
from an underlying feature distribution for the group.  Our approach is to
generalize kernel machines from vectorial inputs to i.i.d. sample sets of
vectors. For this purpose, we use a nonparametric estimator that can consistently
estimate the inner product and certain kernel functions of two distributions.
The projection of the estimated Gram matrix to the cone of semi-definite
matrices enables us to employ the kernel trick, and hence use kernel machines
for classification, regression, anomaly detection, and low-dimensional
embedding in the space of distributions. We present several numerical
experiments both on real and simulated datasets to demonstrate the advantages
of our new approach.

Full text

Download (application/pdf, 2.5 MB)

Approximate BibTeX Entry

@techreport{poczos12SDM,
    Howpublished = {Technical Report},
    Year = {2012},
    Journal = {Technical Report},
    Booktitle = {Technical Report},
    Author = { Barnabas Poczos, Liang Xiong, Dougal J. Sutherland, Jeff Schneider },
    Title = {Support Distribution Machines}
}