2020

CRISP: A Probabilistic Model for Individual-Level COVID-19 Infection Risk Estimation Based on Contact Data. Herbrich, Ralf; Rastogi, Rajeev; Vollgraf, Roland in arXiv:2006.04942 (2020).
[ Abstract ] [ BibTeX ] [ URL ]
 
We present CRISP (COVID-19 Risk Score Prediction), a probabilistic graphical model for COVID-19 infection spread through a population based on the SEIR model where we assume access to (1) mutual contacts between pairs of individuals across time across various channels (e.g., Bluetooth contact traces), as well as (2) test outcomes at given times for infection, exposure and immunity tests. Our micro-level model keeps track of the infection state for each individual at every point in time, ranging from susceptible, exposed, infectious to recovered. We develop a Monte Carlo EM algorithm to infer contact-channel specific infection transmission probabilities. Our algorithm uses Gibbs sampling to draw samples of the latent infection status of each individual over the entire time period of analysis, given the latent infection status of all contacts and test outcome data. Experimental results with simulated data demonstrate our CRISP model can be parametrized by the reproduction factor R0 and exhibits population-level infectiousness and recovery time series similar to those of the classical SEIR model. However, due to the individual contact data, this model allows fine grained control and inference for a wide range of COVID-19 mitigation and suppression policy measures. Moreover, the algorithm is able to support efficient testing in a test-trace-isolate approach to contain COVID-19 infection spread. To the best of our knowledge, this is the first model with efficient inference for COVID-19 infection spread based on individual-level contact data; most epidemic models are macro-level models that reason over entire populations.
@article{herbrich2020crisp, abstract = {We present CRISP (COVID-19 Risk Score Prediction), a probabilistic graphical model for COVID-19 infection spread through a population based on the SEIR model where we assume access to (1) mutual contacts between pairs of individuals across time across various channels (e.g., Bluetooth contact traces), as well as (2) test outcomes at given times for infection, exposure and immunity tests. Our micro-level model keeps track of the infection state for each individual at every point in time, ranging from susceptible, exposed, infectious to recovered. We develop a Monte Carlo EM algorithm to infer contact-channel specific infection transmission probabilities. Our algorithm uses Gibbs sampling to draw samples of the latent infection status of each individual over the entire time period of analysis, given the latent infection status of all contacts and test outcome data. Experimental results with simulated data demonstrate our CRISP model can be parametrized by the reproduction factor R0 and exhibits population-level infectiousness and recovery time series similar to those of the classical SEIR model. However, due to the individual contact data, this model allows fine grained control and inference for a wide range of COVID-19 mitigation and suppression policy measures. Moreover, the algorithm is able to support efficient testing in a test-trace-isolate approach to contain COVID-19 infection spread. To the best of our knowledge, this is the first model with efficient inference for COVID-19 infection spread based on individual-level contact data; most epidemic models are macro-level models that reason over entire populations.}, author = {Herbrich, Ralf and Rastogi, Rajeev and Vollgraf, Roland}, journal = {arXiv:2006.04942}, keywords = {imported ralf.herbrich}, title = {{CRISP}: A Probabilistic Model for Individual-Level {COVID-19} Infection Risk Estimation Based on Contact Data}, year = 2020 }

2014

Practical Lessons from Predicting Clicks on Ads at Facebook. He, Xinran; Pan, Junfeng; Jin, Ou; Xu, Tianbing; Liu, Bo; Xu, Tao; Shi, Yanxin; Atallah, Antoine; Herbrich, Ralf; Bowers, Stuart; Candela, Joaquin Qui nonero (2014). 1–9.
[ Abstract ] [ BibTeX ] [ URL ]
 
Online advertising allows advertisers to only bid and pay for measurable user responses, such as clicks on ads. As a consequence, click prediction systems are central to most on-line advertising systems. With over 750 million daily active users and over 1 million active advertisers, predicting clicks on Facebook ads is a challenging machine learning task. In this paper we introduce a model which combines decision trees with logistic regression, outperforming either of these methods on its own by over 3%, an improvement with significant impact to the overall system performance. We then explore how a number of fundamental parameters impact the final prediction performance of our system. Not surprisingly, the most important thing is to have the right features: those capturing historical information about the user or ad dominate other types of features. Once we have the right features and the right model (decisions trees plus logistic regression), other factors play small roles (though even small improvements are important at scale). Picking the optimal handling for data freshness, learning rate schema and data sampling improve the model slightly, though much less than adding a high-value feature, or picking the right model to begin with.
@inproceedings{he2014practical, abstract = {Online advertising allows advertisers to only bid and pay for measurable user responses, such as clicks on ads. As a consequence, click prediction systems are central to most on-line advertising systems. With over 750 million daily active users and over 1 million active advertisers, predicting clicks on Facebook ads is a challenging machine learning task. In this paper we introduce a model which combines decision trees with logistic regression, outperforming either of these methods on its own by over 3%, an improvement with significant impact to the overall system performance. We then explore how a number of fundamental parameters impact the final prediction performance of our system. Not surprisingly, the most important thing is to have the right features: those capturing historical information about the user or ad dominate other types of features. Once we have the right features and the right model (decisions trees plus logistic regression), other factors play small roles (though even small improvements are important at scale). Picking the optimal handling for data freshness, learning rate schema and data sampling improve the model slightly, though much less than adding a high-value feature, or picking the right model to begin with.}, author = {He, Xinran and Pan, Junfeng and Jin, Ou and Xu, Tianbing and Liu, Bo and Xu, Tao and Shi, Yanxin and Atallah, Antoine and Herbrich, Ralf and Bowers, Stuart and Candela, Joaquin Qui\ {n}onero}, booktitle = {Proceedings of 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining}, keywords = {imported ralf.herbrich}, organization = {ACM}, pages = {1–9}, title = {Practical Lessons from Predicting Clicks on Ads at {F}acebook}, year = 2014 }

2013

Speeding Up Large-Scale Learning with a Social Prior. Chakrabarti, Deepayan; Herbrich, Ralf (2013). 650–658.
[ Abstract ] [ BibTeX ] [ URL ]
 
Slow convergence and poor initial accuracy are two problems that plague efforts to use very large feature sets in online learning. This is especially true when only a few features are active in any training example, and the frequency of activations of different features is skewed. We show how these problems can be mitigated if a graph of relationships between features is known. We study this problem in a fully Bayesian setting, focusing on the problem of using Facebook user-IDs as features, with the social network giving the relationship structure. Our analysis uncovers significant problems with the obvious regularizations, and motivates a two-component mixture-model social prior that is provably better. Empirical results on large-scale click prediction problems show that our algorithm can learn as well as the baseline with 12M fewer training examples, and continuously outperforms it for over 60M examples. On a second problem using binned features, our model outperforms the baseline even after the latter sees 5x as much data.
@inproceedings{chakrabarti2013speeding, abstract = {Slow convergence and poor initial accuracy are two problems that plague efforts to use very large feature sets in online learning. This is especially true when only a few features are active in any training example, and the frequency of activations of different features is skewed. We show how these problems can be mitigated if a graph of relationships between features is known. We study this problem in a fully Bayesian setting, focusing on the problem of using Facebook user-IDs as features, with the social network giving the relationship structure. Our analysis uncovers significant problems with the obvious regularizations, and motivates a two-component mixture-model social prior that is provably better. Empirical results on large-scale click prediction problems show that our algorithm can learn as well as the baseline with 12M fewer training examples, and continuously outperforms it for over 60M examples. On a second problem using binned features, our model outperforms the baseline even after the latter sees 5x as much data.}, author = {Chakrabarti, Deepayan and Herbrich, Ralf}, booktitle = {Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining}, keywords = {imported ralf.herbrich}, organization = {ACM}, pages = {650–658}, title = {Speeding Up Large-Scale Learning with a Social Prior}, year = 2013 }

2012

Transparent User Models for Personalization. El-Arini, Khalid; Paquet, Ulrich; Herbrich, Ralf; Van Gael, Jurgen; Agüera y Arcas, Blaise (2012). 678–686.
[ Abstract ] [ BibTeX ] [ URL ]
 
Personalization is a ubiquitous phenomenon in our daily online experience. While such technology is critical for helping us combat the overload of information we face, in many cases, we may not even realize that our results are being tailored to our personal tastes and preferences. Worse yet, when such a system makes a mistake, we have little recourse to correct it. In this work, we propose a framework for addressing this problem by developing a new user-interpretable feature set upon which to base personalized recommendations. These features, which we call badges, represent fundamental traits of users (e.g., ``vegetarian'' or ``Apple fanboy'') inferred by modeling the interplay between a user's behavior and self-reported identity. Specifically, we consider the microblogging site Twitter, where users provide short descriptions of themselves in their profiles, as well as perform actions such as tweeting and retweeting. Our approach is based on the insight that we can define badges using high precision, low recall rules (e.g., ``Twitter profile contains the phrase `Apple fanboy'''), and with enough data, generalize to other users by observing shared behavior. We develop a fully Bayesian, generative model that describes this interaction, while allowing us to avoid the pitfalls associated with having positive-only data. Experiments on real Twitter data demonstrate the effectiveness of our model at capturing rich and interpretable user traits that can be used to provide transparency for personalization.
@inproceedings{elarini2012transparent, abstract = {Personalization is a ubiquitous phenomenon in our daily online experience. While such technology is critical for helping us combat the overload of information we face, in many cases, we may not even realize that our results are being tailored to our personal tastes and preferences. Worse yet, when such a system makes a mistake, we have little recourse to correct it. In this work, we propose a framework for addressing this problem by developing a new user-interpretable feature set upon which to base personalized recommendations. These features, which we call badges, represent fundamental traits of users (e.g., ``vegetarian'' or ``Apple fanboy'') inferred by modeling the interplay between a user's behavior and self-reported identity. Specifically, we consider the microblogging site Twitter, where users provide short descriptions of themselves in their profiles, as well as perform actions such as tweeting and retweeting. Our approach is based on the insight that we can define badges using high precision, low recall rules (e.g., ``Twitter profile contains the phrase `Apple fanboy'''), and with enough data, generalize to other users by observing shared behavior. We develop a fully Bayesian, generative model that describes this interaction, while allowing us to avoid the pitfalls associated with having positive-only data. Experiments on real Twitter data demonstrate the effectiveness of our model at capturing rich and interpretable user traits that can be used to provide transparency for personalization.}, author = {El-Arini, Khalid and Paquet, Ulrich and Herbrich, Ralf and Van Gael, Jurgen and Ag{ü}era y Arcas, Blaise}, booktitle = {Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining}, keywords = {imported ralf.herbrich}, organization = {ACM}, pages = {678–686}, title = {Transparent User Models for Personalization}, year = 2012 }

Kernel Topic Models. Hennig, Philipp; Stern, David; Herbrich, Ralf; Graepel, Thore (2012). 511–519.
[ Abstract ] [ BibTeX ] [ URL ]
 
Latent Dirichlet Allocation models discrete data as a mixture of discrete distributions, using Dirichlet beliefs over the mixture weights. We study a variation of this concept, in which the documents' mixture weight beliefs are replaced with squashed Gaussian distributions. This allows documents to be associated with elements of a Hilbert space, admitting kernel topic models (KTM), modelling temporal, spatial, hierarchical, social and other structure between documents. The main challenge is efficient approximate inference on the latent Gaussian. We present an approximate algorithm cast around a Laplace approximation in a transformed basis. The KTM can also be interpreted as a type of Gaussian process latent variable model, or as a topic model conditional on document features, uncovering links between earlier work in these areas.
@inproceedings{hennig2012kernel, abstract = {Latent Dirichlet Allocation models discrete data as a mixture of discrete distributions, using Dirichlet beliefs over the mixture weights. We study a variation of this concept, in which the documents' mixture weight beliefs are replaced with squashed Gaussian distributions. This allows documents to be associated with elements of a Hilbert space, admitting kernel topic models (KTM), modelling temporal, spatial, hierarchical, social and other structure between documents. The main challenge is efficient approximate inference on the latent Gaussian. We present an approximate algorithm cast around a Laplace approximation in a transformed basis. The KTM can also be interpreted as a type of Gaussian process latent variable model, or as a topic model conditional on document features, uncovering links between earlier work in these areas.}, author = {Hennig, Philipp and Stern, David and Herbrich, Ralf and Graepel, Thore}, booktitle = {Proceedings of the 15th International Conference on Artificial Intelligence and Statistics (AISTATS)}, keywords = {imported ralf.herbrich}, pages = {511–519}, title = {Kernel Topic Models}, year = 2012 }

Distributed, Real-time Bayesian Learning in Online Services. Herbrich, Ralf (2012). 203–204.
[ Abstract ] [ BibTeX ] [ URL ]
 
The last ten years have seen a tremendous growth in Internet-based online services such as search, advertising, gaming and social networking. Today, it is important to analyze large collections of user interaction data as a first step in building predictive models for these services as well as learn these models in real-time. One of the biggest challenges in this setting is scale: not only does the sheer scale of data necessitate parallel processing but it also necessitates distributed models; with over 900 million active users at Facebook, any user-specific sets of features in a linear or non-linear model yields models of a size bigger than can be stored in a single system. In this talk, I will give a hands-on introduction to one of the most versatile tools for handling large collections of data with distributed probabilistic models: the sum-product algorithm for approximate message passing in factor graphs. I will discuss the application of this algorithm for the specific case of generalized linear models and outline the challenges of both approximate and distributed message passing including an in-depth discussion of expectation propagation and Map-Reduce.
@inproceedings{herbrich2012distributed, abstract = {The last ten years have seen a tremendous growth in Internet-based online services such as search, advertising, gaming and social networking. Today, it is important to analyze large collections of user interaction data as a first step in building predictive models for these services as well as learn these models in real-time. One of the biggest challenges in this setting is scale: not only does the sheer scale of data necessitate parallel processing but it also necessitates distributed models; with over 900 million active users at Facebook, any user-specific sets of features in a linear or non-linear model yields models of a size bigger than can be stored in a single system. In this talk, I will give a hands-on introduction to one of the most versatile tools for handling large collections of data with distributed probabilistic models: the sum-product algorithm for approximate message passing in factor graphs. I will discuss the application of this algorithm for the specific case of generalized linear models and outline the challenges of both approximate and distributed message passing including an in-depth discussion of expectation propagation and Map-Reduce.}, author = {Herbrich, Ralf}, booktitle = {Proceedings of the 6th ACM Conference on Recommender Systems}, keywords = {imported ralf.herbrich}, organization = {ACM}, pages = {203–204}, title = {Distributed, Real-time {Bayesian} Learning in Online Services}, year = 2012 }

De-Layering Social Networks by Shared Tastes of Friendships. Dietz, Laura; Gamari, Ben; Guiver, John; Snelson, Edward; Herbrich, Ralf (2012).
[ Abstract ] [ BibTeX ] [ URL ]
 
Traditionally, social network analyses are applied to data from a particular social domain. With the advent of online social networks such as Facebook, we observe an aggregate of various social domains resulting in a layered mix of professional contacts, family ties, and different circles. These aggregates dilute the community structure. We provide a method for de-layering social networks according to shared interests. Instead of relying on changes in the edge density, our shared taste model uses content of users to disambiguate the underlying shared interest of each friendship. We successfully de-layer real world networks from LibraryThing and Boards.ie, obtaining topics that significantly outperform LDA on unsupervised prediction of group membership.
@inproceedings{dietz2012delayering, abstract = {Traditionally, social network analyses are applied to data from a particular social domain. With the advent of online social networks such as Facebook, we observe an aggregate of various social domains resulting in a layered mix of professional contacts, family ties, and different circles. These aggregates dilute the community structure. We provide a method for de-layering social networks according to shared interests. Instead of relying on changes in the edge density, our shared taste model uses content of users to disambiguate the underlying shared interest of each friendship. We successfully de-layer real world networks from LibraryThing and Boards.ie, obtaining topics that significantly outperform LDA on unsupervised prediction of group membership.}, author = {Dietz, Laura and Gamari, Ben and Guiver, John and Snelson, Edward and Herbrich, Ralf}, booktitle = {International AAAI Conference on Web and Social Media}, keywords = {imported ralf.herbrich}, title = {De-Layering Social Networks by Shared Tastes of Friendships}, year = 2012 }

A Bayesian Treatment of Social Links in Recommender Systems Gartrell, Mike; Paquet, Ulrich; Herbrich, Ralf (2012).
[ Abstract ] [ BibTeX ] [ URL ]
 
Recommender systems are increasingly driving user experiences on the Internet. This personalization is often achieved through the factorization of a large but sparse observation matrix of user-item feedback signals. In instances where the user's social network is known, its inclusion can significantly improve recommendations for cold start users. There are numerous ways in which the network can be incorporated into a probabilistic graphical model. We propose and investigate two ways for including a social network, either as a Markov Random Field that describes a user similarity in the prior over user features, or an explicit model that treats social links as observations. State of the art performance is reported on the Flixster online social network dataset.
@techreport{gartrell2012bayesian, abstract = {Recommender systems are increasingly driving user experiences on the Internet. This personalization is often achieved through the factorization of a large but sparse observation matrix of user-item feedback signals. In instances where the user's social network is known, its inclusion can significantly improve recommendations for cold start users. There are numerous ways in which the network can be incorporated into a probabilistic graphical model. We propose and investigate two ways for including a social network, either as a Markov Random Field that describes a user similarity in the prior over user features, or an explicit model that treats social links as observations. State of the art performance is reported on the Flixster online social network dataset.}, author = {Gartrell, Mike and Paquet, Ulrich and Herbrich, Ralf}, institution = {CU Technical Report CU-CS-1092-12}, keywords = {imported ralf.herbrich}, title = {A {Bayesian} Treatment of Social Links in Recommender Systems}, year = 2012 }

2011

Sociable Killers: Understanding Social Relationships in an Online First-Person Shooter Game. Xu, Yan; Cao, Xiang; Sellen, Abigail; Herbrich, Ralf; Graepel, Thore (2011). 197–206.
[ Abstract ] [ BibTeX ] [ URL ]
 
Online video games can be seen as medium for the formation and maintenance of social relationships. In this paper, we explore what social relationships mean under the context of online First-Person Shooter (FPS) games, how these relationships influence game experience, and how players manage them. We combine qualitative interview and quantitative game log data, and find that despite the gap between the non-persistent game world and potentially persistent social relationships, a diversity of social relationships emerge and play a central role in the enjoyment of online FPS games. We report the forms, development, and impact of such relationships, and discuss our findings in light of design implications and comparison with other game genres.
@inproceedings{xu2011sociable, abstract = {Online video games can be seen as medium for the formation and maintenance of social relationships. In this paper, we explore what social relationships mean under the context of online First-Person Shooter (FPS) games, how these relationships influence game experience, and how players manage them. We combine qualitative interview and quantitative game log data, and find that despite the gap between the non-persistent game world and potentially persistent social relationships, a diversity of social relationships emerge and play a central role in the enjoyment of online FPS games. We report the forms, development, and impact of such relationships, and discuss our findings in light of design implications and comparison with other game genres.}, author = {Xu, Yan and Cao, Xiang and Sellen, Abigail and Herbrich, Ralf and Graepel, Thore}, booktitle = {Proceedings of the 2011 ACM Conference on Computer Supported Cooperative Work}, keywords = {imported ralf.herbrich}, pages = {197–206}, title = {Sociable Killers: Understanding Social Relationships in an Online First-Person Shooter Game}, year = 2011 }

Behavioral Game Theory on Online Social Networks: Colonel Blotto is on Facebook. Kohli, Pushmeet; Bachrach, Yoram; Graepel, Thore; Smyth, Gavin; Armstrong, Michael; Stillwell, David; Kearns, Michael (2011).
[ Abstract ] [ BibTeX ] [ URL ]
 
We show how online social networks such as Facebook can be used in Behavioral Game Theory research. We report the deployment of a Facebook application Project Waterloo that allows users to play the Colonel Blotto game against their friends and strangers. Unlike conventional studies performed in the laboratory environment, which rely on monetary incentives to attract human subjects to play games, our framework does not use money and instead relies on reputation and entertainment incentives. We describe the Facebook application we created for conducting this experiment, and perform a preliminary analysis of the data collected in the game. We conclude by discussing the advantages of our approach and list some ideas for future work.
@inproceedings{kohli2011behavioral, abstract = {We show how online social networks such as Facebook can be used in Behavioral Game Theory research. We report the deployment of a Facebook application Project Waterloo that allows users to play the Colonel Blotto game against their friends and strangers. Unlike conventional studies performed in the laboratory environment, which rely on monetary incentives to attract human subjects to play games, our framework does not use money and instead relies on reputation and entertainment incentives. We describe the Facebook application we created for conducting this experiment, and perform a preliminary analysis of the data collected in the game. We conclude by discussing the advantages of our approach and list some ideas for future work.}, author = {Kohli, Pushmeet and Bachrach, Yoram and Graepel, Thore and Smyth, Gavin and Armstrong, Michael and Stillwell, David and Kearns, Michael}, booktitle = {Proceedings of the 2nd Workshop on Computational Social Science and the Wisdom of Crowds}, keywords = {imported ralf.herbrich}, title = {Behavioral Game Theory on Online Social Networks: Colonel Blotto is on {Facebook}}, year = 2011 }

Automated Feature Generation From Structured Knowledge. Cheng, Weiwei; Kasneci, Gjergji; Graepel, Thore; Stern, David H; Herbrich, Ralf (2011). 1395–1404.
[ Abstract ] [ BibTeX ] [ URL ]
 
The prediction accuracy of any learning algorithm highly depends on the quality of the selected features; but often, the task of feature construction and selection is tedious and non-scalable. In recent years, however, there have been numerous projects with the goal of constructing general-purpose or domain-specific knowledge bases with entity-relationship-entity triples extracted from various Web sources or collected from user communities, e.g., YAGO, DBpedia, Free- base, UMLS, etc. This paper advocates the simple and yet far-reaching idea that the structured knowledge contained in such knowledge bases can be exploited to automatically extract features for general learning tasks. We introduce an expressive graph-based language for extracting features from such knowledge bases and a theoretical framework for constructing feature vectors from the extracted features. Our experimental evaluation on different learning scenarios provides evidence that the features derived through our framework can considerably improve the prediction accuracy, especially when the labeled data at hand is sparse.
@inproceedings{cheng2011automated, abstract = {The prediction accuracy of any learning algorithm highly depends on the quality of the selected features; but often, the task of feature construction and selection is tedious and non-scalable. In recent years, however, there have been numerous projects with the goal of constructing general-purpose or domain-specific knowledge bases with entity-relationship-entity triples extracted from various Web sources or collected from user communities, e.g., YAGO, DBpedia, Free- base, UMLS, etc. This paper advocates the simple and yet far-reaching idea that the structured knowledge contained in such knowledge bases can be exploited to automatically extract features for general learning tasks. We introduce an expressive graph-based language for extracting features from such knowledge bases and a theoretical framework for constructing feature vectors from the extracted features. Our experimental evaluation on different learning scenarios provides evidence that the features derived through our framework can considerably improve the prediction accuracy, especially when the labeled data at hand is sparse.}, author = {Cheng, Weiwei and Kasneci, Gjergji and Graepel, Thore and Stern, David H and Herbrich, Ralf}, booktitle = {Proceedings of the 20th ACM Conference on Information and Knowledge Management}, keywords = {imported ralf.herbrich}, pages = {1395–1404}, title = {Automated Feature Generation From Structured Knowledge}, year = 2011 }

A Penny for Your Thoughts? The Value of Information in Recommendation Systems. Passos, Alexandre; Van Gael, Juergen; Herbrich, Ralf; Paquet, Ulrich (2011). 9–14.
[ Abstract ] [ BibTeX ] [ URL ]
 
Most recommendation systems are trained to predict behavioral data and then used to generate more such data by recommending items and receiving feedback on the quality of these recommendations. This data in then fed back into the training process. This creates a feedback loop: as long as the low-cost way to interact with the service is through the recommender, the recommender will only ever see behavioral data on the items it chooses. This process can lead to hidden biases, as it effectively limits how much information the recommender system will ever see. On the other hand, there is a cost to making exploratory recommendations, as they should, myopically, be worse than the best bets of a recommendation system. In this paper we explore the notion that recommender systems are a special kind of active learning agents, with the peculiarity that the cost of asking for the label of an instance depends on its true label, as the cost of showing a bad recommendation when exploring is higher than the cost of showing a good recommendation.
@inproceedings{passos2011penny, abstract = {Most recommendation systems are trained to predict behavioral data and then used to generate more such data by recommending items and receiving feedback on the quality of these recommendations. This data in then fed back into the training process. This creates a feedback loop: as long as the low-cost way to interact with the service is through the recommender, the recommender will only ever see behavioral data on the items it chooses. This process can lead to hidden biases, as it effectively limits how much information the recommender system will ever see. On the other hand, there is a cost to making exploratory recommendations, as they should, myopically, be worse than the best bets of a recommendation system. In this paper we explore the notion that recommender systems are a special kind of active learning agents, with the peculiarity that the cost of asking for the label of an instance depends on its true label, as the cost of showing a bad recommendation when exploring is higher than the cost of showing a good recommendation.}, author = {Passos, Alexandre and Van Gael, Juergen and Herbrich, Ralf and Paquet, Ulrich}, booktitle = {{NIPS} Workshop on {Bayesian} Optimization, Experimental Design, and Bandits}, keywords = {imported ralf.herbrich}, pages = {9–14}, title = {A Penny for Your Thoughts? The Value of Information in Recommendation Systems}, year = 2011 }

2010

Web-Scale Bayesian Click-Through Rate Prediction for Sponsored Search Advertising in Microsoft’s Bing Search Engine. Graepel, Thore; Candela, Joaquin Qui nonero; Borchert, Thomas; Herbrich, Ralf (2010). 13–20.
[ Abstract ] [ BibTeX ] [ URL ]
 
We describe a new Bayesian click-through rate (CTR) prediction algorithm used for Sponsored Search in Microsoft's Bing search engine. The algorithm is based on a probit regression model that maps discrete or real-valued input features to probabilities. It maintains Gaussian beliefs over weights of the model and performs Gaussian online updates derived from approximate message passing. Scalability of the algorithm is ensured through a principled weight pruning procedure and an approximate parallel implementation. We discuss the challenges arising from evaluating and tuning the predictor as part of the complex system of sponsored search where the predictions made by the algorithm decide about future training sample composition. Finally, we show experimental results from the production system and compare to a calibrated Naive Bayes algorithm.
@inproceedings{graepel2010webscale, abstract = {We describe a new Bayesian click-through rate (CTR) prediction algorithm used for Sponsored Search in Microsoft's Bing search engine. The algorithm is based on a probit regression model that maps discrete or real-valued input features to probabilities. It maintains Gaussian beliefs over weights of the model and performs Gaussian online updates derived from approximate message passing. Scalability of the algorithm is ensured through a principled weight pruning procedure and an approximate parallel implementation. We discuss the challenges arising from evaluating and tuning the predictor as part of the complex system of sponsored search where the predictions made by the algorithm decide about future training sample composition. Finally, we show experimental results from the production system and compare to a calibrated Naive Bayes algorithm.}, author = {Graepel, Thore and Candela, Joaquin Qui\ {n}onero and Borchert, Thomas and Herbrich, Ralf}, booktitle = {Proceedings of the 27th International Conference on Machine Learning}, keywords = {imported ralf.herbrich}, pages = {13–20}, title = {Web-Scale Bayesian Click-Through Rate Prediction for Sponsored Search Advertising in Microsoft's {B}ing Search Engine}, year = 2010 }

Vuvuzelas & Active Learning for Online Classification. Paquet, Ulrich; Van Gael, Jurgen; Stern, David; Kasneci, Gjergji; Herbrich, Ralf; Graepel, Thore (2010).
[ Abstract ] [ BibTeX ] [ URL ]
 
Many online service systems leverage user-generated content from Web 2.0 style platforms such as Wikipedia, Twitter, Facebook, and many more. Often, the value lies in the freshness of this information (e.g. tweets, event-based articles, blog posts, etc.). This freshness poses a challenge for supervised learning models as they frequently have to deal with previously unseen features. In this paper we address the problem of online classification for tweets, namely, how can a classifier be updated in an online manner, so that it can correctly classify the latest hype on Twitter? We propose a two-step strategy to solve this problem. The first step follows an active learning strategy that enables the selection of tweets for which a label would be most useful; the selected tweet is then forwarded to Amazon Mechanical Turk where it is labeled by multiple users. The second step builds on a Bayesian corroboration model that aggregates the noisy labels provided by the users by taking their reliabilities into account.
@inproceedings{paquet2010vuvuzelas, abstract = {Many online service systems leverage user-generated content from Web 2.0 style platforms such as Wikipedia, Twitter, Facebook, and many more. Often, the value lies in the freshness of this information (e.g. tweets, event-based articles, blog posts, etc.). This freshness poses a challenge for supervised learning models as they frequently have to deal with previously unseen features. In this paper we address the problem of online classification for tweets, namely, how can a classifier be updated in an online manner, so that it can correctly classify the latest hype on Twitter? We propose a two-step strategy to solve this problem. The first step follows an active learning strategy that enables the selection of tweets for which a label would be most useful; the selected tweet is then forwarded to Amazon Mechanical Turk where it is labeled by multiple users. The second step builds on a Bayesian corroboration model that aggregates the noisy labels provided by the users by taking their reliabilities into account.}, author = {Paquet, Ulrich and {Van Gael}, Jurgen and Stern, David and Kasneci, Gjergji and Herbrich, Ralf and Graepel, Thore}, booktitle = {Proceedings of Computational Social Science and the Wisdom of Crowds Workshop}, keywords = {imported ralf.herbrich}, title = {{Vuvuzelas \& Active Learning for Online Classification}}, year = 2010 }

Predicting Information Spreading in Twitter. Zaman, Tauhid R; Herbrich, Ralf; Van Gael, Jurgen; Stern, David (2010).
[ Abstract ] [ BibTeX ] [ URL ]
 
We present a new methodology for predicting the spread of information in a social network. We focus on the Twitter network, where information is in the form of 140 character messages called tweets, and information is spread by users forwarding tweets, a practice known as retweeting. Using data of who and what was retweeted, we train a probabilistic collaborative filter model to predict future retweets. We find that the most important features for prediction are the identity of the source of the tweet and retweeter. Our methodology is quite flexible and be used as a basis for other prediction models in social networks.
@inproceedings{zaman2010predicting, abstract = {We present a new methodology for predicting the spread of information in a social network. We focus on the Twitter network, where information is in the form of 140 character messages called tweets, and information is spread by users forwarding tweets, a practice known as retweeting. Using data of who and what was retweeted, we train a probabilistic collaborative filter model to predict future retweets. We find that the most important features for prediction are the identity of the source of the tweet and retweeter. Our methodology is quite flexible and be used as a basis for other prediction models in social networks.}, author = {Zaman, Tauhid R and Herbrich, Ralf and {Van Gael}, Jurgen and Stern, David}, booktitle = {Proceedings of Computational Social Science and the Wisdom of Crowds Workshop}, keywords = {imported ralf.herbrich}, title = {Predicting Information Spreading in Twitter}, year = 2010 }

Fingerprinting Ratings for Collaborative Filtering - Theoretical and Empirical Analysis. Bachrach, Yoram; Herbrich, Ralf (2010). 25–36.
[ Abstract ] [ BibTeX ] [ URL ]
 
We consider fingerprinting methods for collaborative filtering (CF) systems. In general, CF systems show their real strength when supplied with enormous data sets. Earlier work already suggests sketching techniques to handle massive amounts of information, but most prior analysis has so far been limited to non-ranking application scenarios and has focused mainly on a theoretical analysis. We demonstrate how to use fingerprinting methods to compute a family of rank correlation coefficients. Our methods allow identifying users who have similar rankings over a certain set of items, a problem that lies at the heart of CF applications. We show that our method allows approximating rank correlations with high accuracy and confidence. We examine the suggested methods empirically through a recommender system for the Netflix dataset, showing that the required fingerprint sizes are even smaller than the theoretical analysis suggests. We also explore the of use standard hash functions rather than min-wise independent hashes and the relation between the quality of the final recommendations and the fingerprint size.
@inproceedings{bachrach2010fingerprinting, abstract = {We consider fingerprinting methods for collaborative filtering (CF) systems. In general, CF systems show their real strength when supplied with enormous data sets. Earlier work already suggests sketching techniques to handle massive amounts of information, but most prior analysis has so far been limited to non-ranking application scenarios and has focused mainly on a theoretical analysis. We demonstrate how to use fingerprinting methods to compute a family of rank correlation coefficients. Our methods allow identifying users who have similar rankings over a certain set of items, a problem that lies at the heart of CF applications. We show that our method allows approximating rank correlations with high accuracy and confidence. We examine the suggested methods empirically through a recommender system for the Netflix dataset, showing that the required fingerprint sizes are even smaller than the theoretical analysis suggests. We also explore the of use standard hash functions rather than min-wise independent hashes and the relation between the quality of the final recommendations and the fingerprint size.}, author = {Bachrach, Yoram and Herbrich, Ralf}, booktitle = {Proceedings of 17th International Symposium on String Processing and Information Retrieval}, keywords = {imported ralf.herbrich}, pages = {25–36}, title = {Fingerprinting Ratings for Collaborative Filtering - Theoretical and Empirical Analysis}, year = 2010 }

Collaborative Expert Portfolio Management. Stern, David; Samulowitz, Horst; Herbrich, Ralf; Graepel, Thore; Pulina, Luca; Tacchella, Armando (2010).
[ Abstract ] [ BibTeX ] [ URL ]
 
We consider the task of assigning experts from a portfolio of specialists in order to solve a set of tasks. We apply a Bayesian model which combines collaborative filtering with a feature-based description of tasks and experts to yield a general framework for managing a portfolio of experts. The model learns an embedding of tasks and problems into a latent space in which affinity is measured by the inner product. The model can be trained incrementally and can track non-stationary data, tracking potentially changing expert and task characteristics. The approach allows us to use a principled decision theoretic framework for expert selection, allowing the user to choose a utility function that best suits their objectives. The model component for taking into account the performance feedback data is pluggable, allowing flexibility. We apply the model to manage a portfolio of algorithms to solve hard combinatorial problems. This is a well studied area and we demonstrate a large improvement on the state of the art in one domain (constraint solving) and in a second domain (combinatorial auctions) created a portfolio that performed significantly better than any single algorithm.
@inproceedings{stern2010collaborative, abstract = {We consider the task of assigning experts from a portfolio of specialists in order to solve a set of tasks. We apply a Bayesian model which combines collaborative filtering with a feature-based description of tasks and experts to yield a general framework for managing a portfolio of experts. The model learns an embedding of tasks and problems into a latent space in which affinity is measured by the inner product. The model can be trained incrementally and can track non-stationary data, tracking potentially changing expert and task characteristics. The approach allows us to use a principled decision theoretic framework for expert selection, allowing the user to choose a utility function that best suits their objectives. The model component for taking into account the performance feedback data is pluggable, allowing flexibility. We apply the model to manage a portfolio of algorithms to solve hard combinatorial problems. This is a well studied area and we demonstrate a large improvement on the state of the art in one domain (constraint solving) and in a second domain (combinatorial auctions) created a portfolio that performed significantly better than any single algorithm.}, author = {Stern, David and Samulowitz, Horst and Herbrich, Ralf and Graepel, Thore and Pulina, Luca and Tacchella, Armando}, booktitle = {Proceedings of the 24th AAAI Conference on Artificial Intelligence}, keywords = {imported ralf.herbrich}, title = {Collaborative Expert Portfolio Management}, year = 2010 }

Bayesian Online Learning for Multi-label and Multi-variate Performance Measures. Zhang, Xinhua; Graepel, Thore; Herbrich, Ralf (2010). 956–963.
[ Abstract ] [ BibTeX ] [ URL ]
 
Many real world applications employ multi-variate performance measures and each example can belong to multiple classes. The currently most popular approaches train an SVM for each class, followed by ad-hoc thresholding. Probabilistic models using Bayesian decision theory are also commonly adopted. In this paper, we propose a Bayesian online multi-label classification framework (BOMC) which learns a probabilistic linear classifier. The likelihood is modeled by a graphical model similar to TrueSkill, and inference is based on Gaussian density filtering with expectation propagation. Using samples from the posterior, we label the testing data by maximizing the expected F1-score. Our experiments on Reuters1-v2 dataset show BOMC compares favorably to the state-of-the-art online learners in macro-averaged F1-score and training time.
@inproceedings{zhang2010bayesian, abstract = {Many real world applications employ multi-variate performance measures and each example can belong to multiple classes. The currently most popular approaches train an SVM for each class, followed by ad-hoc thresholding. Probabilistic models using Bayesian decision theory are also commonly adopted. In this paper, we propose a Bayesian online multi-label classification framework (BOMC) which learns a probabilistic linear classifier. The likelihood is modeled by a graphical model similar to TrueSkill, and inference is based on Gaussian density filtering with expectation propagation. Using samples from the posterior, we label the testing data by maximizing the expected F1-score. Our experiments on Reuters1-v2 dataset show BOMC compares favorably to the state-of-the-art online learners in macro-averaged F1-score and training time.}, author = {Zhang, Xinhua and Graepel, Thore and Herbrich, Ralf}, booktitle = {Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS)}, keywords = {imported ralf.herbrich}, pages = {956–963}, title = {{Bayesian} Online Learning for Multi-label and Multi-variate Performance Measures}, year = 2010 }

Bayesian Knowledge Corroboration with Logical Rules and User Feedback. Kasneci, Gjergji; Gael, Jurgen Van; Herbrich, Ralf; Graepel, Thore (2010). 1–18.
[ Abstract ] [ BibTeX ] [ URL ]
 
Current knowledge bases suffer from either low coverage or low accuracy. The underlying hypothesis of this work is that user feedback can greatly improve the quality of automatically extracted knowledge bases. The feedback could help quantify the uncertainty associated with the stored statements and would enable mechanisms for searching, ranking and reasoning at entity-relationship level. Most importantly, a principled model for exploiting user feedback to learn the truth values of statements in the knowledge base would be a major step forward in addressing the issue of knowledge base curation. We present a family of probabilistic graphical models that builds on user feedback and logical inference rules derived from the popular Semantic-Web formalism of RDFS. Through internal inference and belief propagation, these models can learn both, the truth values of the statements in the knowledge base and the reliabilities of the users who give feedback. We demonstrate the viability of our approach in extensive experiments on real-world datasets, with feedback collected from Amazon Mechanical Turk.
@inproceedings{kasneci2010bayesian, abstract = {Current knowledge bases suffer from either low coverage or low accuracy. The underlying hypothesis of this work is that user feedback can greatly improve the quality of automatically extracted knowledge bases. The feedback could help quantify the uncertainty associated with the stored statements and would enable mechanisms for searching, ranking and reasoning at entity-relationship level. Most importantly, a principled model for exploiting user feedback to learn the truth values of statements in the knowledge base would be a major step forward in addressing the issue of knowledge base curation. We present a family of probabilistic graphical models that builds on user feedback and logical inference rules derived from the popular Semantic-Web formalism of RDFS. Through internal inference and belief propagation, these models can learn both, the truth values of the statements in the knowledge base and the reliabilities of the users who give feedback. We demonstrate the viability of our approach in extensive experiments on real-world datasets, with feedback collected from Amazon Mechanical Turk.}, author = {Kasneci, Gjergji and Gael, Jurgen Van and Herbrich, Ralf and Graepel, Thore}, booktitle = {Proceedings of European Conference on Machine Learning and Knowledge Discovery in Databases}, keywords = {imported ralf.herbrich}, pages = {1–18}, title = {{Bayesian} Knowledge Corroboration with Logical Rules and User Feedback}, year = 2010 }

2009

Sketching Algorithms for Approximating Rank Correlations in Collaborative Filtering Systems. Bachrach, Yoram; Herbrich, Ralf; Porat, Ely (2009). 344–352.
[ Abstract ] [ BibTeX ] [ URL ]
 
Collaborative filtering (CF) shares information between users to provide each with recommendations. Previous work suggests using sketching techniques to handle massive data sets in CF systems, but only allows testing whether users have a high proportion of items they have both ranked. We show how to determine the correlation between the rankings of two users, using concise sketches of the rankings. The sketches allow approximating Kendall's Tau, a known rank correlation, with high accuracy \($\epsilon$\) and high confidence \($1-\delta$\). The required sketch size is logarithmic in the confidence and polynomial in the accuracy.
@inproceedings{bachrach2009sketching, abstract = {Collaborative filtering (CF) shares information between users to provide each with recommendations. Previous work suggests using sketching techniques to handle massive data sets in CF systems, but only allows testing whether users have a high proportion of items they have both ranked. We show how to determine the correlation between the rankings of two users, using concise sketches of the rankings. The sketches allow approximating Kendall's Tau, a known rank correlation, with high accuracy $\epsilon$ and high confidence $1-\delta$. The required sketch size is logarithmic in the confidence and polynomial in the accuracy.}, author = {Bachrach, Yoram and Herbrich, Ralf and Porat, Ely}, booktitle = {Proceedings of 16th International Symposium String Processing and Information Retrieval}, keywords = {imported ralf.herbrich}, pages = {344–352}, title = {Sketching Algorithms for Approximating Rank Correlations in Collaborative Filtering Systems}, year = 2009 }

Scalable Clustering and Keyword Suggestion for Online Advertisements. Schwaighofer, Anton; Candela, Joaquin Qui nonero; Borchert, Thomas; Graepel, Thore; Herbrich, Ralf (2009). 27–36.
[ Abstract ] [ BibTeX ] [ URL ]
 
We present an efficient Bayesian online learning algorithm for clustering vectors of binary values based on a well known model, the mixture of Bernoulli profiles. The model includes conjugate Beta priors over the success probabilities and maintains discrete probability distributions for cluster assignments. Clustering is then formulated as inference in a factor graph which is solved efficiently using online approximate message passing. The resulting algorithm has three key features: a) it requires only a single pass across the data and can hence be used on data streams, b) it maintains the uncertainty of parameters and cluster assignments, and c) it implements an automatic step size adaptation based on the current model uncertainty. The model is tested on an artificially generated toy dataset and applied to a large scale real-world data set from online advertising, the data being online ads characterized by the set of keywords to which they have been subscribed. The proposed approach scales well for large datasets, and compares favorably to other clustering algorithms on the ads dataset. As a concrete application to online advertising we show how the learned model can be used to recommend new keywords for given ads.
@inproceedings{schwaighofer2009scalable, abstract = {We present an efficient Bayesian online learning algorithm for clustering vectors of binary values based on a well known model, the mixture of Bernoulli profiles. The model includes conjugate Beta priors over the success probabilities and maintains discrete probability distributions for cluster assignments. Clustering is then formulated as inference in a factor graph which is solved efficiently using online approximate message passing. The resulting algorithm has three key features: a) it requires only a single pass across the data and can hence be used on data streams, b) it maintains the uncertainty of parameters and cluster assignments, and c) it implements an automatic step size adaptation based on the current model uncertainty. The model is tested on an artificially generated toy dataset and applied to a large scale real-world data set from online advertising, the data being online ads characterized by the set of keywords to which they have been subscribed. The proposed approach scales well for large datasets, and compares favorably to other clustering algorithms on the ads dataset. As a concrete application to online advertising we show how the learned model can be used to recommend new keywords for given ads.}, author = {Schwaighofer, Anton and Candela, Joaquin Qui\ {n}onero and Borchert, Thomas and Graepel, Thore and Herbrich, Ralf}, booktitle = {Proceedings of 3rd Annual International Workshop on Data Mining and Audience Intelligence for Advertising}, keywords = {imported ralf.herbrich}, pages = {27–36}, title = {Scalable Clustering and Keyword Suggestion for Online Advertisements}, year = 2009 }

Novel Tools to Streamline the Conference Review Process: Experiences from SIGKDD’09. Flach, Peter; Spiegler, Sebastian; Golenia, Bruno; Price, Simon; Guiver, John; Herbrich, Ralf; Graepel, Thore; Zaki, Mohammed in SIGKDD Explorations (2009). 11(2) 63–67.
[ Abstract ] [ BibTeX ] [ URL ]
 
The SIGKDD'09 Research Track received 537 paper submissions, which were reviewed by a Program Committee of 199 members, and a Senior Program Committee of 22 members. We used techniques from artificial intelligence and data mining to streamline and support this complicated process at three crucial stages: bidding by PC members on papers, assigning papers to reviewers, and calibrating scores obtained from the reviews. In this paper we report on the approaches taken, evaluate how well they worked, and describe some further work done after the conference.
@article{flach2009novel, abstract = {The SIGKDD'09 Research Track received 537 paper submissions, which were reviewed by a Program Committee of 199 members, and a Senior Program Committee of 22 members. We used techniques from artificial intelligence and data mining to streamline and support this complicated process at three crucial stages: bidding by PC members on papers, assigning papers to reviewers, and calibrating scores obtained from the reviews. In this paper we report on the approaches taken, evaluate how well they worked, and describe some further work done after the conference.}, author = {Flach, Peter and Spiegler, Sebastian and Golenia, Bruno and Price, Simon and Guiver, John and Herbrich, Ralf and Graepel, Thore and Zaki, Mohammed}, journal = {SIGKDD Explorations}, keywords = {imported ralf.herbrich}, number = 2, pages = {63–67}, title = {Novel Tools to Streamline the Conference Review Process: Experiences from SIGKDD'09}, volume = 11, year = 2009 }

Matchbox: Large Scale Online Bayesian Recommendations. Stern, David; Herbrich, Ralf; Graepel, Thore (2009). 111–120.
[ Abstract ] [ BibTeX ] [ URL ]
 
We present a probabilistic model for generating personalised recommendations of items to users of a web service. The Matchbox system makes use of content information in the form of user and item meta data in combination with collaborative filtering information from previous user behavior in order to predict the value of an item for a user. Users and items are represented by feature vectors which are mapped into a low-dimensional trait space in which similarity is measured in terms of inner products. The model can be trained from different types of feedback in order to learn user-item preferences. Here we present three alternatives: direct observation of an absolute rating each user gives to some items, observation of a binary preference (like/ don't like) and observation of a set of ordinal ratings on a userspecific scale. Efficient inference is achieved by approximate message passing involving a combination of Expectation Propagation (EP) and Variational Message Passing. We also include a dynamics model which allows an item's popularity, a user's taste or a user's personal rating scale to drift over time. By using Assumed-Density Filtering (ADF) for training, the model requires only a single pass through the training data. This is an on-line learning algorithm capable of incrementally taking account of new data so the system can immediately reflect the latest user preferences. We evaluate the performance of the algorithm on the MovieLens and Netflix data sets consisting of approximately 1,000,000 and 100,000,000 ratings respectively. This demonstrates that training the model using the on-line ADF approach yields state-of-the-art performance with the option of improving performance further if computational resources are available by performing multiple EP passes over the training data.
@inproceedings{stern2009matchbox, abstract = {We present a probabilistic model for generating personalised recommendations of items to users of a web service. The Matchbox system makes use of content information in the form of user and item meta data in combination with collaborative filtering information from previous user behavior in order to predict the value of an item for a user. Users and items are represented by feature vectors which are mapped into a low-dimensional trait space in which similarity is measured in terms of inner products. The model can be trained from different types of feedback in order to learn user-item preferences. Here we present three alternatives: direct observation of an absolute rating each user gives to some items, observation of a binary preference (like/ don't like) and observation of a set of ordinal ratings on a userspecific scale. Efficient inference is achieved by approximate message passing involving a combination of Expectation Propagation (EP) and Variational Message Passing. We also include a dynamics model which allows an item's popularity, a user's taste or a user's personal rating scale to drift over time. By using Assumed-Density Filtering (ADF) for training, the model requires only a single pass through the training data. This is an on-line learning algorithm capable of incrementally taking account of new data so the system can immediately reflect the latest user preferences. We evaluate the performance of the algorithm on the MovieLens and Netflix data sets consisting of approximately 1,000,000 and 100,000,000 ratings respectively. This demonstrates that training the model using the on-line ADF approach yields state-of-the-art performance with the option of improving performance further if computational resources are available by performing multiple EP passes over the training data.}, author = {Stern, David and Herbrich, Ralf and Graepel, Thore}, booktitle = {Proceedings of the 18th International Conference on World Wide Web}, keywords = {imported ralf.herbrich}, pages = {111–120}, title = {{Matchbox}: Large Scale Online {Bayesian} Recommendations}, year = 2009 }

2008

Large Scale Data Analysis and Modelling in Online Services and Advertising. Graepel, Thore; Herbrich, Ralf (2008). 2.
[ BibTeX ] [ URL ]
 
@inproceedings{graepel2008large, author = {Graepel, Thore and Herbrich, Ralf}, booktitle = {Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining}, keywords = {imported ralf.herbrich}, pages = 2, title = {Large Scale Data Analysis and Modelling in Online Services and Advertising}, year = 2008 }

2007

TrueSkill Through Time: Revisiting the History of Chess. Dangauthier, Pierre; Herbrich, Ralf; Minka, Tom; Graepel, Thore (2007). 931–938.
[ Abstract ] [ BibTeX ] [ URL ]
 
We extend the Bayesian skill rating system TrueSkill to infer entire time series of skills of players by smoothing through time instead of filtering. The skill of each participating player, say, every year is represented by a latent skill variable which is affected by the relevant game outcomes that year, and coupled with the skill variables of the previous and subsequent year. Inference in the resulting factor graph is carried out by approximate message passing (EP) along the time series of skills. As before the system tracks the uncertainty about player skills, explicitly models draws, can deal with any number of competing entities and can infer individual skills from team results. We extend the system to estimate player-specific draw margins. Based on these models we present an analysis of the skill curves of important players in the history of chess over the past 150 years. Results include plots of players' lifetime skill development as well as the ability to compare the skills of different players across time. Our results indicate that a) the overall playing strength has increased over the past 150 years, and b) that modelling a player's ability to force a draw provides significantly better predictive power.
@inproceedings{dangauthier2007trueskill, abstract = {We extend the Bayesian skill rating system TrueSkill to infer entire time series of skills of players by smoothing through time instead of filtering. The skill of each participating player, say, every year is represented by a latent skill variable which is affected by the relevant game outcomes that year, and coupled with the skill variables of the previous and subsequent year. Inference in the resulting factor graph is carried out by approximate message passing (EP) along the time series of skills. As before the system tracks the uncertainty about player skills, explicitly models draws, can deal with any number of competing entities and can infer individual skills from team results. We extend the system to estimate player-specific draw margins. Based on these models we present an analysis of the skill curves of important players in the history of chess over the past 150 years. Results include plots of players' lifetime skill development as well as the ability to compare the skills of different players across time. Our results indicate that a) the overall playing strength has increased over the past 150 years, and b) that modelling a player's ability to force a draw provides significantly better predictive power.}, author = {Dangauthier, Pierre and Herbrich, Ralf and Minka, Tom and Graepel, Thore}, booktitle = {Advances in Neural Information Processing Systems 20}, keywords = {imported ralf.herbrich}, pages = {931–938}, publisher = {The MIT Press}, title = {TrueSkill Through Time: Revisiting the History of Chess}, year = 2007 }

Structure From Failure. Herbrich, Ralf; Graepel, Thore; Murphy, Brendan (2007).
[ Abstract ] [ BibTeX ] [ URL ]
 
We investigate the problem of learning the dependencies among servers in large networks based on failure patterns in their up-time behaviour. We model up-times in terms of exponential distributions whose inverse lifetime parameters lmay vary with the state of other servers. Based on a conjugate Gamma prior over inverse lifetimes we identify the most likely network configuration given that any node has at most one parent. The method can be viewed as a special case of learning a continuous time Bayesian network. Our approach enables us to easily incorporate existing expert prior knowledge. Furthermore our method enjoys advantages over a state-of-the-art rule-based approach. We validate the approach on synthetic data and apply it to five year data for a set of over 500 servers at a server farm of a major Microsoft web site.
@inproceedings{herbrich2007structure, abstract = {We investigate the problem of learning the dependencies among servers in large networks based on failure patterns in their up-time behaviour. We model up-times in terms of exponential distributions whose inverse lifetime parameters lmay vary with the state of other servers. Based on a conjugate Gamma prior over inverse lifetimes we identify the most likely network configuration given that any node has at most one parent. The method can be viewed as a special case of learning a continuous time Bayesian network. Our approach enables us to easily incorporate existing expert prior knowledge. Furthermore our method enjoys advantages over a state-of-the-art rule-based approach. We validate the approach on synthetic data and apply it to five year data for a set of over 500 servers at a server farm of a major Microsoft web site.}, author = {Herbrich, Ralf and Graepel, Thore and Murphy, Brendan}, booktitle = {Proceedings of 2nd Workshop on Tackling Computer Systems Problems with Machine Learning Techniques}, keywords = {imported ralf.herbrich}, publisher = {USENIX}, title = {Structure From Failure}, year = 2007 }

Learning to Solve Game Trees. Stern, David; Herbrich, Ralf; Graepel, Thore (2007). 839–846.
[ Abstract ] [ BibTeX ] [ URL ]
 
We apply probability theory to the task of proving whether a goal can be achieved by a player in an adversarial game. Such problems are solved by searching the game tree. We view this tree as a graphical model which yields a distribution over the (Boolean) outcome of the search before it terminates. Experiments show that a best-first search algorithm guided by this distribution explores a similar number of nodes as Proof-Number Search to solve Go problems. Knowledge is incorporated into search by using domain-specific models to provide prior distributions over the values of leaf nodes of the game tree. These are surrogate for the unexplored parts of the tree. The parameters of these models can be learned from previous search trees. Experiments on Go show that the speed of problem solving can be increased by orders of magnitude by this technique but care must be taken to avoid over-fitting.
@inproceedings{stern2007learning, abstract = {We apply probability theory to the task of proving whether a goal can be achieved by a player in an adversarial game. Such problems are solved by searching the game tree. We view this tree as a graphical model which yields a distribution over the (Boolean) outcome of the search before it terminates. Experiments show that a best-first search algorithm guided by this distribution explores a similar number of nodes as Proof-Number Search to solve Go problems. Knowledge is incorporated into search by using domain-specific models to provide prior distributions over the values of leaf nodes of the game tree. These are surrogate for the unexplored parts of the tree. The parameters of these models can be learned from previous search trees. Experiments on Go show that the speed of problem solving can be increased by orders of magnitude by this technique but care must be taken to avoid over-fitting.}, author = {Stern, David and Herbrich, Ralf and Graepel, Thore}, booktitle = {Proceedings of the 24th International Conference on Machine Learning}, keywords = {imported ralf.herbrich}, pages = {839–846}, title = {Learning to Solve Game Trees}, year = 2007 }

2006

TrueSkill(TM): A Bayesian Skill Rating System. Herbrich, Ralf; Minka, Tom; Graepel, Thore (2006). 569–576.
[ Abstract ] [ BibTeX ] [ URL ]
 
We present a new Bayesian skill rating system which can be viewed as a generalisation of the Elo system used in Chess. The new system tracks the uncertainty about player skills, explicitly models draws, can deal with any number of competing entities and can infer individual skills from team results. Inference is performed by approximate message passing on a factor graph representation of the model. We present experimental evidence on the increased accuracy and convergence speed of the system compared to Elo and report on our experience with the new rating system running in a large-scale commercial online gaming service under the name of TrueSkill.
@inproceedings{herbrich2006trueskilltm, abstract = {We present a new Bayesian skill rating system which can be viewed as a generalisation of the Elo system used in Chess. The new system tracks the uncertainty about player skills, explicitly models draws, can deal with any number of competing entities and can infer individual skills from team results. Inference is performed by approximate message passing on a factor graph representation of the model. We present experimental evidence on the increased accuracy and convergence speed of the system compared to Elo and report on our experience with the new rating system running in a large-scale commercial online gaming service under the name of TrueSkill.}, author = {Herbrich, Ralf and Minka, Tom and Graepel, Thore}, booktitle = {Advances in Neural Information Procesing Systems 19}, keywords = {imported ralf.herbrich}, pages = {569–576}, publisher = {The MIT Press}, title = {{TrueSkill(TM)}: A {Bayesian} Skill Rating System}, year = 2006 }

Ranking and Matchmaking. Graepel, Thore; Herbrich, Ralf in Game Developer Magazine (2006). (10)
[ Abstract ] [ BibTeX ] [ URL ]
 
Online game modes are gradually becoming the standard for console games, rather than an added bonus. But how do you get the right players playing against the right people, close enough in skill to be challenging, but not so difficult as to be frustrating?
@article{graepel2006ranking, abstract = {Online game modes are gradually becoming the standard for console games, rather than an added bonus. But how do you get the right players playing against the right people, close enough in skill to be challenging, but not so difficult as to be frustrating?}, author = {Graepel, Thore and Herbrich, Ralf}, journal = {Game Developer Magazine}, keywords = {imported ralf.herbrich}, number = 10, title = {Ranking and Matchmaking}, year = 2006 }

Bayesian Pattern Ranking for Move Prediction in the Game of Go. Stern, David; Herbrich, Ralf; Graepel, Thore (2006). 873–880.
[ Abstract ] [ BibTeX ] [ URL ]
 
We investigate the problem of learning to predict moves in the board game of Go from game records of expert players. In particular, we obtain a probability distribution over legal moves for professional play in a given position. This distribution has numerous applications in computer Go, including serving as an efficient stand-alone Go player. It would also be effective as a move selector and move sorter for game tree search and as a training tool for Go players. Our method has two major components: a) a pattern extraction scheme for efficiently harvesting patterns of given size and shape from expert game records and b) a Bayesian learning algorithm (in two variants) that learns a distribution over the values of a move given a board position based on the local pattern context. The system is trained on 181,000 expert games and shows excellent prediction performance as indicated by its ability to perfectly predict the moves made by professional Go players in 34% of test positions.
@inproceedings{stern2006bayesian, abstract = {We investigate the problem of learning to predict moves in the board game of Go from game records of expert players. In particular, we obtain a probability distribution over legal moves for professional play in a given position. This distribution has numerous applications in computer Go, including serving as an efficient stand-alone Go player. It would also be effective as a move selector and move sorter for game tree search and as a training tool for Go players. Our method has two major components: a) a pattern extraction scheme for efficiently harvesting patterns of given size and shape from expert game records and b) a Bayesian learning algorithm (in two variants) that learns a distribution over the values of a move given a board position based on the local pattern context. The system is trained on 181,000 expert games and shows excellent prediction performance as indicated by its ability to perfectly predict the moves made by professional Go players in 34\% of test positions.}, author = {Stern, David and Herbrich, Ralf and Graepel, Thore}, booktitle = {Proceedings of the 23rd International Conference on Machine Learning}, keywords = {imported ralf.herbrich}, pages = {873–880}, title = {{Bayesian} Pattern Ranking for Move Prediction in the Game of Go}, year = 2006 }

2005

The Structure of Version Space. Herbrich, Ralf; Graepel, Thore; Williamson, Robert C in Innovations in Machine Learning: Theory and Applications (2005). 257–274.
[ Abstract ] [ BibTeX ] [ URL ]
 
We investigate the generalisation performance of consistent classifiers, i.e. classifiers that are contained in the so-called version space, both from a theoretical and experimental angle. In contrast to classical VC analysis - where no single classifier within version space is singled out on grounds of a generalisation error bound - the data dependent structural risk minimisation framework suggests that there exists one particular classifier that is to be preferred because it minimises the generalisation error bound. This is usually taken to provide a theoretical justification for learning algorithms such as the well known support vector machine. A reinterpretation of a recent PAC-Bayesian result, however, reveals that given a suitably chosen hypothesis space there exists a large fraction of classifiers with small generalisation error although we cannot readily identify them for a specific learning task. In the particular case of linear classifiers we show that classifiers found by the classical perceptron algorithm have guarantees bounded by the size of version space. These results are complemented with an empirical study for kernel classifiers on the task of handwritten digit recognition which demonstrates that even classifiers with a small margin may exhibit excellent generalisation. In order to perform this analysis we introduce the kernel Gibbs sampler - an algorithm which can be used to sample consistent kernel classifiers.
@incollection{herbrich2005structure, abstract = {We investigate the generalisation performance of consistent classifiers, i.e. classifiers that are contained in the so-called version space, both from a theoretical and experimental angle. In contrast to classical VC analysis - where no single classifier within version space is singled out on grounds of a generalisation error bound - the data dependent structural risk minimisation framework suggests that there exists one particular classifier that is to be preferred because it minimises the generalisation error bound. This is usually taken to provide a theoretical justification for learning algorithms such as the well known support vector machine. A reinterpretation of a recent PAC-Bayesian result, however, reveals that given a suitably chosen hypothesis space there exists a large fraction of classifiers with small generalisation error although we cannot readily identify them for a specific learning task. In the particular case of linear classifiers we show that classifiers found by the classical perceptron algorithm have guarantees bounded by the size of version space. These results are complemented with an empirical study for kernel classifiers on the task of handwritten digit recognition which demonstrates that even classifiers with a small margin may exhibit excellent generalisation. In order to perform this analysis we introduce the kernel Gibbs sampler - an algorithm which can be used to sample consistent kernel classifiers.}, author = {Herbrich, Ralf and Graepel, Thore and Williamson, Robert C}, booktitle = {Innovations in Machine Learning: Theory and Applications}, chapter = 9, keywords = {imported ralf.herbrich}, pages = {257–274}, publisher = {Springer}, title = {The Structure of Version Space}, year = 2005 }

PAC-Bayesian Compression Bounds on the Prediction Error of Learning Algorithms for Classification. Graepel, Thore; Herbrich, Ralf; Shawe-Taylor, John in Machine Learning (2005). 59 55–76.
[ Abstract ] [ BibTeX ] [ URL ]
 
We consider bounds on the prediction error of classification algorithms based on sample compression. We refine the notion of a compression scheme to distinguish permutation and repetition invariant and non-permutation and repetition invariant compression schemes leading to different prediction error bounds. Also, we extend known results on compression to the case of non-zero empirical risk. We provide bounds on the prediction error of classifiers returned by mistakedriven online learning algorithms by interpreting mistake bounds as bounds on the size of the respective compression scheme of the algorithm. This leads to a bound on the prediction error of perceptron solutions that depends on the margin a support vector machine would achieve on the same training sample. Furthermore, using the property of compression we derive bounds on the average prediction error of kernel classifiers in the PAC-Bayesian framework. These bounds assume a prior measure over the expansion coefficients in the data-dependent kernel expansion and bound the average prediction error uniformly over subsets of the space of expansion coefficients.
@article{graepel2005pacbayesian, abstract = {We consider bounds on the prediction error of classification algorithms based on sample compression. We refine the notion of a compression scheme to distinguish permutation and repetition invariant and non-permutation and repetition invariant compression schemes leading to different prediction error bounds. Also, we extend known results on compression to the case of non-zero empirical risk. We provide bounds on the prediction error of classifiers returned by mistakedriven online learning algorithms by interpreting mistake bounds as bounds on the size of the respective compression scheme of the algorithm. This leads to a bound on the prediction error of perceptron solutions that depends on the margin a support vector machine would achieve on the same training sample. Furthermore, using the property of compression we derive bounds on the average prediction error of kernel classifiers in the PAC-Bayesian framework. These bounds assume a prior measure over the expansion coefficients in the data-dependent kernel expansion and bound the average prediction error uniformly over subsets of the space of expansion coefficients.}, author = {Graepel, Thore and Herbrich, Ralf and Shawe-Taylor, John}, journal = {Machine Learning}, keywords = {imported ralf.herbrich}, pages = {55–76}, title = {{PAC-Bayesian} Compression Bounds on the Prediction Error of Learning Algorithms for Classification}, volume = 59, year = 2005 }

On Gaussian Expectation Propagation Herbrich, Ralf (2005).
[ Abstract ] [ BibTeX ] [ URL ]
 
In this short note we will re-derive the Gaussian expectation propagation (Gaussian EP) algorithm as presented in Minka (2001) and demonstrate an application of Gaussian EP to approximate multi-dimensional truncated Gaussians.
@techreport{herbrich2005gaussian, abstract = {In this short note we will re-derive the Gaussian expectation propagation (Gaussian EP) algorithm as presented in Minka (2001) and demonstrate an application of Gaussian EP to approximate multi-dimensional truncated Gaussians.}, author = {Herbrich, Ralf}, institution = {Microsoft Research}, keywords = {imported ralf.herbrich}, title = {On {Gaussian} Expectation Propagation}, year = 2005 }

Minimising the Kullback-Leibler Divergence Herbrich, Ralf (2005).
[ Abstract ] [ BibTeX ] [ URL ]
 
In this note we show that minimising the Kullback--Leibler divergence over a family in the class of exponential distributions is achieved by matching the expected natural statistic. We will also give an explicit update formula for distributions with only one likelihood term.
@techreport{herbrich2005minimising, abstract = {In this note we show that minimising the Kullback–Leibler divergence over a family in the class of exponential distributions is achieved by matching the expected natural statistic. We will also give an explicit update formula for distributions with only one likelihood term.}, author = {Herbrich, Ralf}, institution = {Microsoft Research}, keywords = {imported ralf.herbrich}, title = {{Minimising the Kullback-Leibler Divergence}}, year = 2005 }

Kernel Methods for Measuring Independence. Gretton, Arthur; Herbrich, Ralf; Smola, Alexander J; Bousquet, Olivier; Schölkopf, Bernhard in Journal of Machine Learning Research (2005). 6 2075–2129.
[ Abstract ] [ BibTeX ] [ URL ]
 
We introduce two new functionals, the constrained covariance and the kernel mutual information, to measure the degree of independence of random variables. These quantities are both based on the covariance between functions of the random variables in reproducing kernel Hilbert spaces (RKHSs). We prove that when the RKHSs are universal, both functionals are zero if and only if the random variables are pairwise independent. We also show that the kernel mutual information is an upper bound near independence on the Parzen window estimate of the mutual information. Analogous results apply for two correlation-based dependence functionals introduced earlier: we show the kernel canonical correlation and the kernel generalised variance to be independence measures for universal kernels, and prove the latter to be an upper bound on the mutual information near independence. The performance of the kernel dependence functionals in measuring independence is verified in the context of independent component analysis.
@article{gretton2005kernel, abstract = {We introduce two new functionals, the constrained covariance and the kernel mutual information, to measure the degree of independence of random variables. These quantities are both based on the covariance between functions of the random variables in reproducing kernel Hilbert spaces (RKHSs). We prove that when the RKHSs are universal, both functionals are zero if and only if the random variables are pairwise independent. We also show that the kernel mutual information is an upper bound near independence on the Parzen window estimate of the mutual information. Analogous results apply for two correlation-based dependence functionals introduced earlier: we show the kernel canonical correlation and the kernel generalised variance to be independence measures for universal kernels, and prove the latter to be an upper bound on the mutual information near independence. The performance of the kernel dependence functionals in measuring independence is verified in the context of independent component analysis.}, author = {Gretton, Arthur and Herbrich, Ralf and Smola, Alexander J and Bousquet, Olivier and Sch{ö}lkopf, Bernhard}, journal = {Journal of Machine Learning Research}, keywords = {imported ralf.herbrich}, pages = {2075–2129}, title = {Kernel Methods for Measuring Independence}, volume = 6, year = 2005 }

Kernel Constrained Covariance for Dependence Measurement. Gretton, Arthur; Smola, Alexander; Bousquet, Olivier; Herbrich, Ralf; Belitski, Andrei; Augath, Mark; Murayama, Yusuke; Pauls, Jon; Schölkopf, Bernhard; Logothetis, Nikos (2005). 112–119.
[ Abstract ] [ BibTeX ] [ URL ]
 
We discuss reproducing kernel Hilbert space (RKHS)-based measures of statistical dependence, with emphasis on constrained covariance (COCO), a novel criterion to test dependence of random variables. We show that COCO is a test for independence if and only if the associated RKHSs are universal. That said, no independence test exists that can distinguish dependent and independent random variables in all circumstances. Dependent random variables can result in a COCO which is arbitrarily close to zero when the source densities are highly non-smooth. All current kernel-based independence tests share this behaviour. We demonstrate exponential convergence between the population and empirical COCO. Finally, we use COCO as a measure of joint neural activity between voxels in MRI recordings of the macaque monkey, and compare the results to the mutual information and the correlation. We also show the effect of removing breathing artefacts from the MRI recording.
@inproceedings{gretton2005kernel, abstract = {We discuss reproducing kernel Hilbert space (RKHS)-based measures of statistical dependence, with emphasis on constrained covariance (COCO), a novel criterion to test dependence of random variables. We show that COCO is a test for independence if and only if the associated RKHSs are universal. That said, no independence test exists that can distinguish dependent and independent random variables in all circumstances. Dependent random variables can result in a COCO which is arbitrarily close to zero when the source densities are highly non-smooth. All current kernel-based independence tests share this behaviour. We demonstrate exponential convergence between the population and empirical COCO. Finally, we use COCO as a measure of joint neural activity between voxels in MRI recordings of the macaque monkey, and compare the results to the mutual information and the correlation. We also show the effect of removing breathing artefacts from the MRI recording.}, author = {Gretton, Arthur and Smola, Alexander and Bousquet, Olivier and Herbrich, Ralf and Belitski, Andrei and Augath, Mark and Murayama, Yusuke and Pauls, Jon and Sch{ö}lkopf, Bernhard and Logothetis, Nikos}, booktitle = {Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics (AISTATS)}, keywords = {imported ralf.herbrich}, pages = {112–119}, title = {Kernel Constrained Covariance for Dependence Measurement}, year = 2005 }

Generalization Bounds for the Area Under the ROC Curve. Agarwal, Shivani; Graepel, Thore; Herbrich, Ralf; Har-Peled, Sariel; Roth, Dan in Journal of Machine Learning Research (2005). 6 393–425.
[ Abstract ] [ BibTeX ] [ URL ]
 
We study generalization properties of the area under the ROC curve (AUC), a quantity that has been advocated as an evaluation criterion for the bipartite ranking problem. The AUC is a different term than the error rate used for evaluation in classification problems; consequently, existing generalization bounds for the classification error rate cannot be used to draw conclusions about the AUC. In this paper, we define the expected accuracy of a ranking function (analogous to the expected error rate of a classification function), and derive distribution-free probabilistic bounds on the deviation of the empirical AUC of a ranking function (observed on a finite data sequence) from its expected accuracy. We derive both a large deviation bound, which serves to bound the expected accuracy of a ranking function in terms of its empirical AUC on a test sequence, and a uniform convergence bound, which serves to bound the expected accuracy of a learned ranking function in terms of its empirical AUC on a training sequence. Our uniform convergence bound is expressed in terms of a new set of combinatorial parameters that we term the bipartite rank-shatter coefficients; these play the same role in our result as do the standard VC-dimension related shatter coefficients (also known as the growth function) in uniform convergence results for the classification error rate. A comparison of our result with a recent uniform convergence result derived by Freund et al. (2003) for a quantity closely related to the AUC shows that the bound provided by our result can be considerably tighter.
@article{agarwal2005generalization, abstract = {We study generalization properties of the area under the ROC curve (AUC), a quantity that has been advocated as an evaluation criterion for the bipartite ranking problem. The AUC is a different term than the error rate used for evaluation in classification problems; consequently, existing generalization bounds for the classification error rate cannot be used to draw conclusions about the AUC. In this paper, we define the expected accuracy of a ranking function (analogous to the expected error rate of a classification function), and derive distribution-free probabilistic bounds on the deviation of the empirical AUC of a ranking function (observed on a finite data sequence) from its expected accuracy. We derive both a large deviation bound, which serves to bound the expected accuracy of a ranking function in terms of its empirical AUC on a test sequence, and a uniform convergence bound, which serves to bound the expected accuracy of a learned ranking function in terms of its empirical AUC on a training sequence. Our uniform convergence bound is expressed in terms of a new set of combinatorial parameters that we term the bipartite rank-shatter coefficients; these play the same role in our result as do the standard VC-dimension related shatter coefficients (also known as the growth function) in uniform convergence results for the classification error rate. A comparison of our result with a recent uniform convergence result derived by Freund et al. (2003) for a quantity closely related to the AUC shows that the bound provided by our result can be considerably tighter.}, author = {Agarwal, Shivani and Graepel, Thore and Herbrich, Ralf and Har-Peled, Sariel and Roth, Dan}, journal = {Journal of Machine Learning Research}, keywords = {imported ralf.herbrich}, pages = {393–425}, title = {Generalization Bounds for the Area Under the {ROC} Curve}, volume = 6, year = 2005 }

2004

Poisson-Networks : A Model for Structured Poisson Processes. Rajaram, Shyamsundar; Graepel, Thore; Herbrich, Ralf (2004). 277–284.
[ Abstract ] [ BibTeX ] [ URL ]
 
Modelling structured multivariate point process data has wide ranging applications like understanding neural activity, developing faster file access systems and learning dependencies among servers in large networks. In this paper, we develop the Poisson network model for representing multivariate structured Poisson processes. In our model each node of the network represents a Poisson process. The novelty of our work is that waiting times of a process are modelled by an exponential distribution with a piecewise constant rate function that depends on the event counts of its parents in the network in a generalised linear way. Our choice of model allows to perform exact sampling from arbitrary structures. We adopt a Bayesian approach for learning the network structure. Further, we discuss fixed point and sampling based approximations for performing inference of rate functions in Poisson networks.
@inproceedings{rajaram2004poissonnetworks, abstract = {Modelling structured multivariate point process data has wide ranging applications like understanding neural activity, developing faster file access systems and learning dependencies among servers in large networks. In this paper, we develop the Poisson network model for representing multivariate structured Poisson processes. In our model each node of the network represents a Poisson process. The novelty of our work is that waiting times of a process are modelled by an exponential distribution with a piecewise constant rate function that depends on the event counts of its parents in the network in a generalised linear way. Our choice of model allows to perform exact sampling from arbitrary structures. We adopt a Bayesian approach for learning the network structure. Further, we discuss fixed point and sampling based approximations for performing inference of rate functions in Poisson networks.}, author = {Rajaram, Shyamsundar and Graepel, Thore and Herbrich, Ralf}, booktitle = {Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics}, keywords = {imported ralf.herbrich}, pages = {277–284}, title = {{Poisson}-Networks : A Model for Structured {Poisson} Processes}, year = 2004 }

Learning to Fight. Graepel, Thore; Herbrich, Ralf; Gold, Julian (2004). 193–200.
[ Abstract ] [ BibTeX ] [ URL ]
 
We apply reinforcement learning to the problem of finding good policies for a fighting agent in a commercial computer game. The learning agent is trained using the SARSA algorithm for on-policy learning of an action-value function represented by linear and neural network function approximators. We discuss the selection and construction of features, actions, and rewards as well as other design choices necessary to integrate the learning process into the game. The learning agent is trained against the built-in AI of the game with different rewards encouraging aggressive or defensive behaviour. We show that the learning agent finds interesting (and partly near optimal) policies in accordance with the reward functions provided. We also discuss the particular challenges arising in the application of reinforcement learning to the domain of computer games.
@inproceedings{graepel2004learning, abstract = {We apply reinforcement learning to the problem of finding good policies for a fighting agent in a commercial computer game. The learning agent is trained using the SARSA algorithm for on-policy learning of an action-value function represented by linear and neural network function approximators. We discuss the selection and construction of features, actions, and rewards as well as other design choices necessary to integrate the learning process into the game. The learning agent is trained against the built-in AI of the game with different rewards encouraging aggressive or defensive behaviour. We show that the learning agent finds interesting (and partly near optimal) policies in accordance with the reward functions provided. We also discuss the particular challenges arising in the application of reinforcement learning to the domain of computer games.}, author = {Graepel, Thore and Herbrich, Ralf and Gold, Julian}, booktitle = {Proceedings of the International Conference on Computer Games: Artificial Intelligence, Design and Education}, keywords = {imported ralf.herbrich}, pages = {193–200}, title = {Learning to Fight}, year = 2004 }

A Large Deviation Bound for the Area Under the ROC Curve. Agarwal, Shivani; Graepel, Thore; Herbrich, Ralf; Roth, Dan (2004). 9–16.
[ Abstract ] [ BibTeX ] [ URL ]
 
The area under the ROC curve (AUC) has been advocated as an evaluation criterion for the bipartite ranking problem. We study large deviation properties of the AUC; in particular, we derive a distribution-free large deviation bound for the AUC which serves to bound the expected accuracy of a ranking function in terms of its empirical AUC on an independent test sequence. A comparison of our result with a corresponding large deviation result for the classification error rate suggests that the test sample size required to obtain an epsilon-accurate estimate of the expected accuracy of a ranking function with delta-confidence is larger than that required to obtain an epsilon-accurate estimate of the expected error rate of a classification function with the same confidence. A simple application of the union bound allows the large deviation bound to be extended to learned ranking functions chosen from finite function classes.
@inproceedings{agarwal2004large, abstract = {The area under the ROC curve (AUC) has been advocated as an evaluation criterion for the bipartite ranking problem. We study large deviation properties of the AUC; in particular, we derive a distribution-free large deviation bound for the AUC which serves to bound the expected accuracy of a ranking function in terms of its empirical AUC on an independent test sequence. A comparison of our result with a corresponding large deviation result for the classification error rate suggests that the test sample size required to obtain an epsilon-accurate estimate of the expected accuracy of a ranking function with delta-confidence is larger than that required to obtain an epsilon-accurate estimate of the expected error rate of a classification function with the same confidence. A simple application of the union bound allows the large deviation bound to be extended to learned ranking functions chosen from finite function classes.}, author = {Agarwal, Shivani and Graepel, Thore and Herbrich, Ralf and Roth, Dan}, booktitle = {Advances in Neural Information Processing Systems 17}, keywords = {imported ralf.herbrich}, pages = {9–16}, title = {A Large Deviation Bound for the Area Under the {ROC} Curve}, year = 2004 }

2003

The Kernel Mutual Information. Gretton, Arthur; Herbrich, Ralf; Smola, Alexander (2003). 880–883.
[ Abstract ] [ BibTeX ] [ URL ]
 
We introduce a new contrast function, the kernel mutual information (KMI), to measure the degree of independence of continuous random variables. This contrast function provides an approximate upper bound on the mutual information, as measured near independence, and is based on a kernel density estimate of the mutual information between a discretised approximation of the continuous random variables. We show that Bach and Jordan's kernel generalised variance (KGV) is also an upper bound on the same kernel density estimate, but is looser. Finally, we suggest that the addition of a regularising term in the KGV causes it to approach the KMI, which motivates the introduction of this regularisation.
@inproceedings{gretton2003kernel, abstract = {We introduce a new contrast function, the kernel mutual information (KMI), to measure the degree of independence of continuous random variables. This contrast function provides an approximate upper bound on the mutual information, as measured near independence, and is based on a kernel density estimate of the mutual information between a discretised approximation of the continuous random variables. We show that Bach and Jordan's kernel generalised variance (KGV) is also an upper bound on the same kernel density estimate, but is looser. Finally, we suggest that the addition of a regularising term in the KGV causes it to approach the KMI, which motivates the introduction of this regularisation.}, author = {Gretton, Arthur and Herbrich, Ralf and Smola, Alexander}, booktitle = {Proceedings of IEEE Internaltional Conference on Acoustics, Speech and Signal Processing}, keywords = {imported ralf.herbrich}, pages = {880–883}, title = {The Kernel Mutual Information}, year = 2003 }

Semi-Definite Programming by Perceptron Learning. Graepel, Thore; Herbrich, Ralf; Kharechko, Andriy; Shawe-Taylor, John (2003). 457–464.
[ Abstract ] [ BibTeX ] [ URL ]
 
We present a modified version of the perceptron learning algorithm (PLA) which solves semidefinite programs (SDPs) in polynomial time. The algorithm is based on the following three observations: (i) Semidefinite programs are linear programs with infinitely many (linear) constraints; (ii) every linear program can be solved by a sequence of constraint satisfaction problems with linear constraints; (iii) in general, the perceptron learning algorithm solves a constraint satisfaction problem with linear constraints in finitely many updates. Combining the PLA with a probabilistic rescaling algorithm (which, on average, increases the size of the feasable region) results in a probabilistic algorithm for solving SDPs that runs in polynomial time. We present preliminary results which demonstrate that the algorithm works, but is not competitive with state-of-the-art interior point methods.
@inproceedings{graepel2003semidefinite, abstract = {We present a modified version of the perceptron learning algorithm (PLA) which solves semidefinite programs (SDPs) in polynomial time. The algorithm is based on the following three observations: (i) Semidefinite programs are linear programs with infinitely many (linear) constraints; (ii) every linear program can be solved by a sequence of constraint satisfaction problems with linear constraints; (iii) in general, the perceptron learning algorithm solves a constraint satisfaction problem with linear constraints in finitely many updates. Combining the PLA with a probabilistic rescaling algorithm (which, on average, increases the size of the feasable region) results in a probabilistic algorithm for solving SDPs that runs in polynomial time. We present preliminary results which demonstrate that the algorithm works, but is not competitive with state-of-the-art interior point methods.}, author = {Graepel, Thore and Herbrich, Ralf and Kharechko, Andriy and Shawe-Taylor, John}, booktitle = {Advances in Neural Information Processing Systems 16}, keywords = {imported ralf.herbrich}, pages = {457–464}, publisher = {The MIT Press}, title = {Semi-Definite Programming by Perceptron Learning}, year = 2003 }

Online Bayes Point Machines. Harrington, Edward; Herbrich, Ralf; Kivinen, Jyrki; Platt, John C; Williamson, Robert C (2003). 241–252.
[ Abstract ] [ BibTeX ] [ URL ]
 
We present a new and simple algorithm for learning large margin classifiers that works in a truly online manner. The algorithm generates a linear classifier by averaging the weights associated with several perceptron-like algorithms run in parallel in order to approximate the Bayes point. A random subsample of the incoming data stream is used to ensure diversity in the perceptron solutions. We experimentally study the algorithm's performance on online and batch learning settings. The online experiments showed that our algorithm produces a low prediction error on the training sequence and tracks the presence of concept drift. On the batch problems its performance is comparable to the maximum margin algorithm which explicitly maximises the margin.
@inproceedings{harrington2003online, abstract = {We present a new and simple algorithm for learning large margin classifiers that works in a truly online manner. The algorithm generates a linear classifier by averaging the weights associated with several perceptron-like algorithms run in parallel in order to approximate the Bayes point. A random subsample of the incoming data stream is used to ensure diversity in the perceptron solutions. We experimentally study the algorithm's performance on online and batch learning settings. The online experiments showed that our algorithm produces a low prediction error on the training sequence and tracks the presence of concept drift. On the batch problems its performance is comparable to the maximum margin algorithm which explicitly maximises the margin.}, author = {Harrington, Edward and Herbrich, Ralf and Kivinen, Jyrki and Platt, John C and Williamson, Robert C}, booktitle = {Proceedings of Advances in Knowledge Discovery and Data Mining}, keywords = {imported ralf.herbrich}, pages = {241–252}, title = {Online {Bayes} Point Machines}, year = 2003 }

Invariant Pattern Recognition by Semidefinite Programming Machines. Graepel, Thore; Herbrich, Ralf (2003). 33–40.
[ Abstract ] [ BibTeX ] [ URL ]
 
Knowledge about local invariances with respect to given pattern transformations can greatly improve the accuracy of classification. Previous approaches are either based on regularisation or on the generation of virtual (transformed) examples. We develop a new framework for learning linear classifiers under known transformations based on semidefinite programming. We present a new learning algorithm - the Semidefinite Programming Machine (SDPM) - which is able to find a maximum margin hyperplane when the training examples are polynomial trajectories instead of single points. The solution is found to be sparse in dual variables and allows to identify those points on the trajectory with minimal real-valued output as virtual support vectors. Extensions to segments of trajectories, to more than one transformation parameter, and to learning with kernels are discussed. In experiments we use a Taylor expansion to locally approximate rotational invariance in pixel images from USPS and find improvements over known methods.
@inproceedings{graepel2003invariant, abstract = {Knowledge about local invariances with respect to given pattern transformations can greatly improve the accuracy of classification. Previous approaches are either based on regularisation or on the generation of virtual (transformed) examples. We develop a new framework for learning linear classifiers under known transformations based on semidefinite programming. We present a new learning algorithm - the Semidefinite Programming Machine (SDPM) - which is able to find a maximum margin hyperplane when the training examples are polynomial trajectories instead of single points. The solution is found to be sparse in dual variables and allows to identify those points on the trajectory with minimal real-valued output as virtual support vectors. Extensions to segments of trajectories, to more than one transformation parameter, and to learning with kernels are discussed. In experiments we use a Taylor expansion to locally approximate rotational invariance in pixel images from USPS and find improvements over known methods.}, author = {Graepel, Thore and Herbrich, Ralf}, booktitle = {Advances in Neural Information Processing Systems 16}, keywords = {imported ralf.herbrich}, pages = {33–40}, title = {Invariant Pattern Recognition by Semidefinite Programming Machines}, year = 2003 }

Introduction to the Special Issue on Learning Theory. Herbrich, Ralf; Graepel, Thore in Journal of Machine Learning Research (2003). 4 755–757.
[ BibTeX ] [ URL ]
 
@article{herbrich2003introduction, author = {Herbrich, Ralf and Graepel, Thore}, journal = {Journal of Machine Learning Research}, keywords = {imported ralf.herbrich}, pages = {755–757}, title = {Introduction to the Special Issue on Learning Theory}, volume = 4, year = 2003 }

2002

The Perceptron Algorithm with Uneven Margins. Li, Yaoyong; Zaragoza, Hugo; Herbrich, Ralf; Shawe-Taylor, John; Kandola, Jasvinder (2002). 379–386.
[ Abstract ] [ BibTeX ] [ URL ]
 
The perceptron algorithm with margins is a simple, fast and effective learning algorithm for linear classifiers; it produces decision hyperplanes within some constant ratio of the maximal margin. In this paper we study this algorithm and a new variant: the perceptron algorithm with uneven margins, tailored for document categorisation problems (i.e. problems where classes are highly unbalanced and performance depends on the ranking of patterns). We discuss the interest of these algorithms from a theoretical point of view, provide a generalisation of Novikoff's theorem for uneven margins, give a geometrically description of these algorithms and show experimentally that both algorithms yield equal or better performances than support vector machines, while reducing training time and sparsity, in classification (USPS) and document categorisation (Reuters) problems.
@inproceedings{li2002perceptron, abstract = {The perceptron algorithm with margins is a simple, fast and effective learning algorithm for linear classifiers; it produces decision hyperplanes within some constant ratio of the maximal margin. In this paper we study this algorithm and a new variant: the perceptron algorithm with uneven margins, tailored for document categorisation problems (i.e. problems where classes are highly unbalanced and performance depends on the ranking of patterns). We discuss the interest of these algorithms from a theoretical point of view, provide a generalisation of Novikoff's theorem for uneven margins, give a geometrically description of these algorithms and show experimentally that both algorithms yield equal or better performances than support vector machines, while reducing training time and sparsity, in classification (USPS) and document categorisation (Reuters) problems.}, author = {Li, Yaoyong and Zaragoza, Hugo and Herbrich, Ralf and Shawe-Taylor, John and Kandola, Jasvinder}, booktitle = {Proceedings of the 19th International Conference of Machine Learning}, keywords = {imported ralf.herbrich}, pages = {379–386}, title = {The Perceptron Algorithm with Uneven Margins}, year = 2002 }

Microsoft Cambridge at TREC 2002: Filtering Track. Robertson, Stephen E.; Walker, Stephen; Zaragoza, Hugo; Herbrich, Ralf (2002). 361–368.
[ Abstract ] [ BibTeX ] [ URL ]
 
Six runs were submitted for the Adaptive Filtering track, four on the adaptive filtering task (ok11af??), and two on the routing task (msPUM?). The adaptive filtering system has been somewhat modified from the one used for TREC-10, largely for efficiently and flexibility reasons; the basic filtering algorithms remain similar to those used in recent TRECs. For the routing task, a completely new system based on perceptrons with uneven margins was used.
@inproceedings{robertson2002microsoft, abstract = {Six runs were submitted for the Adaptive Filtering track, four on the adaptive filtering task (ok11af??), and two on the routing task (msPUM?). The adaptive filtering system has been somewhat modified from the one used for TREC-10, largely for efficiently and flexibility reasons; the basic filtering algorithms remain similar to those used in recent TRECs. For the routing task, a completely new system based on perceptrons with uneven margins was used.}, author = {Robertson, Stephen E. and Walker, Stephen and Zaragoza, Hugo and Herbrich, Ralf}, booktitle = {Proceedings of the 9th Text Retrieval Conference (TREC-9)}, keywords = {imported ralf.herbrich}, pages = {361–368}, title = {{Microsoft Cambridge} at {TREC} 2002: Filtering Track}, year = 2002 }

Learning Kernel Classifiers: Theory and Algorithms Herbrich, Ralf (2002). (2nd edition ) The MIT Press.
[ BibTeX ]
 
@book{herbrich2002learning, author = {Herbrich, Ralf}, edition = {2nd edition}, keywords = {imported ralf.herbrich}, publisher = {The MIT Press}, title = {Learning Kernel Classifiers: Theory and Algorithms}, year = 2002 }

Learning and Generalization: Theoretical Bounds. Herbrich, Ralf; Williamson, Robert C in Handbook of Brain Theory and Neural Networks (2002). (2nd edition ) 619–623.
[ BibTeX ] [ URL ]
 
@incollection{herbrich2002learning, author = {Herbrich, Ralf and Williamson, Robert C}, booktitle = {Handbook of Brain Theory and Neural Networks}, edition = {2nd edition}, keywords = {imported ralf.herbrich}, pages = {619–623}, publisher = {The MIT Press}, title = {Learning and Generalization: Theoretical Bounds}, year = 2002 }

Fast Sparse Gaussian Process Methods: The Informative Vector Machine. Lawrence, Neil; Seeger, Matthias; Herbrich, Ralf (2002). 609–616.
[ Abstract ] [ BibTeX ] [ URL ]
 
We present a framework for sparse Gaussian process (GP) methods which uses forward selection with criteria based on information-theoretical principles, previously suggested for active learning. In contrast to most previous work on sparse GPs, our goal is not only to learn sparse predictors (which can be evaluated in \($O(d)$\) rather than \($O(n)$\), \($d \ll n$\), \($n$\) the number of training points), but also to perform training under strong restrictions on time and memory requirements. The scaling of our method is at most \($O(n \cdot d^2)$\), and in large real-world classification experiments we show that it can match prediction performance of the popular support vector machine (SVM), yet it requires only a fraction of the training time. In contrast to the SVM, our approximation produces estimates of predictive probabilities ("error bars"), allows for Bayesian model selection and is less complex in implementation.
@inproceedings{lawrence2002sparse, abstract = {We present a framework for sparse Gaussian process (GP) methods which uses forward selection with criteria based on information-theoretical principles, previously suggested for active learning. In contrast to most previous work on sparse GPs, our goal is not only to learn sparse predictors (which can be evaluated in $O(d)$ rather than $O(n)$, $d \ll n$, $n$ the number of training points), but also to perform training under strong restrictions on time and memory requirements. The scaling of our method is at most $O(n \cdot d^2)$, and in large real-world classification experiments we show that it can match prediction performance of the popular support vector machine (SVM), yet it requires only a fraction of the training time. In contrast to the SVM, our approximation produces estimates of predictive probabilities ("error bars"), allows for Bayesian model selection and is less complex in implementation.}, author = {Lawrence, Neil and Seeger, Matthias and Herbrich, Ralf}, booktitle = {Advances in Neural Information Processing Systems 15}, keywords = {imported ralf.herbrich}, pages = {609–616}, title = {Fast Sparse {Gaussian} Process Methods: The Informative Vector Machine}, year = 2002 }

Average Precision and the Problem of Generalisation. Hill, Simon; Zaragoza, Hugo; Herbrich, Ralf; Rayner, Peter (2002).
[ Abstract ] [ BibTeX ] [ URL ]
 
In this paper we study the problem of generalisation in information retrieval. In particular we study precision-recall curves and the average precision value. We provide two types of bounds: large-deviation bounds of the average precision and maximum deviation bounds with respect to a given point of the precision recall curve. The first type of bounds are useful to answer the question: how far can true average precision be from the value observed on a test collection? The second is useful for obtaining bounds on average precision when tight bounds on a particular point of the curve can be established, as is the case when training SVMs or Perceptrons for document categorisation.
@inproceedings{hill2002average, abstract = {In this paper we study the problem of generalisation in information retrieval. In particular we study precision-recall curves and the average precision value. We provide two types of bounds: large-deviation bounds of the average precision and maximum deviation bounds with respect to a given point of the precision recall curve. The first type of bounds are useful to answer the question: how far can true average precision be from the value observed on a test collection? The second is useful for obtaining bounds on average precision when tight bounds on a particular point of the curve can be established, as is the case when training SVMs or Perceptrons for document categorisation.}, author = {Hill, Simon and Zaragoza, Hugo and Herbrich, Ralf and Rayner, Peter}, booktitle = {Proceedings of the ACM SIGIR Workshop on Mathematical and Formal Methods in Information Retrieval}, keywords = {imported ralf.herbrich}, title = {Average Precision and the Problem of Generalisation}, year = 2002 }

Algorithmic Luckiness. Herbrich, Ralf; Williamson, Robert C in Journal of Machine Learning Research (2002). 3 175–212.
[ Abstract ] [ BibTeX ] [ URL ]
 
Classical statistical learning theory studies the generalisation performance of machine learning algorithms rather indirectly. One of the main detours is that algorithms are studied in terms of the hypothesis class that they draw their hypotheses from. In this paper, motivated by the luckiness framework of Shawe-Taylor et al. (1998), we study learning algorithms more directly and in a way that allows us to exploit the serendipity of the training sample. The main difference to previous approaches lies in the complexity measure; rather than covering all hypotheses in a given hypothesis space it is only necessary to cover the functions which could have been learned using the fixed learning algorithm. We show how the resulting framework relates to the VC, luckiness and compression frameworks. Finally, we present an application of this framework to the maximum margin algorithm for linear classifiers which results in a bound that exploits the margin, the sparsity of the resultant weight vector, and the degree of clustering of the training data in feature space.
@article{herbrich2002algorithmic, abstract = {Classical statistical learning theory studies the generalisation performance of machine learning algorithms rather indirectly. One of the main detours is that algorithms are studied in terms of the hypothesis class that they draw their hypotheses from. In this paper, motivated by the luckiness framework of Shawe-Taylor et al. (1998), we study learning algorithms more directly and in a way that allows us to exploit the serendipity of the training sample. The main difference to previous approaches lies in the complexity measure; rather than covering all hypotheses in a given hypothesis space it is only necessary to cover the functions which could have been learned using the fixed learning algorithm. We show how the resulting framework relates to the VC, luckiness and compression frameworks. Finally, we present an application of this framework to the maximum margin algorithm for linear classifiers which results in a bound that exploits the margin, the sparsity of the resultant weight vector, and the degree of clustering of the training data in feature space.}, author = {Herbrich, Ralf and Williamson, Robert C}, journal = {Journal of Machine Learning Research}, keywords = {imported ralf.herbrich}, pages = {175–212}, title = {Algorithmic Luckiness}, volume = 3, year = 2002 }

A PAC-Bayesian Margin Bound for Linear Classifiers. Herbrich, Ralf; Graepel, Thore in IEEE Transactions on Information Theory (2002). 48(12) 3140–3150.
[ Abstract ] [ BibTeX ] [ URL ]
 
We present a bound on the generalisation error of linear classifiers in terms of a refined margin quantity on the training sample. The result is obtained in a PAC-Bayesian framework and is based on geometrical arguments in the space of linear classifiers. The new bound constitutes an exponential improvement of the so far tightest margin bound, which was developed in the luckiness framework, and scales logarithmically in the inverse margin. Even in the case of less training examples than input dimensions sufficiently large margins lead to non-trivial bound values and â€'' for maximum margins â€'' to a vanishing complexity term. In contrast to previous results, however, the new bound does depend on the dimensionality of feature space. The analysis shows that the classical margin is too coarse a measure for the essential quantity that controls the generalisation error: the fraction of hypothesis space consistent with the training sample. The practical relevance of the result lies in the fact that the well-known support vector machine is optimal with respect to the new bound only if the feature vectors in the training sample are all of the same length. As a consequence we recommend to use SVMs on normalised feature vectors only. Numerical simulations support this recommendation and demonstrate that the new error bound can be used for the purpose of model selection.
@article{herbrich2002pacbayesian, abstract = {We present a bound on the generalisation error of linear classifiers in terms of a refined margin quantity on the training sample. The result is obtained in a PAC-Bayesian framework and is based on geometrical arguments in the space of linear classifiers. The new bound constitutes an exponential improvement of the so far tightest margin bound, which was developed in the luckiness framework, and scales logarithmically in the inverse margin. Even in the case of less training examples than input dimensions sufficiently large margins lead to non-trivial bound values and {â}€'' for maximum margins {â}€'' to a vanishing complexity term. In contrast to previous results, however, the new bound does depend on the dimensionality of feature space. The analysis shows that the classical margin is too coarse a measure for the essential quantity that controls the generalisation error: the fraction of hypothesis space consistent with the training sample. The practical relevance of the result lies in the fact that the well-known support vector machine is optimal with respect to the new bound only if the feature vectors in the training sample are all of the same length. As a consequence we recommend to use SVMs on normalised feature vectors only. Numerical simulations support this recommendation and demonstrate that the new error bound can be used for the purpose of model selection.}, author = {Herbrich, Ralf and Graepel, Thore}, journal = {IEEE Transactions on Information Theory}, keywords = {imported ralf.herbrich}, number = 12, pages = {3140–3150}, title = {A {PAC-Bayesian} Margin Bound for Linear Classifiers}, volume = 48, year = 2002 }

2001

Support Vector Regression for Black-Box System Identification. Gretton, Arthur; Doucet, Arnaud; Herbrich, Ralf; Rayner, Peter; Schölkopf, Bernhard (2001). 341–344.
[ Abstract ] [ BibTeX ]
 
In this paper, we demonstrate the use of support vector regression (SVR) techniques for black-box system identification. There methods derive from statistical learning theory, and are of great theoretical and practical interest. We briefly describe the theory underpinning SVR, and compare support vector methods with other approaches using radial basis networks. Finally, we apply SVR to modeling the behaviour of a hydralic robot arm, and show that SVR improves on previously published results.
@inproceedings{gretton2001support, abstract = {In this paper, we demonstrate the use of support vector regression (SVR) techniques for black-box system identification. There methods derive from statistical learning theory, and are of great theoretical and practical interest. We briefly describe the theory underpinning SVR, and compare support vector methods with other approaches using radial basis networks. Finally, we apply SVR to modeling the behaviour of a hydralic robot arm, and show that SVR improves on previously published results.}, author = {Gretton, Arthur and Doucet, Arnaud and Herbrich, Ralf and Rayner, Peter and Sch{ö}lkopf, Bernhard}, booktitle = {Proceedings of the 11th IEEE Workshop on Statistical Signal Processing}, keywords = {imported ralf.herbrich}, pages = {341–344}, title = {Support Vector Regression for Black-Box System Identification}, year = 2001 }

Learning on Graphs in the Game of Go. Graepel, Thore; Goutrié, Mike; Krüger, Marco; Herbrich, Ralf (2001). 347–352.
[ Abstract ] [ BibTeX ] [ URL ]
 
We consider the game of Go from the point of view of machine learning and as a well-defined domain for learning on graph representations. We discuss the representation of both board positions and candidate moves and introduce the common fate graph (CFG) as an adequate representation of board positions for learning. Single candidate moves are represented as feature vectors with features given by subgraphs relative to the given move in the CFG. Using this representation we train a support vector machine (SVM) and a kernel perceptron to discriminate good moves from bad moves on a collection of life-and-death problems and on 9x9 game records. We thus obtain kernel machines that solve Go problems and play 9x9 Go.
@inproceedings{graepel2001learning, abstract = {We consider the game of Go from the point of view of machine learning and as a well-defined domain for learning on graph representations. We discuss the representation of both board positions and candidate moves and introduce the common fate graph (CFG) as an adequate representation of board positions for learning. Single candidate moves are represented as feature vectors with features given by subgraphs relative to the given move in the CFG. Using this representation we train a support vector machine (SVM) and a kernel perceptron to discriminate good moves from bad moves on a collection of life-and-death problems and on 9x9 game records. We thus obtain kernel machines that solve Go problems and play 9x9 Go.}, author = {Graepel, Thore and Goutri{é}, Mike and Kr{ü}ger, Marco and Herbrich, Ralf}, booktitle = {Proceedings of the International Conference on Artifical Neural Networks}, keywords = {imported ralf.herbrich}, pages = {347–352}, title = {Learning on Graphs in the Game of Go}, year = 2001 }

Bayes Point Machines. Herbrich, Ralf; Graepel, Thore; Campbell, Colin in Journal of Machine Learning Research (2001). 1 245–279.
[ Abstract ] [ BibTeX ] [ URL ]
 
Kernel-classifiers comprise a powerful class of non-linear decision functions for binary classification. The support vector machine is an example of a learning algorithm for kernel classifiers that singles out the consistent classifier with the largest margin, i.e. minimal real-valued output on the training sample, within the set of consistent hypotheses, the so-called version space. We suggest the Bayes point machine as a well-founded improvement which approximates the Bayes-optimal decision by the centre of mass of version space. We present two algorithms to stochastically approximate the centre of mass of version space: a billiard sampling algorithm and a sampling algorithm based on the well known perceptron algorithm. It is shown how both algorithms can be extended to allow for soft-boundaries in order to admit training errors. Experimentally, we find that --- for the zero training error case --- Bayes point machines consistently outperform support vector machines on both surrogate data and real-world benchmark data sets. In the soft-boundary/soft-margin case, the improvement over support vector machines is shown to be reduced. Finally, we demonstrate that the real-valued output of single Bayes points on novel test points is a valid confidence measure and leads to a steady decrease in generalisation error when used as a rejection criterion.
@article{herbrich2001bayes, abstract = {Kernel-classifiers comprise a powerful class of non-linear decision functions for binary classification. The support vector machine is an example of a learning algorithm for kernel classifiers that singles out the consistent classifier with the largest margin, i.e. minimal real-valued output on the training sample, within the set of consistent hypotheses, the so-called version space. We suggest the Bayes point machine as a well-founded improvement which approximates the Bayes-optimal decision by the centre of mass of version space. We present two algorithms to stochastically approximate the centre of mass of version space: a billiard sampling algorithm and a sampling algorithm based on the well known perceptron algorithm. It is shown how both algorithms can be extended to allow for soft-boundaries in order to admit training errors. Experimentally, we find that — for the zero training error case — Bayes point machines consistently outperform support vector machines on both surrogate data and real-world benchmark data sets. In the soft-boundary/soft-margin case, the improvement over support vector machines is shown to be reduced. Finally, we demonstrate that the real-valued output of single Bayes points on novel test points is a valid confidence measure and leads to a steady decrease in generalisation error when used as a rejection criterion.}, author = {Herbrich, Ralf and Graepel, Thore and Campbell, Colin}, journal = {Journal of Machine Learning Research}, keywords = {imported jmlr ralf.herbrich}, pages = {245–279}, title = {{Bayes} Point Machines}, volume = 1, year = 2001 }

Algorithmic Luckiness. Herbrich, Ralf; Williamson, Robert C (2001). 391–397.
[ Abstract ] [ BibTeX ] [ URL ]
 
In contrast to standard statistical learning theory which studies uniform bounds on the expected error we present a framework that exploits the specific learning algorithm used. Motivated by the luckiness framework [Taylor et al., 1998] we are also able to exploit the serendipity of the training sample. The main difference to previous approaches lies in the complexity measure; rather than covering all hypotheses in a given hypothesis space it is only necessary to cover the functions which could have been learned using the fixed learning algorithm. We show how the resulting framework relates to the VC, luckiness and compression frameworks. Finally, we present an application of this framework to the maximum margin algorithm for linear classifiers which results in a bound that exploits both the margin and the distribution of the data in feature space.
@inproceedings{herbrich2001algorithmic, abstract = {In contrast to standard statistical learning theory which studies uniform bounds on the expected error we present a framework that exploits the specific learning algorithm used. Motivated by the luckiness framework [Taylor et al., 1998] we are also able to exploit the serendipity of the training sample. The main difference to previous approaches lies in the complexity measure; rather than covering all hypotheses in a given hypothesis space it is only necessary to cover the functions which could have been learned using the fixed learning algorithm. We show how the resulting framework relates to the VC, luckiness and compression frameworks. Finally, we present an application of this framework to the maximum margin algorithm for linear classifiers which results in a bound that exploits both the margin and the distribution of the data in feature space.}, author = {Herbrich, Ralf and Williamson, Robert C}, booktitle = {Advances in Neural Information Processing Systems 14}, keywords = {imported ralf.herbrich}, pages = {391–397}, title = {Algorithmic Luckiness}, year = 2001 }

A Generalized Representer Theorem. Schölkopf, Bernhard; Herbrich, Ralf; Smola, Alexander (2001). 416–426.
[ Abstract ] [ BibTeX ] [ URL ]
 
Wahba's classical representer theorem states that the solutions of certain risk minimization problems involving an empirical risk term and a quadratic regularizer can be written as expansions in terms of the training examples. We generalize the theorem to a larger class of regularizers and empirical risk terms, and give a self-contained proof utilizing the feature space associated with a kernel. The result shows that a wide range of problems have optimal solutions that live in the finite dimensional span of the training examples mapped into feature space, thus enabling us to carry out kernel algorithms independent of the (potentially infinite) dimensionality of the feature space.
@inproceedings{scholkopf2001generalized, abstract = {Wahba's classical representer theorem states that the solutions of certain risk minimization problems involving an empirical risk term and a quadratic regularizer can be written as expansions in terms of the training examples. We generalize the theorem to a larger class of regularizers and empirical risk terms, and give a self-contained proof utilizing the feature space associated with a kernel. The result shows that a wide range of problems have optimal solutions that live in the finite dimensional span of the training examples mapped into feature space, thus enabling us to carry out kernel algorithms independent of the (potentially infinite) dimensionality of the feature space.}, author = {Sch{ö}lkopf, Bernhard and Herbrich, Ralf and Smola, Alexander}, booktitle = {Proceedings of the Fourteenth Annual Conference on Computational Learning Theory}, keywords = {imported ralf.herbrich}, pages = {416–426}, title = {A Generalized Representer Theorem}, year = 2001 }

2000

The Kernel Gibbs Sampler. Graepel, Thore; Herbrich, Ralf (2000). 514–520.
[ Abstract ] [ BibTeX ] [ URL ]
 
We present an algorithm that samples the hypothesis space of kernel classifiers. Given a uniform prior over normalised weight vectors and a likelihood based on a model of label noise leads to a piecewise constant posterior that can be sampled by the kernel Gibbs sampler (KGS). The KGS is a Markov Chain Monte Carlo method that chooses a random direction in parameter space and samples from the resulting piecewise constant density along the line chosen. The KGS can be used as an analytical tool for the exploration of Bayesian transduction, Bayes point machines, active learning, and evidence-based model selection on small data sets that are contaminated with label noise. For a simple toy example we demonstrate experimentally how a Bayes point machine based on the KGS outperforms an SVM that is incapable of taking into account label noise.
@inproceedings{graepel2000kernel, abstract = {We present an algorithm that samples the hypothesis space of kernel classifiers. Given a uniform prior over normalised weight vectors and a likelihood based on a model of label noise leads to a piecewise constant posterior that can be sampled by the kernel Gibbs sampler (KGS). The KGS is a Markov Chain Monte Carlo method that chooses a random direction in parameter space and samples from the resulting piecewise constant density along the line chosen. The KGS can be used as an analytical tool for the exploration of Bayesian transduction, Bayes point machines, active learning, and evidence-based model selection on small data sets that are contaminated with label noise. For a simple toy example we demonstrate experimentally how a Bayes point machine based on the KGS outperforms an SVM that is incapable of taking into account label noise.}, author = {Graepel, Thore and Herbrich, Ralf}, booktitle = {Advances in Neural Information Processing Systems 13}, keywords = {imported ralf.herbrich}, pages = {514–520}, publisher = {The MIT Press}, title = {The Kernel Gibbs Sampler}, year = 2000 }

Sparsity vs. Large Margins for Linear Classifiers. Herbrich, Ralf; Graepel, Thore; Shawe-Taylor, John (2000). 304–308.
[ Abstract ] [ BibTeX ] [ URL ]
 
We provide small sample size bounds on the generalisation error of linear classifiers that take advantage of large observed margins on the training set and sparsity in the data dependent expansion coefficients. It is already known from results in the luckiness framework that both criteria independently have a large impact on the generalisation error. Our new results show that they can be combined which theoretically justifies learning algorithms like the Support Vector Machine or the Relevance Vector Machine. In contrast to previous studies we avoid using the classical technique of symmetrisation by a ghost sample but directly using the sparsity for the estimation of the generalisation error. We demonstrate that our result leads to practical useful results even in case of small sample size if the training set witnesses our prior belief in sparsity and large margins.
@inproceedings{herbrich2000sparsity, abstract = {We provide small sample size bounds on the generalisation error of linear classifiers that take advantage of large observed margins on the training set and sparsity in the data dependent expansion coefficients. It is already known from results in the luckiness framework that both criteria independently have a large impact on the generalisation error. Our new results show that they can be combined which theoretically justifies learning algorithms like the Support Vector Machine or the Relevance Vector Machine. In contrast to previous studies we avoid using the classical technique of symmetrisation by a ghost sample but directly using the sparsity for the estimation of the generalisation error. We demonstrate that our result leads to practical useful results even in case of small sample size if the training set witnesses our prior belief in sparsity and large margins.}, author = {Herbrich, Ralf and Graepel, Thore and Shawe-Taylor, John}, booktitle = {Proceedings of the 13th1 Annual Conference on Computational Learning Theory}, keywords = {imported ralf.herbrich}, pages = {304–308}, title = {Sparsity vs. Large Margins for Linear Classifiers}, year = 2000 }

Robust Bayes Point Machines. Herbrich, Ralf; Graepel, Thore; Campbell, Colin (2000). 49–54.
[ Abstract ] [ BibTeX ] [ URL ]
 
Support Vector Machines choose the hypothesis corresponding to the centre of the largest hypersphere that can be inscribed in version space. If version space is elongated or irregularly shaped a potentially superior approach is take into account the whole of version space. We propose to construct the Bayes point which is approximated by the centre of mass. Our implementation of a Bayes Point Machine (BPM) uses an ergodic billiard to estimate this point in the kernel space. We show that BPMs outperform hard margin Support Vector Machines (SVMs) on real world datasets. We introduce a technique that allows the BPM to construct hypotheses with non-zero training error similar to soft margin SVMs with quadratic penelisation of the margin slacks. An experimental study reveals that with decreasing penelisation of training error the improvement of BPMs over SVMs decays, a finding that is explained by geometrical considerations.
@inproceedings{herbrich2000robust, abstract = {Support Vector Machines choose the hypothesis corresponding to the centre of the largest hypersphere that can be inscribed in version space. If version space is elongated or irregularly shaped a potentially superior approach is take into account the whole of version space. We propose to construct the Bayes point which is approximated by the centre of mass. Our implementation of a Bayes Point Machine (BPM) uses an ergodic billiard to estimate this point in the kernel space. We show that BPMs outperform hard margin Support Vector Machines (SVMs) on real world datasets. We introduce a technique that allows the BPM to construct hypotheses with non-zero training error similar to soft margin SVMs with quadratic penelisation of the margin slacks. An experimental study reveals that with decreasing penelisation of training error the improvement of BPMs over SVMs decays, a finding that is explained by geometrical considerations.}, author = {Herbrich, Ralf and Graepel, Thore and Campbell, Colin}, booktitle = {Proceedings of European Symposium on Artificial Neural Networks}, keywords = {imported ralf.herbrich}, pages = {49–54}, title = {Robust {Bayes} Point Machines}, year = 2000 }

Learning Linear Classifiers - Theory and Algorithms. Technical Report (PhD dissertation), Herbrich, Ralf (2000).
[ BibTeX ]
 
@phdthesis{herbrich2000learning, author = {Herbrich, Ralf}, keywords = {imported ralf.herbrich}, school = {Technical University of Berlin}, title = {Learning Linear Classifiers - Theory and Algorithms}, year = 2000 }

Large Scale Bayes Point Machines. Herbrich, Ralf; Graepel, Thore (2000). 528–534.
[ Abstract ] [ BibTeX ] [ URL ]
 
The concept of averaging over classifiers is fundamental to the Bayesian analysis of learning. Based on this viewpoint, it has recently been demonstrated for linear classifiers that the centre of mass of version space (the set of all classifiers consistent with the training set) - also known as the Bayes point - exhibits excellent generalisation abilities. However, the billiard algorithm as presented in [Herbrich et al., 2000] is restricted to small sample size because it requires O(m*m) of memory and O(N*m*m) computational steps where m is the number of training patterns and N is the number of random draws from the posterior distribution. In this paper we present a method based on the simple perceptron learning algorithm which allows to overcome this algorithmic drawback. The method is algorithmically simple and is easily extended to the multi-class case. We present experimental results on the MNIST data set of handwritten digits which show that Bayes Point Machines are competitive with the current world champion, the support vector machine. In addition, the computational complexity of BPMs can be tuned by varying the number of samples from the posterior. Finally, rejecting test points on the basis of their (approximative) posterior probability leads to a rapid decrease in generalisation error, e.g. 0.1% generalisation error for a given rejection rate of 10%.
@inproceedings{herbrich2000large, abstract = {The concept of averaging over classifiers is fundamental to the Bayesian analysis of learning. Based on this viewpoint, it has recently been demonstrated for linear classifiers that the centre of mass of version space (the set of all classifiers consistent with the training set) - also known as the Bayes point - exhibits excellent generalisation abilities. However, the billiard algorithm as presented in [Herbrich et al., 2000] is restricted to small sample size because it requires O(m*m) of memory and O(N*m*m) computational steps where m is the number of training patterns and N is the number of random draws from the posterior distribution. In this paper we present a method based on the simple perceptron learning algorithm which allows to overcome this algorithmic drawback. The method is algorithmically simple and is easily extended to the multi-class case. We present experimental results on the MNIST data set of handwritten digits which show that Bayes Point Machines are competitive with the current world champion, the support vector machine. In addition, the computational complexity of BPMs can be tuned by varying the number of samples from the posterior. Finally, rejecting test points on the basis of their (approximative) posterior probability leads to a rapid decrease in generalisation error, e.g. 0.1\% generalisation error for a given rejection rate of 10\%.}, author = {Herbrich, Ralf and Graepel, Thore}, booktitle = {Advances in Neural Information Processing Systems 13}, keywords = {imported ralf.herbrich}, pages = {528–534}, publisher = {The MIT Press}, title = {Large Scale Bayes Point Machines}, year = 2000 }

Generalisation Error Bounds for Sparse Linear Classifiers. Graepel, Thore; Herbrich, Ralf; Shawe-Taylor, John (2000). 298–303.
[ Abstract ] [ BibTeX ] [ URL ]
 
We provide small sample size bounds on the generalisation error of linear classi�ers that are sparse in their dual representation given by the expansion coe�cients of the weight vector in terms of the training data. These results theoretically justify algorithms like the Support Vector Machine, the Relevance Vector Machine and K-nearest-neighbour. The bounds are a-posteriori bounds to be evaluated after learning when the attained level of sparsity is known. In a PAC-Bayesian style prior knowledge about the expected sparsity is incorporated into the bounds. The proofs avoid the use of double sample arguments by taking into account the sparsity that leaves unused training points for the evaluation of classi�ers. We furthermore give a PAC-Bayesian bound on the average generalisation error over subsets of parameter space that may pave the way combining sparsity in the expansion coefficients and margin in a single bound. Finally, reinterpreting a mistake bound for the classical perceptron algorithm due to Noviko� we demonstrate that our new results put classifiers found by this algorithm on a �firm theoretical basis.
@inproceedings{graepel2000generalisation, abstract = {We provide small sample size bounds on the generalisation error of linear classi�ers that are sparse in their dual representation given by the expansion coe�cients of the weight vector in terms of the training data. These results theoretically justify algorithms like the Support Vector Machine, the Relevance Vector Machine and K-nearest-neighbour. The bounds are a-posteriori bounds to be evaluated after learning when the attained level of sparsity is known. In a PAC-Bayesian style prior knowledge about the expected sparsity is incorporated into the bounds. The proofs avoid the use of double sample arguments by taking into account the sparsity that leaves unused training points for the evaluation of classi�ers. We furthermore give a PAC-Bayesian bound on the average generalisation error over subsets of parameter space that may pave the way combining sparsity in the expansion coefficients and margin in a single bound. Finally, reinterpreting a mistake bound for the classical perceptron algorithm due to Noviko� we demonstrate that our new results put classifiers found by this algorithm on a �firm theoretical basis.}, author = {Graepel, Thore and Herbrich, Ralf and Shawe-Taylor, John}, booktitle = {Proceedings of the 13th Annual Conference on Computational Learning Theory}, keywords = {imported ralf.herbrich}, pages = {298–303}, title = {Generalisation Error Bounds for Sparse Linear Classifiers}, year = 2000 }

From Margin to Sparsity. Graepel, Thore; Herbrich, Ralf; Williamson, Robert C (2000). 210–216.
[ Abstract ] [ BibTeX ] [ URL ]
 
We present an improvement of Novikoff's perceptron convergence theorem. Reinterpreting this mistake bound as a margin dependent sparsity guarantee allows us to give a PAC-style generalisation error bound for the classifier learned by the dual perceptron learning algorithm. The bound value crucially depends on the margin a support vector machine would achieve on the same data set using the same kernel. Ironically, the bound yields better guarantees than are currently available for the support vector solution itself.
@inproceedings{graepel2000margin, abstract = {We present an improvement of Novikoff's perceptron convergence theorem. Reinterpreting this mistake bound as a margin dependent sparsity guarantee allows us to give a PAC-style generalisation error bound for the classifier learned by the dual perceptron learning algorithm. The bound value crucially depends on the margin a support vector machine would achieve on the same data set using the same kernel. Ironically, the bound yields better guarantees than are currently available for the support vector solution itself.}, author = {Graepel, Thore and Herbrich, Ralf and Williamson, Robert C}, booktitle = {Advances in Neural Information Processing Systems 13}, keywords = {imported ralf.herbrich}, pages = {210–216}, publisher = {The MIT Press}, title = {From Margin to Sparsity}, year = 2000 }

A PAC-Bayesian Margin Bound for Linear Classifiers: Why SVMs work. Herbrich, Ralf; Graepel, Thore (2000). 224–230.
[ Abstract ] [ BibTeX ] [ URL ]
 
We present a bound on the generalisation error of linear classifiers in terms of a refined margin quantity on the training set. The result is obtained in a PAC-Bayesian framework and is based on geometrical arguments in the space of linear classifiers. The new bound constitutes an exponential improvement of the so far tightest margin bound by Shawe-Taylor et al. and scales logarithmically in the inverse margin. Even in the case of less training examples than input dimensions sufficiently large margins lead to non-trivial bound values and - for maximum margins - to a vanishing complexity term. Furthermore, the classical margin is too coarse a measure for the essential quantity that controls the generalisation error: the volume ratio between the whole hypothesis space and the subset of consistent hypotheses. The practical relevance of the result lies in the fact that the well-known support vector machine is optimal w.r.t. the new bound only if the feature vectors are all of the same length. As a consequence we recommend to use SVMs on normalised feature vectors only - a recommendation that is well supported by our numerical experiments on two benchmark data sets.
@inproceedings{herbrich2000pacbayesian, abstract = {We present a bound on the generalisation error of linear classifiers in terms of a refined margin quantity on the training set. The result is obtained in a PAC-Bayesian framework and is based on geometrical arguments in the space of linear classifiers. The new bound constitutes an exponential improvement of the so far tightest margin bound by Shawe-Taylor et al. and scales logarithmically in the inverse margin. Even in the case of less training examples than input dimensions sufficiently large margins lead to non-trivial bound values and - for maximum margins - to a vanishing complexity term. Furthermore, the classical margin is too coarse a measure for the essential quantity that controls the generalisation error: the volume ratio between the whole hypothesis space and the subset of consistent hypotheses. The practical relevance of the result lies in the fact that the well-known support vector machine is optimal w.r.t. the new bound only if the feature vectors are all of the same length. As a consequence we recommend to use SVMs on normalised feature vectors only - a recommendation that is well supported by our numerical experiments on two benchmark data sets.}, author = {Herbrich, Ralf and Graepel, Thore}, booktitle = {Advances in Neural Information Processing Systems 13}, keywords = {imported ralf.herbrich}, pages = {224–230}, title = {A {PAC-Bayesian} Margin Bound for Linear Classifiers: Why SVMs work}, year = 2000 }

1999

Support Vector Learning for Ordinal Regression. Herbrich, Ralf; Graepel, Thore; Obermayer, Klaus (1999). 97–102.
[ Abstract ] [ BibTeX ] [ URL ]
 
We investigate the problem of predicting variables of ordinal scale. This task is referred to as ordinal regression and is complementary to the standard machine learning tasks of classification and metric regression. In contrast to statistical models we present a distribution independent formulation of the problem together with uniform bounds of the risk functional. The approach presented is based on a mapping from objects to scalar utility values. Similar to Support Vector methods we derive a new learning algorithm for the task of ordinal regression based on large margin rank boundaries. We give experimental results for an information retrieval task: learning the order of documents w.r.t. an initial query. Experimental results indicate that the presented algorithm outperforms more naive approaches to ordinal regression such as Support Vector classification and Support Vector regression in the case of more than two ranks.
@inproceedings{herbrich1999support, abstract = {We investigate the problem of predicting variables of ordinal scale. This task is referred to as ordinal regression and is complementary to the standard machine learning tasks of classification and metric regression. In contrast to statistical models we present a distribution independent formulation of the problem together with uniform bounds of the risk functional. The approach presented is based on a mapping from objects to scalar utility values. Similar to Support Vector methods we derive a new learning algorithm for the task of ordinal regression based on large margin rank boundaries. We give experimental results for an information retrieval task: learning the order of documents w.r.t. an initial query. Experimental results indicate that the presented algorithm outperforms more naive approaches to ordinal regression such as Support Vector classification and Support Vector regression in the case of more than two ranks.}, author = {Herbrich, Ralf and Graepel, Thore and Obermayer, Klaus}, booktitle = {Proceedings of the 9th International Conference on Artificial Neural Networks}, keywords = {imported ralf.herbrich}, pages = {97–102}, title = {Support Vector Learning for Ordinal Regression}, year = 1999 }

Neural Networks in Economics : Background, Applications and New Developments. Herbrich, Ralf; Keilbach, Max; Bollmann-Sdorra, Peter; Obermayer, Klaus in Advances in Computational Economics (1999). 11 169–196.
[ Abstract ] [ BibTeX ] [ URL ]
 
Neural Networks were developed in the sixties as devices for classification and regression. The approach was originally inspired from Neuroscience. Its attractiveness lies in the ability to 'learn', i.e. to generalize to as yet unseen observations. One aim of this paper is to give an introduction to the technique of Neural Networks and an overview of the most popular architectures. We start from statistical learning theory to introduce the basics of learning. Then, we give an overview of the general principles of neural networks and of their use in the field of Economics. A second purpose is to introduce a recently developed Neural Network Learning technique, so called Support Vector Network Learning, which is an application of ideas from statistical learning theory. This approach has shown very promising results on problems with a limited amount of training examples. Moreover, utilizing a technique that is known as the kernel trick, Support Vector Networks can easily be adapted to nonlinear models. Finally, we present an economic application of this approach from the field of preference learning.
@article{herbrich1999neural, abstract = {Neural Networks were developed in the sixties as devices for classification and regression. The approach was originally inspired from Neuroscience. Its attractiveness lies in the ability to 'learn', i.e. to generalize to as yet unseen observations. One aim of this paper is to give an introduction to the technique of Neural Networks and an overview of the most popular architectures. We start from statistical learning theory to introduce the basics of learning. Then, we give an overview of the general principles of neural networks and of their use in the field of Economics. A second purpose is to introduce a recently developed Neural Network Learning technique, so called Support Vector Network Learning, which is an application of ideas from statistical learning theory. This approach has shown very promising results on problems with a limited amount of training examples. Moreover, utilizing a technique that is known as the kernel trick, Support Vector Networks can easily be adapted to nonlinear models. Finally, we present an economic application of this approach from the field of preference learning.}, author = {Herbrich, Ralf and Keilbach, Max and Bollmann-Sdorra, Peter and Obermayer, Klaus}, journal = {Advances in Computational Economics}, keywords = {imported ralf.herbrich}, pages = {169–196}, title = {Neural Networks in Economics : Background, Applications and New Developments}, volume = 11, year = 1999 }

Large Margin Rank Boundaries for Ordinal Regression. Herbrich, Ralf; Graepel, Thore; Obermayer, Klause in Advances in Large Margin Classifiers (1999). 115–132.
[ Abstract ] [ BibTeX ] [ URL ]
 
In contrast to the standard machine learning tasks of classification and metric regression we investigate the problem of predicting variables of ordinal scale, a setting referred to as ordinal regression. This problem arises frequently in the social sciences and in information retrieval where human preferences play a major role. Whilst approaches proposed in statistics rely on a probability model of a latent (unobserved) variable we present a distribution independent risk formulation of ordinal regression which allows us to derive a uniform convergence bound. Applying this bound we present a large margin algorithm that is based on a mapping from objects to scalar utility values thus classifying pairs of objects. We give experimental results for an information retrieval task which show that our algorithm outperforms more naive approaches to ordinal regression such as Support Vector Classification and Support Vector Regression in the case of more than two ranks.
@incollection{herbrich1999large, abstract = {In contrast to the standard machine learning tasks of classification and metric regression we investigate the problem of predicting variables of ordinal scale, a setting referred to as ordinal regression. This problem arises frequently in the social sciences and in information retrieval where human preferences play a major role. Whilst approaches proposed in statistics rely on a probability model of a latent (unobserved) variable we present a distribution independent risk formulation of ordinal regression which allows us to derive a uniform convergence bound. Applying this bound we present a large margin algorithm that is based on a mapping from objects to scalar utility values thus classifying pairs of objects. We give experimental results for an information retrieval task which show that our algorithm outperforms more naive approaches to ordinal regression such as Support Vector Classification and Support Vector Regression in the case of more than two ranks.}, author = {Herbrich, Ralf and Graepel, Thore and Obermayer, Klause}, booktitle = {Advances in Large Margin Classifiers}, chapter = 7, keywords = {imported ralf.herbrich}, pages = {115–132}, publisher = {The MIT Press}, title = {Large Margin Rank Boundaries for Ordinal Regression}, year = 1999 }

Classification on Proximity Data with LP-Machines. Graepel, Thore; Herbrich, Ralf; Schölkopf, Bernhard; Smola, Alex; Bartlett, Peter; Müller, Klaus Robert; Obermayer, Klaus; Williamson, Robert C (1999). 304–309.
[ Abstract ] [ BibTeX ] [ URL ]
 
We provide a new linear program to deal with classification of data in the case of data given in terms of pairwise proximities. This allows to avoid the problems inherent in using feature spaces with indefinite metric in Support Vector Machines, since the notion of a margin is purely needed in input space where the classification actually occurs. Moreover in our approach we can enforce sparsity in the proximity representation by sacrificing training error. This turns out to be favorable for proximity data. Similar to nu-SV methods, the only parameter needed in the algorithm is the (asymptotical) number of data points being classified with a margin. Finally, the algorithm is successfully compared with nu-SV learning in proximity space and K-nearest-neighbors on real world data from Neuroscience and molecular biology.
@inproceedings{graepel1999classification, abstract = {We provide a new linear program to deal with classification of data in the case of data given in terms of pairwise proximities. This allows to avoid the problems inherent in using feature spaces with indefinite metric in Support Vector Machines, since the notion of a margin is purely needed in input space where the classification actually occurs. Moreover in our approach we can enforce sparsity in the proximity representation by sacrificing training error. This turns out to be favorable for proximity data. Similar to nu-SV methods, the only parameter needed in the algorithm is the (asymptotical) number of data points being classified with a margin. Finally, the algorithm is successfully compared with nu-SV learning in proximity space and K-nearest-neighbors on real world data from Neuroscience and molecular biology.}, author = {Graepel, Thore and Herbrich, Ralf and Sch{ö}lkopf, Bernhard and Smola, Alex and Bartlett, Peter and M{ü}ller, Klaus Robert and Obermayer, Klaus and Williamson, Robert C}, booktitle = {Proceedings of the 9th International Conference on Artificial Neural Networks}, keywords = {imported ralf.herbrich}, pages = {304–309}, title = {Classification on Proximity Data with {LP}-Machines}, year = 1999 }

Bayesian Transduction. Graepel, Thore; Herbrich, Ralf; Obermayer, Klaus (1999). 456–462.
[ Abstract ] [ BibTeX ] [ URL ]
 
Transduction is an inference principle that takes a training sample and aims at estimating the values of a function at given points contained in the so-called working sample. Hence, transduction is a less ambitious task than induction which aims at inferring a functional dependency on the whole of input space. As a consequence, however, transduction provides a confidence measure on single predictions rather than classifiers, a feature particularly important for risk-sensitive applications. We consider the case of binary classification by linear discriminant functions (perceptrons) in kernel space. From the transductive point of view, the infinite number of perceptrons is boiled down to a finite number of equivalence classes on the working sample each of which corresponds to a polyhedron in parameter space. In the Bayesian spirit the posteriori probability of a labelling of the working sample is determined as the ratio between the volume of the corresponding polyhedron and the volume of version space. Then the maximum posteriori scheme recommends to choose the labelling of maximum volume. We suggest to sample version space by an ergodic billiard in kernel space. Experimental results on real world data indicate that Bayesian Transduction compares favourably to the well-known Support Vector Machine, in particular if the posteriori probability of labellings is used as a confidence measure to exclude test points of low confidence.
@inproceedings{graepel1999bayesian, abstract = {Transduction is an inference principle that takes a training sample and aims at estimating the values of a function at given points contained in the so-called working sample. Hence, transduction is a less ambitious task than induction which aims at inferring a functional dependency on the whole of input space. As a consequence, however, transduction provides a confidence measure on single predictions rather than classifiers, a feature particularly important for risk-sensitive applications. We consider the case of binary classification by linear discriminant functions (perceptrons) in kernel space. From the transductive point of view, the infinite number of perceptrons is boiled down to a finite number of equivalence classes on the working sample each of which corresponds to a polyhedron in parameter space. In the Bayesian spirit the posteriori probability of a labelling of the working sample is determined as the ratio between the volume of the corresponding polyhedron and the volume of version space. Then the maximum posteriori scheme recommends to choose the labelling of maximum volume. We suggest to sample version space by an ergodic billiard in kernel space. Experimental results on real world data indicate that Bayesian Transduction compares favourably to the well-known Support Vector Machine, in particular if the posteriori probability of labellings is used as a confidence measure to exclude test points of low confidence.}, author = {Graepel, Thore and Herbrich, Ralf and Obermayer, Klaus}, booktitle = {Advances in Neural Information Processing Systems 12}, keywords = {imported ralf.herbrich}, pages = {456–462}, publisher = {The MIT Press}, title = {{Bayesian} Transduction}, year = 1999 }

Bayes Point Machines : Estimating the Bayes Point in Kernel Space. Herbrich, Ralf; Graepel, Thore; Campbell, Colin (1999). 23–27.
[ Abstract ] [ BibTeX ] [ URL ]
 
From a Bayesian perspective Support Vector Machines choose the hypothesis corresponding to the largest possible hypersphere that can be inscribed in version space, i.e. in the space of all consistent hypotheses given a training set. Those boundaries of version space which are tangent to the hypersphere define the support vectors. An alternative and potentially better approach is to construct the hypothesis using the whole of version space. This is achieved by using a Bayes Point Machine which finds the midpoint of the region of intersection of all hyperplanes bisecting version space into two halves of equal volume (the Bayes point). It is known that the center of mass of version space approximates the Bayes point. We suggest estimating the center of mass by averaging over the trajectory of a billiard ball bouncing in version space. Experimental results are presented indicating that Bayes Point Machines consistently outperform Support Vector Machines.
@inproceedings{herbrich1999bayes, abstract = {From a Bayesian perspective Support Vector Machines choose the hypothesis corresponding to the largest possible hypersphere that can be inscribed in version space, i.e. in the space of all consistent hypotheses given a training set. Those boundaries of version space which are tangent to the hypersphere define the support vectors. An alternative and potentially better approach is to construct the hypothesis using the whole of version space. This is achieved by using a Bayes Point Machine which finds the midpoint of the region of intersection of all hyperplanes bisecting version space into two halves of equal volume (the Bayes point). It is known that the center of mass of version space approximates the Bayes point. We suggest estimating the center of mass by averaging over the trajectory of a billiard ball bouncing in version space. Experimental results are presented indicating that Bayes Point Machines consistently outperform Support Vector Machines.}, author = {Herbrich, Ralf and Graepel, Thore and Campbell, Colin}, booktitle = {Proceedings of IJCAI Workshop Support Vector Machines}, keywords = {imported ralf.herbrich}, pages = {23–27}, title = {{Bayes} Point Machines : Estimating the {Bayes} Point in Kernel Space}, year = 1999 }

Adaptive Margin Support Vector Machines for Classification. Herbrich, Ralf; Weston, Jason (1999). 97–102.
[ Abstract ] [ BibTeX ] [ URL ]
 
In this paper we propose a new learning algorithm for classification learning based on the Support Vector Machine (SVM) approach. Existing approaches for constructing SVMs are based on minimization of a regularized margin loss where the margin is treated equivalently for each train- ing pattern. We propose a reformulation of the minimization problem such that adaptive margins for each training pattern are utilized, which we call the Adaptive Margin (AM)-SVM. We give bounds on the generalization error of AM-SVMs which justify their robustness against outliers, and show experimentally that the generalization error of AM-SVMs is comparable to classical SVMs on benchmark datasets from the UCI repository.
@inproceedings{herbrich1999adaptive, abstract = {In this paper we propose a new learning algorithm for classification learning based on the Support Vector Machine (SVM) approach. Existing approaches for constructing SVMs are based on minimization of a regularized margin loss where the margin is treated equivalently for each train- ing pattern. We propose a reformulation of the minimization problem such that adaptive margins for each training pattern are utilized, which we call the Adaptive Margin (AM)-SVM. We give bounds on the generalization error of AM-SVMs which justify their robustness against outliers, and show experimentally that the generalization error of AM-SVMs is comparable to classical SVMs on benchmark datasets from the UCI repository.}, author = {Herbrich, Ralf and Weston, Jason}, booktitle = {Proceedings of the 9th International Conference on Artificial Neural Networks}, keywords = {imported ralf.herbrich}, pages = {97–102}, title = {Adaptive Margin Support Vector Machines for Classification}, year = 1999 }

Adaptive Margin Support Vector Machines. Weston, Jason; Herbrich, Ralf in Advances in Large Margin Classifiers (1999). 281–296.
[ Abstract ] [ BibTeX ] [ URL ]
 
In this chapter we present a new learning algorithm, Leave-One-Out (LOO -) SVMs and its generalization Adaptive Margin (AM-) SVMs, inspired by a recent upper bound on the leave-one-out error proved for kernel classifiers by Jaakkola and Haussler. The new approach minimizes the expression given by the bound in an attempt to minimize the leave-one-out error. This gives a convex optimization problem which constructs a sparse linear classifier in feature space using the kernel technique. As such the algorithm possesses many of the same properties as SVMs and Linear Programming (LP-) SVMs. These former techniques are based on the minimization of a regularized margin loss, where the margin is treated equivalently for each training pattern. We propose a minimization problem such that adaptive margins for each training pattern are utilized. Furthermore, we give bounds on the generalization error of the approach which justifies its robustness against outliers. We show experimentally that the generalization error of AM-SVMs is comparable to SVMs and LP-SVMs on benchmark datasets from the UCI repository.
@incollection{weston1999adaptive, abstract = {In this chapter we present a new learning algorithm, Leave-One-Out (LOO -) SVMs and its generalization Adaptive Margin (AM-) SVMs, inspired by a recent upper bound on the leave-one-out error proved for kernel classifiers by Jaakkola and Haussler. The new approach minimizes the expression given by the bound in an attempt to minimize the leave-one-out error. This gives a convex optimization problem which constructs a sparse linear classifier in feature space using the kernel technique. As such the algorithm possesses many of the same properties as SVMs and Linear Programming (LP-) SVMs. These former techniques are based on the minimization of a regularized margin loss, where the margin is treated equivalently for each training pattern. We propose a minimization problem such that adaptive margins for each training pattern are utilized. Furthermore, we give bounds on the generalization error of the approach which justifies its robustness against outliers. We show experimentally that the generalization error of AM-SVMs is comparable to SVMs and LP-SVMs on benchmark datasets from the UCI repository.}, author = {Weston, Jason and Herbrich, Ralf}, booktitle = {Advances in Large Margin Classifiers}, chapter = 15, keywords = {imported ralf.herbrich}, pages = {281–296}, publisher = {The MIT Press}, title = {Adaptive Margin Support Vector Machines}, year = 1999 }

1998

Learning Preference Relations for Information Retrieval. Herbrich, Ralf; Graepel, Thore; Bollmann-Sdorra, Peter; Obermayer, Klaus (1998). 80–84.
[ Abstract ] [ BibTeX ] [ URL ]
 
In this paper we investigate the problem of learning a preference relation from a given set of ranked documents. We show that the Bayes's optimal decision function, when applied to learning a preference relation, may violate transitivity. This is undesirable for information retrieval, because it is in conflict with a document ranking based on the user's preferences. To overcome this problem we present a vector space based method that performs a linear mapping from documents to scalar utility values and thus guarantees transitivity. The learning of the relation between documents is formulated as a classification problem on pairs of documents and is solved using the principle of structural risk minimization for good generalization. The approach is extended to polynomial utility functions by using the potential function method (the so called "kernel trick"), which allows to incorporate higher order correlations of features into the utility function at minimal computational costs. The resulting algorithm is tested on an example with artificial data. The algorithm successfully learns the utility function underlying the training examples and shows good classification performance.
@inproceedings{herbrich1998learning, abstract = {In this paper we investigate the problem of learning a preference relation from a given set of ranked documents. We show that the Bayes's optimal decision function, when applied to learning a preference relation, may violate transitivity. This is undesirable for information retrieval, because it is in conflict with a document ranking based on the user's preferences. To overcome this problem we present a vector space based method that performs a linear mapping from documents to scalar utility values and thus guarantees transitivity. The learning of the relation between documents is formulated as a classification problem on pairs of documents and is solved using the principle of structural risk minimization for good generalization. The approach is extended to polynomial utility functions by using the potential function method (the so called "kernel trick"), which allows to incorporate higher order correlations of features into the utility function at minimal computational costs. The resulting algorithm is tested on an example with artificial data. The algorithm successfully learns the utility function underlying the training examples and shows good classification performance.}, author = {Herbrich, Ralf and Graepel, Thore and Bollmann-Sdorra, Peter and Obermayer, Klaus}, booktitle = {Proceedings of the International Conference on Machine Learning Workshop: Text Categorization and Machine learning}, keywords = {imported ralf.herbrich}, pages = {80–84}, title = {Learning Preference Relations for Information Retrieval}, year = 1998 }

Classification on Pairwise Proximity Data. Graepel, Thore; Herbrich, Ralf; Bollmann-Sdorra, Peter; Obermayer, Klaus (1998). 438–444.
[ Abstract ] [ BibTeX ] [ URL ]
 
We investigate the problem of learning a classification task on data represented in terms of their pairwise proximities. This representation does not refer to an explicit feature representation of the data items and is thus more general than the standard approach of using Euclidean feature vectors, from which pairwise proximities can always be calculated. Our first approach is based on a combined linear embedding and classification procedure resulting in an extension of the Optimal Hyperplane algorithm to pseudo-Euclidean data. As an alternative we present another approach based on a linear threshold model in the proximity values themselves, which is optimized using Structural Risk Minimization. We show that prior knowledge about the problem can be incorporated by the choice of distance measures and examine different metrics w.r.t. their generalization. Finally, the algorithms are successfully applied to protein structure data and to data from the cat's cerebral cortex. They show better performance than K-nearest-neighbor classification.
@inproceedings{graepel1998classification, abstract = {We investigate the problem of learning a classification task on data represented in terms of their pairwise proximities. This representation does not refer to an explicit feature representation of the data items and is thus more general than the standard approach of using Euclidean feature vectors, from which pairwise proximities can always be calculated. Our first approach is based on a combined linear embedding and classification procedure resulting in an extension of the Optimal Hyperplane algorithm to pseudo-Euclidean data. As an alternative we present another approach based on a linear threshold model in the proximity values themselves, which is optimized using Structural Risk Minimization. We show that prior knowledge about the problem can be incorporated by the choice of distance measures and examine different metrics w.r.t. their generalization. Finally, the algorithms are successfully applied to protein structure data and to data from the cat's cerebral cortex. They show better performance than K-nearest-neighbor classification.}, author = {Graepel, Thore and Herbrich, Ralf and Bollmann-Sdorra, Peter and Obermayer, Klaus}, booktitle = {Advances in Neural Information Processing Systems 11}, keywords = {imported ralf.herbrich}, pages = {438–444}, publisher = {The MIT Press}, title = {Classification on Pairwise Proximity Data}, year = 1998 }

1997

Unbiased Assesment of Learning Algorithms. Scheffer, Tobias; Herbrich, Ralf (1997). 798–803.
[ Abstract ] [ BibTeX ] [ URL ]
 
In order to rank the performance of machine learning algorithms, many researchs conduct experiments on benchmark datasets. Since most learning algorithms have domain-specific parameters, it is a popular custom to adapt these parameters to obtain a minimal error rate on the test set. The same rate is used to rank the algorithm which causes an optimistic bias. We quantify this bias, showing in particular that an algorithm with more parameters will probably be ranked higher than an equally good algorithm with fewer parameters. We demonstrate this result, showing the number of parameters and trials required in order to pretend to outperform C4.5 or FOIL, respectively, for various benchmark problems. We then describe how unbiased ranking experiments should be conducted.
@inproceedings{scheffer1997unbiased, abstract = {In order to rank the performance of machine learning algorithms, many researchs conduct experiments on benchmark datasets. Since most learning algorithms have domain-specific parameters, it is a popular custom to adapt these parameters to obtain a minimal error rate on the test set. The same rate is used to rank the algorithm which causes an optimistic bias. We quantify this bias, showing in particular that an algorithm with more parameters will probably be ranked higher than an equally good algorithm with fewer parameters. We demonstrate this result, showing the number of parameters and trials required in order to pretend to outperform C4.5 or FOIL, respectively, for various benchmark problems. We then describe how unbiased ranking experiments should be conducted.}, author = {Scheffer, Tobias and Herbrich, Ralf}, booktitle = {Proceedings of the International Joint Conference on Artificial Intelligence}, keywords = {imported ralf.herbrich}, pages = {798–803}, title = {Unbiased Assesment of Learning Algorithms}, year = 1997 }

Segmentierung mit Gaborfiltern zur Induktion struktureller Klassifikatoren auf Bilddaten. Technical Report (Master thesis), Herbrich, Ralf PhD thesis, Technical University Berlin. (1997).
[ BibTeX ] [ URL ]
 
@mastersthesis{herbrich1997segmentierung, author = {Herbrich, Ralf}, keywords = {imported ralf.herbrich}, school = {Technical University Berlin}, title = {Segmentierung mit Gaborfiltern zur Induktion struktureller Klassifikatoren auf Bilddaten}, year = 1997 }

Generation of Task-Specific Segmentation Procedures as a Model Selection Task. Herbrich, Ralf; Scheffer, Tobias (1997). 11–21.
[ Abstract ] [ BibTeX ] [ URL ]
 
In image segmentation problems, there is usually a vast amount of filter operations available, a subset of which has to be selected and instantiated in order to obtain a satisfactory segmentation procedure for a particular do- main. In supervised segmentation, a mapping from features, such as filter outputs for individual pixels, to classes is induced automatically. However, since the sample size required for supervised learning grows exponentially in the number of features it is not feasible to learn a segmentation procedure from a large amount of possible filters. But we argue that automatic model selection methods are able to select a region model in terms of some filters.
@inproceedings{herbrich1997generation, abstract = {In image segmentation problems, there is usually a vast amount of filter operations available, a subset of which has to be selected and instantiated in order to obtain a satisfactory segmentation procedure for a particular do- main. In supervised segmentation, a mapping from features, such as filter outputs for individual pixels, to classes is induced automatically. However, since the sample size required for supervised learning grows exponentially in the number of features it is not feasible to learn a segmentation procedure from a large amount of possible filters. But we argue that automatic model selection methods are able to select a region model in terms of some filters.}, author = {Herbrich, Ralf and Scheffer, Tobias}, booktitle = {Proceedings of Workshop on Visual Information Processing}, keywords = {imported ralf.herbrich}, pages = {11–21}, title = {Generation of Task-Specific Segmentation Procedures as a Model Selection Task}, year = 1997 }

1996

Efficient \($\Theta$\)-Subsumption Based on Graph Algorithms. Scheffer, Tobias; Herbrich, Ralf; Wysotzki, Fritz (1996). (Vol. 1314) 212–228.
[ Abstract ] [ BibTeX ] [ URL ]
 
The \($\theta$\)-subsumption problem is crucial to the efficiency of ILP learning systems. We discuss two \($\theta$\)-subsumption algorithms based on strategies for preselecting suitable matching literals. The class of clauses, for which subsumption becomes polynomial, is a superset of the deterministic clauses. We further map the general problem of \($\theta$\)-subsumption to a certain problem of finding a clique of fixed size in a graph, and in return show that a specialization of the pruning strategy of the Carraghan and Pardalos clique algorithm provides a dramatic reduction of the subsumption search space. We also present empirical results for the mesh design data set.
@inproceedings{scheffer1996efficient, abstract = {The $\theta$-subsumption problem is crucial to the efficiency of ILP learning systems. We discuss two $\theta$-subsumption algorithms based on strategies for preselecting suitable matching literals. The class of clauses, for which subsumption becomes polynomial, is a superset of the deterministic clauses. We further map the general problem of $\theta$-subsumption to a certain problem of finding a clique of fixed size in a graph, and in return show that a specialization of the pruning strategy of the Carraghan and Pardalos clique algorithm provides a dramatic reduction of the subsumption search space. We also present empirical results for the mesh design data set.}, author = {Scheffer, Tobias and Herbrich, Ralf and Wysotzki, Fritz}, booktitle = {Lecture Notes in Artifical Intelligence: 6th International Workshop on Inductive Logic Programming,}, keywords = {imported ralf.herbrich}, pages = {212–228}, title = {Efficient $\Theta$-Subsumption Based on Graph Algorithms}, volume = 1314, year = 1996 }