Data Subsampling For Model Selection In Automl Frameworks

Data Subsampling for Model Selection in Automl Frameworks

This project studies methods of using data subsampling to perform model selection. Most commonly used methods for model selection require training all models on the entire training data several times in order to pick the best one. This is often one of the most computationally expensive aspects of model selection. It would therefore be valuable to understand how resources can be better allocated to pick the best model for a given dataset. This project explores this question of how to optimize resource allocation for model selection by subsampling data. We try three different approaches to model selection starting with (1) a randomized multi-armed bandit approach, (2) subsampling using influence functions and finally (3) a new boosting based method that can be called iterative boosting. The first method uses 10 tabular datasets while the following two approaches use MNIST and CIFAR-10 image datasets and deep learning models. Each of these approaches uses a unique set of assumptions which provide some pros and cons for the intended task of model selection. Analysis of these three methods is done to better understand how subsampling can be better approached in order to take meaningful subsets of data to accurately estimate a model's relative test performance. The hyperband method for subsampling seems to be the most effective in terms of computational complexity as well as getting good relative model performance. The iterative boosting method shows some promise on MNIST but requires more work in order to make it significantly better than random subsampling for more complex datasets like CIFAR-10.

Travel Mode Substitution in São Paulo

Swiss National Forest Inventory – Methods and Models of the Fourth Assessment

The Swiss National Forest Inventory (NFI) is a forest survey on national level which started in 1982 and has already reached its 5th survey cycle (NFI5). It can be characterized as a multisource and multipurpose inventory where information is mainly collected from terrestrial field surveys using permanent sample plots. In addition, data from aerial photography, GIS and forest service questionnaires are also included. The NFI's main objective is to provide statistically reliable and sound figures to stakeholders such as politicians, researchers, ecologists, forest service, timber industry, national and international organizations as well as to international projects such as the Forest Resources Assessment of the United Nations. For Switzerland, NFI results are typically reported on national and regional level. State of the art methods are applied in all fields of data collection which have been proven to be of international interest and have even served as a basis for other European NFIs. The presented methods are applicable to any sample based forest inventory around the globe. In 2001 the Swiss NFI published its methods for the first time. Since then, many methodological changes and improvements have been introduced. This book describes the complete set of methods and revisions since NFI2. It covers various topics ranging from inventory design and statistics to remote sensing, field survey methods and modelling. It also describes data quality concepts and the software framework used for data storage, statistical analysis and result presentation.