Exploring And Exploiting Structure And Self Supervision In Sequence Learning
Download Exploring And Exploiting Structure And Self Supervision In Sequence Learning PDF/ePub or read online books in Mobi eBooks. Click Download or Read Online button to get Exploring And Exploiting Structure And Self Supervision In Sequence Learning book now. This website allows unlimited access to, at the time of writing, more than 1.5 million titles, including hundreds of thousands of titles in various foreign languages.
Exploring and Exploiting Structure and Self-supervision in Sequence Learning
Sequence data, which consists of values organized in a certain order, is one of the most commonly seen data types in our everyday lives. For instance, the daily temperature and precipitation measurements throughout a year form a sequence of weather conditions. The crop yields each year over the past several decades depict a trend in agricultural production. These are also known as time series data. Time-indexed data is not the only kind of sequence data. Linguistic data such as speech and texts are sequential in nature. DNA sequences are indexed based on the physical order of the bases and materials' density of states are indexed by energy levels. In fact, any reasonably ordered data can be viewed as a sequence. Sequence data has been a long-standing area of interest in the artificial intelligence (AI) field, and this class of problems is often called sequence learning. Various kinds of sequence learning tasks have been defined, such as predicting the general properties of a sequence, tagging the sequence with labels at each index, or generating a new sequence from the input. Different types of sequence data have unique structures, and it is often challenging to develop a model to encode or decode the data taking account the inherent sequential relationship, so each sub-field has historically relied on separate sequence learning tools and frameworks. However, recent advances in machine learning (ML) and deep neural networks (DNN) have provided the capacity to handle arbitrarily long sequences and store historical states in a more unified fashion, regardless of the modality of the data. Deep models like recurrent neural networks (RNN), long short-term memory (LSTM), gated recurrent units (GRU) and transformers have become the foundation of most modern sequence learning and feature extraction methods. A new challenge is to efficiently and effectively utilize these deep models to capture intrinsic features from the input sequences. In this thesis, I will study both supervised and self-supervised sequence learning using deep models. Conventional methods for supervised sequence learning are typically designed to study sequences of scalar values or vectors, and are not suitable for structured data such as graphs. I will illustrate three novel yet challenging scenarios involving graphs and sequences: dynamic node property prediction for a fixed graph, sequence prediction from a graph, and multi-label prediction of sequential inputs. Structured input data can be modeled using a framework that combines graph neural networks (GNN) with sequence models (e.g., GRU and transformer). This framework is validated on several tasks, including crop yield prediction and density of states prediction. Self-supervised learning is another trending direction in the sequence learning field. Self-supervision obtains supervisory signals from the data itself and leverages the underlying structure in the data. It has the potential to improve the sample efficiency for downstream tasks and contribute to better model interpretability. In recent years, self-supervised sequence learning has been successfully applied to language and acoustic model pretraining. In my thesis, I will demonstrate that self-supervision can enforce latent structure, disentangle static and dynamic factors, and supplement supervised signals in model training, by applying it to speech recognition, video understanding and sequence generation. In general, I will show in this thesis different methods to capture and exploit structure from sequence data, and diverse explorations of the self-supervision for sequence learning.
ECAI 2023
Artificial intelligence, or AI, now affects the day-to-day life of almost everyone on the planet, and continues to be a perennial hot topic in the news. This book presents the proceedings of ECAI 2023, the 26th European Conference on Artificial Intelligence, and of PAIS 2023, the 12th Conference on Prestigious Applications of Intelligent Systems, held from 30 September to 4 October 2023 and on 3 October 2023 respectively in Kraków, Poland. Since 1974, ECAI has been the premier venue for presenting AI research in Europe, and this annual conference has become the place for researchers and practitioners of AI to discuss the latest trends and challenges in all subfields of AI, and to demonstrate innovative applications and uses of advanced AI technology. ECAI 2023 received 1896 submissions – a record number – of which 1691 were retained for review, ultimately resulting in an acceptance rate of 23%. The 390 papers included here, cover topics including machine learning, natural language processing, multi agent systems, and vision and knowledge representation and reasoning. PAIS 2023 received 17 submissions, of which 10 were accepted after a rigorous review process. Those 10 papers cover topics ranging from fostering better working environments, behavior modeling and citizen science to large language models and neuro-symbolic applications, and are also included here. Presenting a comprehensive overview of current research and developments in AI, the book will be of interest to all those working in the field.
Pattern Recognition
Author: Christian Bauckhage
language: en
Publisher: Springer Nature
Release Date: 2022-01-13
This book constitutes the refereed proceedings of the 43rd DAGM German Conference on Pattern Recognition, DAGM GCPR 2021, which was held during September 28 – October 1, 2021. The conference was planned to take place in Bonn, Germany, but changed to a virtual event due to the COVID-19 pandemic. The 46 papers presented in this volume were carefully reviewed and selected from 116 submissions. They were organized in topical sections as follows: machine learning and optimization; actions, events, and segmentation; generative models and multimodal data; labeling and self-supervised learning; applications; and 3D modelling and reconstruction.