|
|
|
|
Professor Lei Zhang
(Dept. of Computing, The Hong Kong Polytechnic University)
Title of Talk : Gradient centralization and feature gradient decent for deep neural network optimization
Abstract:
The normalization methods are very important for the effective and efficient training of deep neural networks (DNNs). Many popular normalization methods operate on weights, such as weight normalization and weight standardization. We propose a very simple yet effective DNN optimization technique, namely gradient centralization (GC), which operates on the gradients of weights directly. GC simply centralizes the gradient vectors to have zero mean. It can be easily embedded into the current gradient based optimization algorithms with just one line of code. GC demonstrates various desired properties, such as accelerating the training process, improving the generalization performance, and the compatibility for fine-tuning pre-trained models.
On the other hand, existing DNN optimizers such as stochastic gradient descent (SGD) mostly perform gradient descent on weight to minimize the loss,
while the final goal of DNN model learning is to obtain a good feature space for data representation.
Instead of performing gradient descent on weight, we propose a method, namely feature SGD (FSGD), to approximate the output feature with one-step gradient descent for linear layers.
FSGD only needs to store an additional second-order statistic matrix of input features,
and use its inverse to adjust the gradient descent of weight.
FSGD demonstrates much better generalization performance than SGD in classification tasks.
Professor Andreas Dengel
(DFKI & University of Kaiserslautern, Germany)
Title of Talk : Combining Bird Eye View and Grass Root View for Earth Observation
Abstract:
This talk will address the multiple opportunities presented by the use of AI approaches to combine the analysis of satellite imagery and data collected on the ground.
In particular, three exemplary areas, namely disaster management, air pollution monitoring, and agricultural yield management, will be discussed to demonstrate the added value of both data sources.
For the first domain, I will show how multimedia data from real-time social media monitoring and spectral data from Earth observation can be tied together to provide deep insight and foresight into disasters.
In the second case, I will explain how mapping human settlements in combination with Sentinel 5 data can reveal the impact of air pollution.
In the third case, I will discuss some insights into monitoring agricultural processes using sensor data from agricultural machinery.
In doing so, I will highlight the potential to combine the relevant findings with approaches to classifying land cover and land use using remote sensing.
Professor Jure Leskovec
(Dept. of Computer Science at Stanford University)
Title of Talk : Graph Neural Networks and Beyond
Abstract:
Machine learning on graphs is an important and ubiquitous task with applications ranging from drug design to friendship recommendation in social networks.
The primary challenge in this domain is finding a way to represent, or encode, graph structure so that it can be easily exploited by machine learning models.
In this talk I will discuss recent advancements in the field of Graph Neural Networks that automatically learn to encode graph structure into low-dimensional embeddings, using techniques based on deep learning.
I will provide a conceptual overview of key advancements in this area of representation learning on graphs, including graph convolutional networks and their representational power.
We will also discuss development of graph-learning benchmarks as well as open research problems.
|
|
|
|
|
Today's Visits
Total Visits
|
|
Address: International Center for Converging Technology, Room 202, Korea University, 145 Anam-ro, Sengbuk-gu, Seoul, 02841, Korea
Tel: +82-2-3290-5920, Business License Number: 302-82-07147
|
E-mail comments to
acpr.korea@gmail.com
Last update: 27, Jan. 2021
|
|
