We are happy to announce that our paper "Dynamic Channel and Layer Gating in Convolutional Neural Networks" by Ali Ehteshami Bejnordi and Ralf Krestel was accepted at the 43rd German Conference on Artificial Intelligence. The paper is based on Ali Ehteshami Bejnordi's Master thesis he has written at the University of Passau.
Abstract:
Convolutional neural networks (CNN) are getting more and more complex, needing enormous computing resources and energy. In this paper, we propose methods for conditional computation in the context of image classification that allows a CNN to dynamically use its channels and layers conditioned on the input. To this end, we combine light-weight gating modules that can make binary decisions without causing much computational overhead. We argue, that combining the recently proposed channel gating mechanism with layer gating can significantly reduce the computational cost of large CNNs. Using discrete optimization algorithms, the gating modules are made aware of the context in which they are used and decide whether a particular channel and/or a particular layer will be executed. This results in neural networks that adapt their own topology conditioned on the input image. Experiments using the CIFAR10 and MNIST datasets show how competitive results in image classification with respect to accuracy can be achieved while saving up to 50% computational resources.
