Blogs

How to approximate difficult-to-compute probability densities is an important problem in statistics. Variational Inference (VI) is a statistical inference framework that addresses this problem using optimization. This allows the use of it along with modern and fast optimization techniques which is ideal to approximate probability density functions of large datasets and complex models. In this post I’m going to review Variational Inference, explaining the concepts that it involves, its derivation from the variational methods and its implications in the bayesian inference problem and in current machine learning techniques....

This post is an analysis of the actual normalization techniques and why and how to implement them for neural networks. It’s based on the paper of Ioffee and Szegedy [1] from 2015, the modification proposed for Layer Normalization [2] and a much more recent work of Group Normalization [3]. Batch Normalization (BN) is a milestone technique in the development of deep learning. - Group Normalization paper. One of the big difficulties of deep neural networks is the training stage and its optimization....