Estimation of the value-at-risk (VaR) of a large portfolio of assets is an important task for financial institutions. As the joint log-returns of asset prices can often be projected to a latent space of a much smaller dimension, the use of a variational autoencoder (VAE) for estimating the VaR is a natural suggestion. To ensure the bottleneck structure of autoencoders when learning sequential data, we use a temporal VAE (TempVAE) that avoids the use of an autoregressive structure for the observation variables. However, the low signal-to-noise ratio of financial data in combination with the auto-pruning property of a VAE typically makes use of a VAE prone to posterior collapse. Therefore, we use annealing of the regularization to mitigate this effect. As a result, the auto-pruning of the TempVAE works properly, which also leads to excellent estimation results for the VaR that beat classical GARCH-type, multivariate versions of GARCH and historical simulation approaches when applied to real data.