The COVID-19 'infodemic' continues to undermine trust in vaccination efforts aiming to bring an end to the pandemic. However, the challenge of vaccine hesitancy is not only a problem of the information ecosystem and it often has little to do with the vaccines themselves. In this Perspective, we argue that the epidemiological and social crises brought about by COVID-19 have magnified widely held social anxieties and trust issues that, in the unique circumstances of this global pandemic, have exacerbated skepticism toward vaccines. We argue that trust is key to overcoming vaccine hesitancy, especially in a context of widespread social uncertainty brought about by the pandemic, where public sentiment can be volatile. Finally, we draw out some implications of our argument for strategies to build vaccine confidence.