•Unifying framework for dealing with different models in uncertain optimization•Characterization based on set optimization, vector optimization, and scalarization•Robust optimization with infinite uncertainty sets•New modeling approaches for uncertainties in optimization problems In this paper we consider uncertain scalar optimization problems with infinite scenario sets. We apply methods from vector optimization in general spaces, set-valued optimization and scalarization techniques to develop a unified characterization of different concepts of robust optimization and stochastic programming. These methods provide new insights on the interrelation between different concepts for handling uncertainties and naturally lead to new concepts of robustness.