Phosphorene, a single atomic layer of black phosphorus, has recently emerged as a new two-dimensional (2D) material that holds promise for electronic and photonic technologies. Here we experimentally demonstrate that the electronic structure of few-layer phosphorene varies significantly with the number of layers, in good agreement with theoretical predictions. The interband optical transitions cover a wide, technologically important spectral range from the visible to the mid-infrared. In addition, we observe strong photoluminescence in few-layer phosphorene at energies that closely match the absorption edge, indicating that they are direct bandgap semiconductors. The strongly layer-dependent electronic structure of phosphorene, in combination with its high electrical mobility, gives it distinct advantages over other 2D materials in electronic and opto-electronic applications.