Defining a sound shift operator for graph signals, similar to the shift operator in classical signal processing, is a crucial problem in graph signal processing (GSP), since almost all operations, such as filtering, transformation, prediction, are directly related to the graph shift operator. We define a set of energy-preserving shift operators that satisfy many properties similar to their counterparts in classical signal processing, but are different from the shift operators defined in the literature, such as the graph adjacency matrix and Laplacian matrix based shift operators, which modify the energy of a graph signal. We decouple the graph structure represented by eigengraphs and the eigenvalues of the adjacency matrix or the Laplacian matrix. We show that the adjacency matrix of a graph is indeed a linear shift invariant (LSI) graph filter with respect to the defined shift operator. We further define autocorrelation and cross-correlation functions of signals on the graph, enabling us to obtain the solution to the optimal filtering on graphs, i.e., the corresponding Wiener filtering on graphs and the efficient spectra analysis and frequency domain filtering in parallel with those in classical signal processing. This new shift operator based GSP framework enables the signal analysis along a correlation structure defined by a graph shift manifold as opposed to classical signal processing operating on the assumption of the correlation structure with a linear time shift manifold. Several illustrative simulations are presented to validate the performance of the designed optimal LSI filters.