Hadamard matrix is defined as a square matrix where any components are -1 or +1, and where any pairs of rows are mutually orthogonal. In this work, we consider the similar matrix on finite field GF(p) where p is an odd prime. In such a matrix, every component is one of the integers on GF(p)\{0}, that is, {1,2,...,p-1}. Any additions and multiplications should be executed under modulo p. In this paper, a method to generate such matrices is proposed. In addition, the paper includes the applications to generate n-shift orthogonal sequences and complete complementary codes. The generated complete complementary code is a family of multi-valued sequences on GF(p)\{0}, where the number of sequence sets, the number of sequences in each sequence set and the sequence length depend on the various divisors of p-1. Such complete complementary codes with various parameters have not been proposed in previous studies.