•Localized Trefftz collocation scheme is proposed for heat conduction analysis.•Generalized reciprocity method is proposed to deal with the nonhomogeneous terms.•The proposed method inherits semi-analytical property and avoids dense matrix.•A large-scale simulation with almost 100,000 discretization nodes is performed. This paper presents a novel localized collocation Trefftz method (LCTM) for heat conduction analysis in two kinds of heterogeneous materials (functionally graded materials and multi-medium materials) under temperature loading. In contrast to the conventional collocation Trefftz method (CTM), the proposed LCTM divides the whole domain into many stencil support domains consisting of several discretization nodes. An efficient technique, the generalized reciprocity method (GRM), is proposed to derive the problem-dependent T-complete functions for approximating the particular solution of the nonhomogeneous equations in the stencil support domains. Based on the moving least square (MLS) technique and T-complete functions, the LCTM numerical differentiation formulation at a certain node can be derived by using a linear combination of the T-complete functions at its adjacent discretization nodes in the related stencil support domain. It inherits the semi-analytical property from the conventional CTM and avoids the ill-conditioned dense matrix problem. Besides, the quadrant criterion is applied to guarantee the stability of the proposed LCTM, and the domain decomposition method (DDM) is introduced to divide the multi-medium domains into several single-medium sub-domains. Numerical results demonstrate the accuracy and efficiency of the proposed LCTM in comparison with the known analytical solutions and the finite element method (FEM) results.