This study compared the relationship between social participation, including work, and incidence of functional decline in rural and urban older people in Japan, by focusing on the number and types of organizations older people participated in. The longitudinal data of the Japan Gerontological Evaluation Study (JAGES) that followed 55,243 individuals aged 65 years or older for six years were used. The Cox proportional hazards model was employed to calculate the hazard ratio (HR) of the incidence of functional decline over six years and the stratification of rural and urban settings. In this model, we adjusted 13 variables as behavioral, psychosocial, and functional confounders. The more rural and urban older people participated in various organizations, the more they were protected from functional decline. Participation in sports (HR: rural = 0.79; urban = 0.83), hobby groups (HR: rural = 0.76; urban = 0.90), and work (HR: rural = 0.83; urban = 0.80) significantly protected against the incidence of decline in both rural and urban areas. For both rural and urban older people, promoting social participation, such as sports and hobby groups and employment support, seemed to be an important aspect of public health policies that would prevent functional decline.