Failure probability of system components may vary with the changes of weather, environment and other operation conditions. The time-varying or condition-based failure probability can be represented using a fuzzy model. Based on credibility theory, this paper presents a novel operation risk assessment method to handle the two-fold uncertainty combining randomness and fuzziness in power system operations. In the proposed method, a random fuzzy model is developed to accommodate impacts due to various operation conditions and other factors on the failure probability of system components. The WECC nine-bus system, IEEE 14-bus system and an actual power system in Northeast China were used to demonstrate feasibility and applicability of the presented method. The operation risk assessment module using the proposed method has been embedded in the EMS system in a district control center of a power company to perform real-time operation risk assessment.