Simple cold-rolling and annealing of martensite starting structure can produce ultrafine grained structure in carbon steel. The microstructural evolution during the process was studied in a 0.13%C steel. The ultrafine lamellar dislocation cells (LDCs) with mean thickness of 60 nm were mainly observed in a 50% cold-rolled specimen as well as the irregularly bent lamellas (IBLs) and the kinked laths (KLs). The LDCs and the IBLs had large local misorientations. The specimens annealed at temperatures from 723 to 773 K showed the multiphased ultrafine structure composed of equiaxed ultrafine ferrite grains with the mean grain size of 180 nm, nano-carbides distributed uniformly and small blocks of tempered martensite. The formation of the ultrafine grained structure was discussed from the viewpoint of characteristics of the martensite starting structure. It was concluded that the fine grained structure of martensite play an important role for ultrafine grain subdivision during plastic deformation.