Using a toolkit of theoretical techniques comprising ab initio density functional theory calculations, the nudged elastic band method and kinetic Monte Carlo (KMC) modeling, we investigate in great detail how para-terphenyl-meta-dicarbonitrile (pTmDC) molecules diffuse and isomerize to self-assemble on the Ag(111) surface. We show that molecules "walk" on the surface via a pivoting mechanism moving each of its two "legs" one at a time. We then identify a peculiar "under-side" isomerization mechanism capable of changing the molecules chirality, and demonstrate that it is fundamental in understanding the growth of hydrogen bonding assembles of ribbons, linkers, clusters and brickwall islands on the Ag(111) surface, as observed in recent scanning tunneling microscopy experiments (ChemPhysChem, 2010, 11, 1446). The discovered underlying atomistic mechanism of self-assembly may be behind the growth of other hydrogen bonding structures of chiral molecules on metal surfaces.