Aim The Comprehensive Assessment of At‐Risk Mental States (CAARMS) was specifically developed to assess and detect young people at ultra‐high risk (UHR) of developing psychosis. The current study was undertaken to test the reliability and validity of the authorized Italian version of the CAARMS (CAARMS‐ITA) in a help‐seeking population. Methods Psychometric properties of the CAARMS‐ITA were established using a sample of 223 Italian adolescents and young adults aged between 13 and 35 years, who were divided into 3 groups according to the CAARMS criteria: UHR‐negative individuals (UHR −; n = 64), UHR‐positive (UHR +; n = 55) and individuals with a first‐episode psychosis (FEP; n = 104). The CAARMS‐ITA's reliability was tested measuring interrater reliability and internal consistency. Construct validity was tested comparing the Positive and Negative Syndrome Scale (PANSS) and CAARMS‐ITA subscale scores across groups (ie, UHR −, UHR + and FEP). For concurrent validity, we studied correlations between symptoms of the CAARMS‐ITA and their equivalents in the PANSS. Finally, the predictive validity was examined by following up with UHR + individuals. The 12‐month transition rate to psychosis was calculated. Results The CAARMS‐ITA showed good interrater reliability. The PANSS “Positive Symptoms” subscale scores in UHR + individuals were intermediate between FEP and UHR − groups. The positive and negative symptoms scores of the CAARMS‐ITA significantly correlated with the corresponding scores of the PANSS. After 12 months, 4 of 41 (9.8%) UHR + individuals had transitioned to psychosis. Conclusions The CAARMS‐ITA is a reliable and valid instrument for assessing and detecting at‐risk mental states in Italian clinical settings. It also appears to be helpful in the prediction of psychosis transition.