For integrated sensing and communication, one important research direction is to employ various beamforming techniques to avoid interference between the two functions. In this work, based on a hybrid beamforming antenna array structure, a physical layer security technique called directional modulation (DM) is studied for multi-beam multiplexing applications. The proposed design can form a more effective directional transmission through both beamforming and DM, while multiplexing multiple user beams through a common set of analog coefficients. In this hybrid beamforming structure, only one digital-to-analog converter (DAC) is connected to each subarray, and finite-precision phase shifters are further considered. Design examples for dual-beam multiplexing with an interleaved subarray structure and a localized subarray structure, respectively, are provided, which show that the interleaved subarray structure can form narrower mainlobe and a lower sidelobe level than the localized structure and has an overall better performance.