Human Immunodeficiency Virus (HIV) is a major global public health issue that affects the quality of life (QoL) of people living with HIV (PLHIV) globally and in Indonesia. As a part of a large-scale qualitative study investigating HIV risk factors and impacts on PLHIV and facilitators of and barriers to their access to HIV care services in Yogyakarta and Belu, Indonesia, this paper describes their in-depth views and experiences of the influence of HIV on their QoL. Ninety-two participants were recruited using the snowball sampling technique. Data were collected using in-depth interviews. In addition, the World Health Organisation Quality of Life questionnaire (WHOQOL-HIV BREF) was also distributed to each of them to fill out prior to the interviews. Chi-Square analysis was used to analyse data from the survey and a framework analysis was applied to guide qualitative data analysis. The findings reported several factors affecting the QoL of the participants. These included (i) environmental factors, such as living in rural areas, the unavailability of HIV care services and public transport, and long-distance travel to healthcare facilities; (ii) personal beliefs associated with HIV; (iii) sexual and social relationships and their influence of the QoL of participants; and (iv) level of independence and physical health condition following HIV diagnosis. The findings indicate the need for intervention programs that address the availability and accessibility of HIV care services to PLHIV within rural communities and support various physical, psychological, and financial needs of PLHIV. These can be implemented by providing supplements and nutritious food, HIV counselling and door-to-door/community-based ART service delivery to PLHIV, which may increase their engagement in and adherence to the treatment and improve their physical and psychological condition and QoL.