Foreign Exchange or Forex is the sale purchase market point of foreign currency pairs. Due to the high volatility in the forex market, it is difficult to predict the future price of any currency pair. This study shows that a significant enhancement in the prediction of forex price can be achieved by incorporating domain knowledge in the process of training machine learning models. The proposed system integrates the Forex Loss Function (FLF) into a Long Short-Term Memory model called FLF-LSTM — that minimizes the difference between the actual and predictive average of Forex candles. Using the data of 10,078 four-hour candles of EURUSD pair, it is found that compared to the classic LSTM model, the proposed FLF-LSTM system shows a decrease in overall mean absolute error rate by 10.96%. It is also reported that the error in forecasting the high and low prices is reduced by 10% and 9%, respectively. The proposed model, in comparison to the Recurrent Neural Network-based prediction system, shows an overall reduction of 73.57% in mean absolute error, by exhibiting up to 68.71% and 72.31% error reduction in high and low prices, respectively. In comparison to Auto-Regressive Integrated Moving Average, our proposed model shows a 13% reduced error. Specifically, in the open, high, and low prices, the error is reduced by 28.5%, 14.2%, 9.3%, respectively. Finally, we compare our model with another well-known time series forecasting model, i.e., FB Prophet — where FLF-LSTM demonstrates 31.8%, 47.7%, 23.6%, 47.7% error reduction in open, high, low, and close prices, respectively. The data and the code used in this study can be accessed at the following URL: https://github.com/slab-itu/forex_flf_lstm. •A novel function has been designed for the Forex Prediction System called FLF-LSTM.•The FLF-LSTM shows a decrease in the prediction error rate by 14%.•The FLF-LSTM outperforms FB Prophet, ARIMA, and RNN-based models of Forex prediction.•The proposed implementation opens a new gateway to predict Forex market trends.