Abstarct: The stock market is known as a major plaxyer in the financial sector of many countries. Most brokerages, which often trade stocks, use technical or fundamental methods to make more profit. However, most of these methods do not work well due to the non-linear and unstable nature of financial markets. In this work, unlike most of the work done to predict stock prices in the coming days, an attempt has been made to use artificial intelligence-baxsed methods to provide a system for optimizing the stock trading strategy. To achieve this goal, first, the Relative strength index(RSI) indicator is calculated baxsed on the closing prices of stocks in recent days, which limits the changes to the range of 0 to 100 values of this indicator. Then this interval is divided into four parts and then using the capabilities of the firefly algorithm in the field of optimal search in the problem space, an attempt is made to find optimal values for each of these parts. Then using these optimal values, considering the RSI previous day value in which of these four areas is, the trade is done on the next trading day. The results show that this system has improved compared to the similar system baxsed on genetic algorithm. In the population size parameter with a maximum number, it has improved at least 6.4% in telecommunications stocks and a maximum of 31% in Parsian stocks. In the data set size parameter, it has improved at least 11% in telecommunication stocks and at most 40% in Parsian stocks.