With the development of China's real estate market, the real estate intermediary industry has not only provided people with information and services, but also appeared various violations, which not only seriously infringes on the legitimate rights and interests of stakeholders, but also hindered the development of the industry. Therefore, from the perspective of game theory, the punishment system was designed for the illegal operation of real estate agents by analyzing the behavior of government regulatory departments and real estate agents, combined with the relevant theories of institutional engineering, the effectiveness of the system through MATLAB simulation, and puts forward improvement suggestions for the existing regulatory system and real estate intermediary industry.