Knowledge resources have been attached much attention by enterprises, and knowledge transfer has become one of the ways for enterprises to gain competitive advantage. In order to promote the knowledge transfer and improve the efficiency of collaborative work in prefabricated building, a hierarchical structure model based on the theory of interpretative structure model is constructed, which is based on the main influencing factors of knowledge transfer in prefabricated building, the driving force-dependence is analyzed with MICMAC. The results show that the key influencing factors are in a five-level hierarchical structure, the transfer ability, the absorptive ability of the recipient and the knowledge potential difference among employees are the objective stable factors of individual differences, the transfer intention and the organizational structure of the enterprise are the bottom factors, the strongest driving force for the entire structural system.