Rubbing between the rotor and the stator is the frequent and harmful malfunction in the rotating machinery, which will cause a very serious accident, even catastrophe, to the machine. Thus, such fault needs timely detection to avoid severe consequences. Vibration based methods are the traditional ones for the detection and diagnosis of the rubbing fault, which are effective when the rubbing has been severe, but always do not work in early detection of such fault. Rubbing between the rotor and the stator will cause elastic strain in the rubbing location and thus can produce acoustic emission (AE). Apparently, such AE contains direct and abundant information about the rubbing and thus can be used to detect and diagnose such fault effectively. In this paper, the AE based method is proposed for detecting and identifying the rubbing of the rotor-bearing system. Using the modal acoustic emission (MAE) theory and reassigned wavelet scalogram, the present study emphasizes on modal analysis and time-frequency characteristics analysis for further understanding the characteristics, phenomenon, and features of the rubbing AE. The results show that the rubbing AE is the elastic wave with multiple modals, which has an impact characteristic and mainly consists of the flexural wave and extensional wave. The transmission of the rubbing AE has some directivity. Moreover, different modals have different transmission characteristics: The flexural wave has lower frequency and its attenuation is influenced mainly by the sectional area of the transmission passage. The extensional wave has higher frequency and its attenuation is mainly sensitive to the interface transmission between the two surfaces. The results also reveal that the reassigned wavelet scalogram is more effective than its original scalogram for the characteristic analysis of the rubbing AE.