More and more attention has been paid to reduce the low frequency interior noise of the elastic cavity, such as automobiles, ships, airplanes, and railway vehicles, to provide the more comfortable riding environment for passengers. Identification of the interior acoustical sources in the low frequency range is vitally important for the sound quality design inside the elastic cavity. By transformation of the sound pressure level into the specific loudness, a multipoint panel acoustic contribution method based on Moore–Glasberg loudness model is proposed to identify the acoustic contribution of local structural panels of an elastic cavity. The finite element (FE) equation of vibro-acoustic coupling structure with the visco-elastic damping is formulated to evaluate the acoustic panel contribution. Two parameters of acoustic contribution sum and total sound field contribution are derived to measure the acoustic contribution of each panel at the important peak frequencies for the multiple evaluation points. A carriage of high-speed train is modeled as the elastic cavity to demonstrate the application of the developed algorithm. The bottom panel of the carriage is identified to make the most significant contribution to the loudness of evaluation points. The reduction effect of the various design parameters of visco-elastic damping layer on the bottom panel is investigated. The proposed method can efficiently arrange the visco-elastic damping layer on the bottom panel to reduce the interior loudness.