In recent dynamic problems dealing with high-frequency excitations, such as ultrasonic vibrations, a proper representation of rods transmitting kinetic energy from the interface attached to the vibrating system to the other end is strongly demanded for effectively reducing computational time and domain. A highly reduced lumped parameter model that properly simulates the dynamic characteristics of a uniform, isotropic, homogeneous, and viscoelastic rod subjected to excitations at its end is proposed in this paper. The model consists of springs, dashpots, and so called “gyro-mass elements.” The gyro-mass element generates a reaction force proportional to the relative acceleration of the nodes between which it is placed. This model consists of units arranged in series, each unit consisting of a spring, a dashpot, and a gyro-mass element arranged in parallel. A formula is proposed for determining the properties of the elements in the units based on the modal expansion. The results show that a notable reduction of 90% in the degrees of freedom is accomplished with high accuracy by using the proposed model consisting of a set of units associated with modes in a target frequency region and a supplemental unit associated with residual stiffness, which is advantageous for efficient numerical computations in recent dynamic problems.