Numerical investigations on the rotordynamic characteristics of a typical hole-pattern seal using transient three-dimensional Reynolds-averaged Navier–Stokes (RANS) solution and the periodic circular orbit model were conducted in this work. The unsteady solutions combined with mesh deformation method were utilized to solve the three-dimensional RANS equations and obtain the transient reaction forces on a typical hole-pattern seal rotor at five different excitation frequencies. The relation between the periodic reaction forces and frequency dependent rotordynamic coefficients of the hole-pattern seal was obtained by considering the rotor with a periodic circular orbit (including forward orbit and backward orbit) of the seal center. The rotordynamic coefficients of the hole-pattern seal were then solved based on the obtained unsteady reaction forces and presented numerical method. Compared with the experimental data, the predicted rotordynamic coefficients of the hole-pattern seal are more agreeable with the experiment than that of the ISO-temperature (ISOT) bulk flow analysis and numerical approach with one-direction-shaking model. Furthermore, the unsteady leakage flow characteristics in the hole-pattern seal were also illustrated and discussed in detail.