Nonmodal model order reduction (MOR) techniques present accurate and efficient ways to approximate input–output behavior of large-scale mechanical structures. In this regard, Krylov-based model reduction techniques for second-order mechanical structures are typically known to require a priori knowledge of the original system parameters, such as expansion points (or eigenfrequencies). The calculation of the eigenfrequencies of the original finite-element (FE) model can be significantly time-consuming for large-scale structures. Existing iterative rational Krylov algorithm (IRKA) addresses this issue by iteratively updating the expansion points for first-order formulations until convergence criteria are achieved. Motivated by preserving the model properties of second-order systems, this paper extends the IRKA method to second-order formulations, typically encountered in mechanical structures. The proposed second-order IRKA method is implemented on a large-scale system as an example and compared with the standard Krylov and Craig-Bampton reduction techniques. The results show that the second-order IRKA method provides tangibly reduced error for a multi-input-multi-output (MIMO) mechanical structure compared to the Craig-Bampton. In addition, unlike the standard Krylov methods, the second-order IRKA does not require the information on expansion points, which eliminates the need to perform a modal analysis on the original structure. This can be especially advantageous for large-scale systems where calculations of the eigenfrequencies of the original structure can be computationally expensive. For such large-scale systems, the proposed MOR technique can lead to significant reductions of the computational time.