This paper is concerned with a nonautonomous Hamiltonian system with two degrees of freedom whose Hamiltonian is a 2π-periodic function of time and analytic in a neighborhood of an equilibrium point. It is assumed that the system exhibits a secondorder resonance, i. e., the system linearized in a neighborhood of the equilibrium point has a double multiplier equal to −1. The case of general position is considered when the monodromy matrix is not reduced to diagonal form and the equilibrium point is linearly unstable. In this case, a nonlinear analysis is required to draw conclusions on the stability (or instability) of the equilibrium point in the complete system.

In this paper, a constructive algorithm for a rigorous stability analysis of the equilibrium point of the above-mentioned system is presented. This algorithm has been developed on the basis of a method proposed in [1]. The main idea of this method is to construct and normalize a symplectic map generated by the phase flow of a Hamiltonian system.

It is shown that the normal form of the Hamiltonian function and the generating function of the corresponding symplectic map contain no third-degree terms. Explicit formulae are obtained which allow one to calculate the coefficients of the normal form of the Hamiltonian in terms of the coefficients of the generating function of a symplectic map.

The developed algorithm is applied to solve the problem of stability of resonant rotations of a symmetric satellite.