In order to improve the stability of the wave power generation device, the mechanism of variable displacement control mechanism of hydraulic motor is proposed and the control strategy of motor speed is built. The detailed mathematical representation and dynamic characteristics of motor control are analyzed; the adaptive sliding mode control method and a back-stepping control strategy are designed for the proposed controller to overcome the influence of the parameter uncertainty in the nonlinear system. The adaptive law is designed by Lyapunov criterion to ensure the stability and convergence of the whole closed-loop system. In order to verify the feasibility of the proposed control strategy, a wave power generation land experimental platform is designed, the experimental platform is mainly consists of the test bench bracket, the hydraulic transmission system and the electric power processing system. Through comparative analysis with PID control method, the advantages of this control method are described and the speed of motor can be controlled accurately.