Various wave energy converters (WECs) have been developed to utilize ocean wave energy resources. Among them, an oscillating water column (OWC) device is a promising wave energy converter that has undergone several pilot tests in the ocean with various prototype devices. Recently, a breakwater-integrated OWC-WEC system was proposed to improve the economic feasibility by reducing the construction and installation costs of the OWC chamber structure. In this study, a numerical investigation was conducted to simulate the hydrodynamic energy conversion problem for a breakwater-integrated OWC-WEC. The boundary value problem with the Laplace equation was solved directly by using a three-dimensional numerical wave tank based on a finite element method in the time domain. In the present numerical method, an efficient numerical model for the tetrapod (TTP), which is used to protect a rubble mound breakwater, was introduced using multiple blocks and numerical damping zone. Further, numerical investigations were performed on the effects of breakwater integration, TTP layer modeling, and the application of the numerical damping zone, and their features in terms of the wavefield and the OWC chamber responses were analyzed. Finally, the validity of the present numerical method was examined by comparing it with the experimental data.