This paper presents a fundamental investigation on optimizing the power takeoff (PTO) for maximizing wave energy conversion of the wave-activated bodies (WABs) wave energy converters. In this research, two relative heave motions are taken for capture power, and a linear PTO is considered in the primary analysis. For such a linear dynamic system, the frequency-domain analysis can be carried out, and an analytical formula can be derived for the optimized frequency-dependent PTO damping coefficient, which can be used to determine an optimized PTO damping for maximizing wave energy conversion in regular waves. However, when an optimized PTO is required for maximizing energy conversion from ocean waves, the PTO damping optimization may be very different and much less certain, because it may be based on one of many characteristic periods of the given sea state and the dependency may change from different sea states. For this reason, the optimized PTO damping coefficient for a given sea state must be studied carefully. Another important aspect of the research is to examine whether an optimized nonlinear PTO can take more energy out from waves than that of an optimized linear PTO. For this purpose, the maximized capture powers by the optimized linear and nonlinear PTOs are compared using the time-domain analysis. It has been shown from the examples that the maximized capture power by a nonlinear PTO system may exceed that by the linear PTO, but only marginally (less than 1%). Hence, it can be generally concluded that the maximized power using a linear PTO system can be a very good indicator for the device in extracting the maximal energy from waves regardless of the linear or nonlinear PTO in actual use. This conclusion may help simplify the analysis of the wave energy converters in terms of the energy production as well as the device optimization for improving energy conversion capacity.