This study investigates and compares the various combinations of renewable energies (solar, wind) and storage technologies (battery, pumped hydro storage, hybrid storage) for an off-grid power supply system. Four configurations (i.e., single RE source system, double RE source system, single storage, and double storage system) based on two scenarios (self-discharge equal to 0% and 1%) are considered, and their operational performance is compared and analyzed. The energy management strategy created for the hybrid pumped battery storage (HPBS) considers that batteries cover low energy surplus/shortages while pumped hydro storage (PHS) is the primary energy storage device for serving high-energy generations/deficits. The developed mathematical model is optimized using Particle Swarm Optimization and the performance and results of the optimizer are discussed in particular detail. The results evidence that self-discharge has a significant impact on the cost of energy (13%–50%) for all configurations due to the substantial increase in renewable energy (RE) generators size compared to the energy storage capacity. Even though solar-wind-PHS is the cost-optimal arrangement, it exhibits lower reliability when compared to solar-wind-HPBS. The study reveals the significance of HPBS in the off-grid RE environment, allowing more flexible energy management, enabling to guarantee a 100% power supply with minimum cost and reducing energy curtailment. Additionally, this study presents and discuss the results of a sensitivity analysis conducted by varying load demand and energy balance of all considered configurations is performed, which reveals the effectiveness of the supplementary functionality of both storages in hybrid mode. Overall, the role of energy storage in hybrid mode improved, and the total energy covered by hybrid storage increased (48%), which reduced the direct dependency on variable RE generation.