High voltage is required for the existing dielectric elastomer (DE) actuators to convert electrical energy to mechanical energy. However, maintaining high voltage on DE membranes can cause various failures, such as current leakage and electrical breakdown, which limits their practical applications, especially in small-scale devices. To overcome the above drawback of DE actuators, this paper proposes a new actuation method using DE membranes with a properly designed bistable structure. Experiment shows that the actuator only requires a high-voltage pulse to drive the structure forward and backward with electromechanical snap-through instability. The actuator can maintain its stroke when the voltage is removed. An analytical model based on continuum mechanics is developed, showing good agreement with experiment. The study may inspire the design and optimization of DE transducers.