Variational image segmentation based on the Mumford and Shah model [31], together with implementation by the piecewise constant level-set method (PCLSM) [26], leads to fully nonlinear Total Variation (TV)-Allen-Cahn equations. The commonly-used numerical approaches usually suffer from the difficulties not only with the non-differentiability of the TV-term, but also with directly evolving the discontinuous piecewise constant-structured solutions. In this paper, we propose efficient dual algorithms to overcome these drawbacks. The use of a splitting-penalty method results in TV-Allen-Cahn type models associated with different "double-well" potentials, which allow for the implementation of the dual algorithm of Chambolle [8]. Moreover, we present a new dual algorithm based on an edge-featured penalty of the dual variable, which only requires to solve a vectorial Allen-Cahn type equation with linear ∇(div)-diffusion rather than fully nonlinear diffusion in the Chambolle's approach. Consequently, more efficient numerical algorithms such as time-splitting method and Fast Fourier Transform (FFT) can be implemented. Various numerical tests show that two dual algorithms are much faster and more stable than the primal gradient descent approach, and the new dual algorithm is at least as efficient as the Chambolle's algorithm but is more accurate. We demonstrate that the new method also provides a viable alternative for image restoration.