Optimization problem of cardinality constrained mean-variance (CCMV) model for sparse portfolio selection is considered. To overcome the difficulties caused by cardinality constraint, an exact penalty approach is employed, then CCMV problem is transferred into a difference-of-convex-functions (DC) problem. By exploiting the DC structure of the gained problem and the superlinear convergence of semismooth Newton (ssN) method, an inexact proximal DC algorithm with sieving strategy based on a majorized ssN method (siPDCA-mssN) is proposed. For solving the inner problems of siPDCA-mssN from dual, the second-order information is wisely incorporated and an efficient mssN method is employed. The global convergence of the sequence generated by siPDCA-mssN is proved. To solve large-scale CCMV problem, a decomposed siPDCA-mssN (DsiPDCA-mssN) is introduced. To demonstrate the efficiency of proposed algorithms, siPDCA-mssN and DsiPDCA-mssN are compared with the penalty proximal alternating linearized minimization method and the CPLEX (12.9) solver by performing numerical experiments on realword market data and large-scale simulated data. The numerical results demonstrate that siPDCA-mssN and DsiPDCA-mssN outperform the other methods from computation time and optimal value. The out-of-sample experiments results display that the solutions of CCMV model are better than those of other portfolio selection models in terms of Sharp ratio and sparsity.