The TAIGA experiment in Tunka valley is expanding the present scintillation detector array with new TAIGA-Muon detector stations. A simulation model was developed for the new stations optimization and study of the identification performance of the array together with HiScore optical stations system. The extensive air showers (EASs) were simulated with the CORSIKA simulation tool, and the detector response was simulated with the GEANT4 package. EASs induced by gamma quanta, proton and nuclei from He to Fe at energies 1 and 3 PeV have been studied. For the identification of the EAS origin, a method based on a neural network was suggested. Preliminary results show that with this method, using information from scintillation and optical detectors it is possible to achieve proton EAS suppression factor of 1000 while having gamma-quanta EAS detection efficiency about 50%. The analogous method was tested for cosmic rays element composition study. Three groups of elements -- p-He, CNO and Fe could be separated with efficiencies variyng from 30 to 90% depending on the EAS type and predefined cuts.