Correct identification of species and numbers of individuals is the key to biodiversity assessment. Crickets are an integral part of tropical forest ecosystems and are conspicuous due to their acoustic calling songs. Cricket calling songs are species-specific, providing reliable cues for non-invasive species identification. A cost-effective and widely used method is trained listener-based acoustic sampling. However, the effectiveness and reliability of this method has rarely been assessed in a quantitative manner. We evaluated trained listener-based acoustic sampling as a reliable and non-invasive method for rapid assessment of cricket species diversity in tropical evergreen forests. We carried out psychoacoustic experiments in the laboratory to assess the effectiveness of species identification and estimation of numbers of calling individuals by a trained listener. Further, we compared psychoacoustic sampling done in the field with the ambient noise recordings that were done simultaneously. The reliability of correct species identification by the trained listener was 100 % for 16 out of 20 species tested in the laboratory. The reliability of identifying the numbers of individuals correctly was 100% for 13 out of 20 species. The human listener performed slightly better than the instrument in detecting low frequency and broadband calls in the field, whereas the recorder detected high frequency calls with greater probability. We propose that for accurate estimation of ensiferan species richness and relative abundance in an area, trained listener-based psychoacoustic sampling is preferable for crickets and low frequency katydids, whereas broadband recorders are preferable for katydid species with high frequency calls.