Human activities are affecting biodiversity to a greater extent than ever. Consequently, tools that can efficiently monitor changes in communities are becoming increasingly important. In the case of birds and other vocalizing animals, it has been suggested that passive acoustic methods can be used for this purpose. Multiple acoustic indices have been developed recently, to be used as proxies for species diversity. Preliminary results have been promising. Yet, before the indices can be applied widely, it is necessary to understand better how well they reflect the communities to be monitored, and how they perform under diverse environmental conditions. Here, we tested seven of the available indices, on sound recordings made in two biodiverse regions in Yunnan Province, south China. We assessed each index’s performance by measuring its correlation to bird species richness and diversity, estimated using point-count surveys. Each survey was conducted by an expert observer, at the same time each recording was made, and for the same duration. We also tested whether the performance of the indices was affected by levels of environmental dissimilarity between the sites sampled. We found that although no index showed a very strong correlation with species richness or diversity, three indices (the acoustic entropy, acoustic diversity and acoustic evenness indices) performed consistently better that the other four, showing moderate correlations. The levels of environmental dissimilarity among the sites did not seem to affect the performance of any of the indices tested, suggesting consistency − an important property for the indices to have. We conclude that although the acoustic indices have the potential to be used for passive acoustic monitoring, perhaps they need to be refined further before they can be applied widely. Meanwhile, they should be tested in more environments to reveal fully their potential and limitations.