Forest secondary succession and conversion to an agricultural use are rapidly altering tropical forests, and this alteration has consequences on soil microbial communities. Using high-throughput sequencing of 16S rRNA genes and internal transcribed spacer regions, we compared bacteria and fungi community composition and diversity in soils between the three stages of forest succession (primary forest, PF; secondary forest, SF; pioneer plant community dominated by Macaranga denticulate trees, PP) and two agricultural drylands (rubber plantation, RP; Plukenetia volubilis plantation, PV) in Xishuangbanna, a tropical region of China, and identified the factors associated with the shift in community composition across the five land-use types. The results indicated that the most abundant bacterial genera (Bacillus, Kitasatospora, Nitrospira and Streptacidiphilus) were found at differentially relative abundances among the five land-use types. Through the analysis of indicator species, several bacterial species were significantly associated with the PP site; one to four species were significantly associated to the other sites. Strikingly, almost all fungal genera had site-specific characteristic. Soil properties, especially pH and available P and K, were associated with microbial community composition. Across the three stages of forest succession, bacterial and fungal richness and bacterial alpha diversity had the lowest levels in the earliest stage (PP), but there were no significant differences in microbial richness or diversity between the late stages (PF vs. SF). Compared to PF and SF with their similar bacterial and fungal diversity, the agricultural drylands (RP and PV) had higher bacterial richness but lower fungal richness, indicating that both PF and SF can act as reservoirs for the recolonisation of forest-associated microbes. Overall, these results showed a distinct difference in soil microbe taxonomic composition, especially in fungi, among the various land-use types, even at a very small geographical scale (< 4 km), but no great difference of microbial diversity was found between the late stages of forest succession and agricultural drylands. Tropical forest succession and forest conversion to agricultural drylands strongly affect the distribution of microbial species, whereas microbial diversity may not always tightly follow the same successional trajectories.