Danqi Xing·Jun Yang·Jing Jin·Xiangyu Luo

Abstract Information on species composition of an urban forest is essential for its management.However,to obtain this information becomes increasingly difficult due to limited taxonomic expertise.In this study,we tested the possibility of using plant identification applications running on mobile platforms to fill this vacuum.Five plant identification apps were compared for their potential in identifying urban tree species in China.An online survey was conducted to determine the features of apps that contributed to users’satisfaction.The results show that identification accuracy varied significantly among the apps.The best performer achieved an accuracy of 74.6% at the species level,which is comparable to the accuracy by professionals in f eild surveys.Among the features of apps,accuracy of identification was the most important factor that contributed to users’ satisfaction.However,plant identification apps did not perform well when used on rare species or outside of the regions where they have been developed.Results indicate that plant identification apps have great potential in urban forest studies and management,but users need to be cautious when deciding which one to use.

Keywords Plant identification·Mobile apps ·Recognition accuracy·User satisfaction·Taxonomy

Introduction

In 2018,55% of the world’s population lived in urban areas,and the number is expected to increase to 68% by 2050(United Nations 2018).Given rapid urbanization,urban forests become increasingly important for their contributions to the liveability of cities (Nowak and Walton 2005).Urban forests generate environmental benefits such as purifying air,alleviating urban heat islands,and reducing stormwater runoff (Foster et al.2011;Hirokawa 2011).Species composition of urban forests influences the benefits that urban forests can generate (McPherson et al.1997;Nowak et al.2008).For example,urban forests with high biodiversity can provide more psychological and physiological benefits to city dwellers (Fuller and Gaston 2009;Carrus et al.2015;Wood et al.2018).Therefore,information on species composition of urban forests is indispensable for studies and practices in urban forest research and management.

In general,surveys are carried out to collect information on species composition of urban forests.However,it is becoming increasingly difficult to get information through this conventional way.On the one hand,cities that are experiencing financial difficulties frequently cut budgets for urban forest management as it is given a lower priority than other essential public services (Escobedo et al.2006).More and more cities are now depending on volunteers to collect information on urban forests,e.g.,New York City recruited 2241 volunteers to conduct its 2015 inventory of street trees(Crown et al.2018).Researchers are also increasingly relying on data generated by citizen scientists to study urban forests (Crall et al.2011;Hawthorne et al.2015;Romana et al.2017).On the other hand,the number of people who have good taxonomic knowledge is dwindling.The decline in funding and poor job prospects have reduced significantly the availability of education on taxonomy and the number of taxonomists (Guerra García et al.2008;Wägele et al.2011).The increased use of volunteers and citizen scientists in urban forest surveys and the decreased level of taxonomic expertise have created a concern over the taxonomic accuracy of collected data.

Researchers and administrators are looking for answers to this shortage of taxonomic expertise.Among the many options,automated plant species identification tools may offer a solution.Automated plant species identification includes two major categories:DNA barcoding (Newmaster et al.2009) and image-based recognition (Saitoh and Kaneko 2000;Du et al.2 007;Wang et al.2008).DNA barcoding uses gene sequences to identify species and can achieve a relatively high identification accuracy.However,the expensive appliances and the relatively slow running time are barriers for its extensive use (Valentini et al.2009).In contrast,automated image-based recognition,which uses descriptive features of plants extracted from images to identify species,has been regarded as a promising method (Joly et al.2014).The method benefits from two trends of technological development:the proliferation of mobile devices with digital cameras that makes capturing images an easy task,and advances in machine learning that facilitates object recognition and detection (Donahue et al.2014;Sharif Razavian et al.2014;Lef land et al.2018).

Image-based recognition of plant species has been developed rapidly over the past decade due to the aforementioned trends.As per the modes of application,the method can be divided into three major types:computerized off -line systems,web-based identif ciation systems,and systems running on mobile application platforms.The computerized off -line system was the first method developed as a choice of automatic plant identification.In an early application,Söderkvist(2001) identified fifteen tree species in Sweden using leaves and a computer vision system and reached an accuracy of 80%.Only five years later,a system based on recognition of intact leaf images attained an accuracy of over 92% of 25 species (Du et al.2006).While the off -line systems can achieve high identification accuracy,their accessibility impedes their full use.Web-based identification systems were the next to be developed.For example,the Computerized Plant Species Recognition System (CPSRS) supports plant recognition using either text or leaf images,and can achieve a favorable recall rate of 71.4% if considering the top five returned images (Ye et al.2004).The Fynbos Leaf Online Recognition Application (FLORA) and the Web Application for Plant Species Identification (WAPSI) are two more examples of web-based identification systems (Caballero and Aranda 2012;Winberg et al.2013).Although these systems off er better accessibility than the off -line system,they still cannot conduct quick on-site identification where access to the internet is often restricted.

Identification systems running as mobile applications on tablets or smartphones hold a great advantage of accessibility and portability over the other two types of systems.They allow people who have little or no training in taxonomy to quickly identify plant species right on the spot (Shrode 2012).Most mobile applications use flowers to identify species because flowers are considered the most identifiable feature of plants (Kim et al.2009).Mobile Flora,a mobile application for iPhone,recognizes 578 species of flowers(Angelova et al.2012).Leaf images are also frequently used in mobile applications.Leafsnap (Kumar et al.2012)and Pl@ntNet (Goëau et al.2013) are two apps based on leaf recognition for iPhone users.AI Botanist (Wang et al.2013) and Treelogy (Çuğu et al.2017) are apps running on the Android system.Recently,an intelligent robot called Botanicum has been developed,which allows users to identify 20 tree species using leaf images on mobile platforms through chatting (Korotaeva et al.2018).It is expected that plant identification systems based on mobile platforms will become more accurate and faster in the near future when new technological advancements in automatic plant species identification are deployed.These advancements will include implementing deep learning techniques with small datasets (Figueroa-Mata and Mata-Montero 2020) and the possibility to use more leaf traits (Keivani et al.2 020;Tan et al.2020).

The proliferation of plant identification systems running on mobile platforms provides a promising opportunity to address shortages of taxonomic expertise in urban forest studies and management.However,practitioners and researchers are faced with deciding which app to use.Because these are built on different sets of training data and algorithms,they are not equal in terms of accuracy and effi -ciency.So far,studies on plant identification apps mainly focused on interface design and identification algorithms(Wang et al.2013;Gao et al.2017).No study has been conducted to test their performance when used in urban forests.Users can only rely on the accuracy and efficiency claimed by the developers when choosing which one to use.These numbers are influenced by the different testing data and the testing procedures used by the developers,and so they are not comparable.This limitation will be addressed in this study that compares major plant recognition apps available in China for their recognition of urban tree species,user experiences and satisfaction levels.The results should help guide researchers and practitioners to select the most appropriate app for use in urban forest studies in China.In addition,the results may help developers to design better plant recognition apps.

Material and methods

Study area

We collected 1000 flower images for 100 tree species belonging to 75 genera,and 1000 leaf images for 100 tree species belonging to 84 genera growing in Chinese cities.Among them,15 species have both flower and leaf images.As a result,185 species belonging to 142 genera have been tested in this study (Appendix 1 Table S1).In order to represent the various growth conditions of the species,these images were collected from 15 cities spanning from south to north in China (Fig.1).In each city,the most commonly seen species were selected and images taken using cell phones or cameras.All images are freely available for download from the Google Drive:https://drive.googl e.com/drive/folde rs/1myfD jDedD G-zaD4h q2FCK hkKVN sYSz6 4?usp=shari ng.

Fig.1 Map of the 15 cities where plant images have been taken

Apps used in the study

Five plant identification apps were tested in the study(Table 1).Based on the functions of the apps,Huabanlv,Xingse,Weiruanshihua,GardenAnswers,and Pl@ntNet were compared for identifying flower images,Xingse,Huabanlv,and Pl@ntNet for identifying leaf images.

Table 1 Information on the five plant identification apps compared in this study

Comparing identification accuracy

The five apps were installed on a mobile phone (iPhone XS,iOS system) and used to identify the 185 species using either flower images or leaf images.The accuracy rate was calculated as the number of images correctly identified divided by the number of images of that species or genus.For each app,the average accuracy rates at the species and genus levels were calculated.In addition,the performance of the five apps were tested on tree species that are rare in cities.The frequencies of occurrence of the 185 species in the fifteen cities were calculated and we chose 20 species ranked at the bottom as the targets for comparison.After identification,multiple comparisons were carried out to examine whether the difference in accuracy rates was significant among the different apps.The data was examined for assumptions of normality and even variance before conducting multiple comparison.The normality of the identification accuracy results was examined using the Shapiro—Wilk test contained in baseRand the variance was tested using the Levene’s test contained inRpackagecars.Because the data did not pass these tests,we used the Games-Howell test inRpackageuserfriendlyscienceto conduct multiple comparisons.The Games-Howell test is a robust nonparametric approach to compare combinations of groups (Lee and Lee 2018).

Surveying users’ preferences

The above comparison results only give a quantitative description of the identification accuracy of the apps.The user experience is often an essential reason for choosing apps (Canfora et al.2016;Simmons and Hoon 2016;Son 2017).An online survey was conducted to evaluate the user experience of the five apps using a simple questionnaire with five sections.In the first section,demographic information (gender and age group) was asked of the survey participant.In the second section,the participants were asked to choose their most favorite plant identification apps from the five apps.If their favorite apps were not included,they were asked to give the name of the application.In the third section,participants were asked to rate their favorite apps on five features,recognition accuracy,acceptability which is defined as individuals’ perception of ease of use when operating mobile apps (Davis 1993),processing speed,navigational design,and visual design (Kapoor and Vij 2020).In the fourth section,the participants were asked to rate the overall satisfaction level of their favorite apps.A five-point Likert scale ranging from“strongly dislike”to“strongly like”was used in rating in the third and fourth sections.In the fifth section,the respondents were asked whether they have recommended his/her favorite plant identification apps to others or not.An electronic copy of the questionnaire can be found in Appendix 1 Table S2.

The questionnaire was first tested in a pilot study with a small group of colleagues and graduate students.After addressing issues found in the pilot study such as the clarity of instructions and the questions,the questionnaire was posted to an online survey platform Wenjuanxing (www.sojum p.com).The web link was distributed through a WeChat platform,Urban Biodiversity and Ecosystem Service of China (UBES_China),a public platform for sharing information on research in urban ecology and urban forestry.This platform was chosen because its subscribers are mostly researchers and practitioners in these fields and their experiences would be most relevant.The survey was conducted between December 22,2018 and February 13,2019.After receiving completed questionnaires,they were screened for completeness and validity of answers.

Using data compiled from screened questionnaires,the significance of difference in apps preference was first examined using the Mann—Whitney U test.Demographic factors and how features of apps affect users’ satisfaction were then investigated by running an optimal scale regression on the data.The overall satisfaction level was treated as the dependent variable and names of apps,sex and age of users,and ratings of the five features were used as independent variables.Optimal scale regression is an extension of the standard linear regression model which can handle categorical variables (van Der Kooij et al.2006).The regression model was fitted using SPSS (version 16.0,IBM SPSS Statistics).

Results

Accuracy of apps

At the species level,Xingse and Huabanlv had the best performance when used to identify flower images (Fig.2).When merged to the genus level,both identified the majority of all images.The differences among the accuracy rates of the five apps at the species level were statistically significant (F=55.23,df=4,P<0.001).Except for the Huabanlv and Xingse pair (t=1.73,df=197.86,P=0.416) and the pair of Pl@ntNet and Weiruanshihua (t=1.42,df=197.92,P=0.618),the differences between other pairs were all significant.The same pattern was observed at the genus level.

Fig.2 Accuracy of identification for the five plant identification apps:Xingse (XS),Huanbanlv (HL),Pl@ntNet (PN),Weiruanshihua(WS),and GardenAnswers (GA)

The results for leaf image recognition were similar to those for the flower images.At the species level,Xingse had the best performance in terms of the mean value for identification accuracy although the difference between it and Huabanlv was not statistically significant (t=1.28,df=195.88,P=0.407).At the genus level,both Xingse and Huabanlv identified the majority of all species correctly.There was no significant difference between the two apps (t=0.51,df=193.47,P=0.865).Pl@ntNet had significantly lower accuracy than the other two apps both at the species and genus level.

The results on the rare species showed that,for flower images,Huabanlv and Xingse had significantly better recognition accuracies at both genus and species levels(Fig.3).In the test on leaf images,both still performed than Pl@ntNet and the difference was statistically significant.The differences in the identification accuracies of flowers and leaves of rare species between Xingse and Huanbanlv were not statistically significant.

Fig.3 Accuracy of rare species identif ciation for the five apps:Xingse (XS),Huanbanlv (HL),Pl@ntNet (PN),Weiruanshihua (WS),and GardenAnswers (GA)

Users’ preferences of the apps

A total of 253 forms were received from the participants,and after an initial examination,forms from 232 respondents were considered complete and valid.Among these,101 chose Huabanlv as their favorite application and 124 chose Xingse.The remaining preferences were five for Weiruanshihua and two for Pl@ntNet.Therefore,our analysis was focused on the 225 participants who chose Huabanlv or Xingse because the sample sizes allowed us to obtain statistically meaningful results.

The demography of the participants indicated that a balanced participation from both sexes (Table 2).In addition,nearly half of the participants were in the 18—28 age class,which might be expected because the younger generation is often more technology savvy.

Table 2 Demographic profile of the survey participants

Users’ satisfaction levels

More than 75% of respondents were strongly satisf ied or satisf ied with Xingse and Huabanlv (Fig.4) .For the five features,the majority had positive perceptions.The differences between Xingse and Huabanlv were not statistically significant for all features perceived by respondents.

Fig.4 Percentages of ranking assigned by respondents to overall satisfaction and the five features for Huabanlv (HL) and Xingse (XS) apps.“5”indicates strongly like while“1”indicates strongly dislike

Users of Xingse have a higher rate of recommendation(47.4%) than for Huabanlv (41.8%).Pl@ntNet and Weiruanshihua had the same recommendation rate (0.9%).Around 9% of respondents would not recommend any of these apps.

The results from the optimal scale regression indicated that the five features explained a large portion of variance in users’ satisfaction rankings (AdjustedR2=0.703,F=67.295,df=8,P<0.001).The importance factor and the tolerance level are shown in Table 3.The importance factor represents the significant percentage of the factor inthis regression model,while higher values of significant variables (P<0.05) indicate higher influences.The tolerance test statistics indicate collinearity among factors with large negative value of the after-transformation tolerance indicating multicollinearity (Meulman and Heiser 1989).More information on the model output can be found in Appendix 2.

Table 3 Output of optimal scaling regression model for Xingse and Huabanlv

The regression results indicated that recognition accuracy,visual design,and processing speed were significant predictors of users’ satisfaction.Recognition accuracy was the most important factor.

Discussion

Overall,when flower images were used,Xingse had the highest mean accuracy rate of 74.6% at the species level for all 100 species;statistically there was no difference between Xingse and Huabanlv.At the genus level,Xingse had a similar average accuracy as Huabanlv,94.9% and 94%,respectively.These two apps also performed better when used to identify leaf images of the 100 species,with a mean accuracy rate of 49.1% and 56.2%,respectively.At the genus level,accuracy rates increased to 76.6% and 78.5%,respectively.These rates are comparable to those in field studies.A study reported that plant identif ciation error for professionals was 20% to species level and 6% to genus level (Gray and Azuma 2005).In another study,88% of plants were correctly identified to species level by professionals (Crall et al.2011).The fact that plant apps achieve an accuracy that is comparable to professionals makes them a valuable tool for urban forest studies and management in situations where taxonomic expertise is not available.

The results show that higher identification accuracy was obtained using flower images than leaf images.There are two possible reasons for this.First,f lowers contain more identifiable characteristics than leaves (Goëau et al.2012).Also,the foliage of many tree species can vary significantly at different developmental stages (Seiwa 1999;Cope et al.2012).For example,the leaf shape of young seedlings of blue gum (Eucalyptus globulusLabill.) is elliptic and changes to lanceolate when becoming adult trees.This feature adds uncertainty to identification based on leaf images.Secondly,in general,people take more photos of flowers than leaves because of their aesthetic value.One result of this intrinsic preference is that the developers of apps obtain more flower images than leaf images to update their algorithms for image recognition.According to Xingse,their daily usage jumped to 4.53 million in the flowering season on 5 April 2019,and 57.6% of photos were uploaded in the spring and summer when flowers are in bloom (Song 2019).

The identification accuracy was clearly affected by the origins of the apps.The three apps developed in China achieved better results than the two developed abroad.The developers select species to be included in the apps based on their target market.Therefore,when choosing apps,those developed by developers in the same region should be given more consideration.In addition,it is not surprising that lower mean accuracy rates were obtained on rare species.Even for the best performer—Xingse—the mean accuracy rate dropped from 66.2% for 155 species to 57.5% for the 20 rare species at the species level.This result reveals a main shortcoming of plant identif ciation apps.The developers use a large number of images to train and update their algorithms.For common species,their images are easy to obtain and makes identification more accurate.It is vice versa for the rare species.This may or may not become an issue for urban forest programs.If the purpose is to investigate biodiversity,it will be better to make up for this shortcoming of plant apps by adding taxonomists to the team.If the purpose is for daily management,this may not be a major problem because common species often account for the majority of trees of urban forests (Kendal et al.2014;Wang et al.2014).

The results show that recognition accuracy is the most important factor explaining overall satisfaction,followed by visual design acceptability and processing speed.Unlike other apps where factors such as ratings,reviews,and pricing are considered as important influencing factors (Chen and Liu 2011;Vasa et al.2012),recognition accuracy is the dominating factor for plant identif ciation apps.People who need this type of app are professionals and/or often botany enthusiasts and pay more attention to identification than to other features.The question of whether the participant has ever recommended his/her favorite plant recognition app to others can ref lect the users’ loyalty to the app (Yoon et al.2013;Chang 2015).Again,Xingse and Huabanlv stood out which is in accordance with their recognition accuracies.Therefore,mobile plant identification apps with high recognition accuracies would considerably raise the overall ratings of apps and generate publicity through recommendations from existing users.

While this study provides a comparison of plant identification apps in terms of their performance in identifying urban species in China,some limitations should be noted.First,this study should not be viewed as a test of the identification accuracy of the selected appsper se.It is a test of the applicability of these apps in urban forest studies in China.Diff erent countries will have different results.A test which subjects the five apps to an image database containing tree species available in all apps is a more standard way in computer science.However,the method developed in this study suited our purpose better.Second,the test images were collected by ourselves for this study.This was done to avoid the possibility that images obtained from the internet or other open sources may have been used by developers to train their algorithms which would unfairly favor them in identif ciation accuracy.Due to the labor involved in this task,only 2000 images for 185 species in 15 cities were collected.Because not all urban tree species could be covered,different rankings of the five apps may occur using species different from these in this study.However,the variance will not change the main conclusion of this study:apps developed locally should be preferred over apps developed in remote areas in urban forestry studies.

Conclusions

To obtain accurate information on species composition of urban forests is a challenging task due to limited taxonomic expertise.Image-based plant identification applications running on mobile platforms such as cell phones and tablets off er a promising solution.However,information on how accurately these apps can identify species in a city,a region,or a country and how to make a better plant identification app is limited.We compared the identification accuracy of five apps to tree species in China’s urban forests and examined features that contributed to their satisfactory usage.Using apps developed in China could achieve an accuracy rate comparable to the accuracy achieved in field surveys.This shows that plant identification apps have great potential in urban forest studies and management.Identification accuracy is the most important factor to users’ satisfaction.This suggests a future direction of work for the development of this type of apps.However,apps performed less satisfactorily outside regions where they have been developed.This should be a primary concern for people when choosing apps.In addition,apps did not perform well when used to identify rare tree species.This suggests that we cannot replace human expertise entirely with apps in urban forest studies and management.

AcknowledgementsWe would like to thank Pengbo Yan for her picture collection assistance and we appreciate all participants who contributed to the online survey in our research.