RDNL Dutch Data Prize: A tool for automated plant identification

Orchids are one of the largest and most diverse fam­i­lies of flow­er­ing plants in the world that com­pris­es more than 25,000 dif­fer­ent species. How can you tell the species apart? Last year’s RDNL Dutch Data Prize win­ner, Diah Apriyan­ti, is devel­op­ing a tool to help!

Diah Apriyan­ti is a PhD stu­dent in the Depart­ment of Data man­age­ment & Bio­met­rics at the Uni­ver­si­ty of Twente. Dur­ing her doc­tor­al research, Diah is devel­op­ing an algo­rithm that can be used to iden­ti­fy dif­fer­ent species of plants for their tax­o­nom­ic clas­si­fi­ca­tion. Curi­ous to learn more, we asked Diah to tell us more about her research and the sto­ry behind her win­ning dataset. 

What inspired your research?

My inter­est in plant iden­ti­fi­ca­tion began dur­ing my time work­ing in the Pur­woda­di botan­i­cal gar­den in my home coun­try of Indone­sia. I recog­nised that botanists and tax­on­o­mists devot­ed a large amount of time to the man­u­al iden­ti­fi­ca­tion of plants for their tax­o­nom­ic clas­si­fi­ca­tion using ref­er­ence col­lec­tions and data­bas­es. 

The man­u­al process is tedious and labo­ri­ous, and the exist­ing image-based iden­ti­fi­ca­tion sys­tems are a ‘black box’. They pro­vide the name of the plant species but no expla­na­tion of the iden­ti­fi­ca­tion process or the plant char­ac­ter­is­tics. 

Deter­mined to improve the process, I have been devel­op­ing a com­pu­ta­tion­al tool to auto­mate plant iden­ti­fi­ca­tion as part of my Master’s degree and PhD research using orchids as a mod­el species. My tool promis­es to makes plant iden­ti­fi­ca­tion quick­er, eas­i­er and more reli­able by pro­vid­ing the species name and a detailed descrip­tion of the flower char­ac­ter­is­tics. What’s more, it can recog­nise the species name from images tak­en from dif­fer­ent angles and under var­i­ous light­ing con­di­tions. 

How have you developed your tool so far, and how does it work? 

The first step was to cre­ate the Orchid Flow­ers Dataset which con­tains more than 7,000 images of orchid flow­ers that belong to almost 160 species. I obtained these images from a vari­ety of sources, includ­ing Go Botany (Native Plant Trust), the Ency­clo­pe­dia of Life (EOL) and Flickr. I devel­oped soft­ware code using Python pro­gram­ming lan­guage to query the Flickr appli­ca­tion pro­gram­ming inter­face (API) using appro­pri­ate key­words about orchid species and flower char­ac­ter­is­tics. All of the images were avail­able under the Cre­ative Com­mons CC-BY license and, there­fore, could be redis­trib­uted and reused.

The har­vest­ed images were all very dif­fer­ent! Some were close-up images of orchid flow­ers whilst oth­ers were of the whole plant, includ­ing the stem and leaves. As my plant iden­ti­fi­ca­tion tool required an image of the orchid  flower, I had to clean the data as much as pos­si­ble to remove irrel­e­vant images and obtain images of flow­ers.

Next, I cre­at­ed a Bayesian deep learn­ing algo­rithm to iden­ti­fy, describe, clas­si­fy and name the orchid in each image. The algo­rithm uses ‘fea­ture clas­si­fiers’ which are vari­ables used to describe the flower char­ac­ter­is­tics. The clas­si­fiers I used were com­pa­ra­ble to those used by botanists and tax­on­o­mists, such as ‘species name’, ‘colour of flower’, ‘tex­ture’, ‘num­ber of flow­ers’, ‘inflo­res­cence’ (arrange­ment of flow­ers) and ‘colour of label­lum’ (lip that attracts insects). These clas­si­fiers were entered into a Bayesian net­work to iden­ti­fy the species name.

Image No. 7034 from the Orchid Flower Dataset
Source: Arthur Haines (Nature Plant Trust)

Species name: Amerorchis rotun­di­fo­lia
Colour of flower: Pur­ple
Colour of label­lum: Pur­ple
Label­lum char­ac­ter­is­tic: lobed, spot­ted
Inflo­res­cence (arrange­ment of flow­ers): raceme, a few flow­ers per clus­ter.


What were your motivations for publishing your dataset?

Plant iden­ti­fi­ca­tion datasets are scarce. Avail­able datasets are lim­it­ed to com­mon plant species rather than wild trop­i­cal species, such as orchids. Many orchid species are endan­gered, grow­ing only in con­ser­va­tion areas and botan­ic gar­dens, so I want­ed to make my Orchid Flow­ers Dataset avail­able to those who strive to pro­tect such vul­ner­a­ble species. 

I pub­lished my image dataset in the Har­vard Data­verse repos­i­to­ry because oth­er sci­en­tists in my research field have used this repos­i­to­ry and I want­ed to fol­low their good prac­tice. I also pub­lished a README file which is a sim­ple text file con­tain­ing the ‘fea­ture clas­si­fiers’ that were used to iden­ti­fy orchid species in the images. Pro­vid­ing the images alone would not be suf­fi­cient. The README file allows oth­ers to under­stand and reuse the images. In addi­tion, my soft­ware code used to query the Flickr API is open­ly avail­able on GitHub. 

Has your data been reused? 

I’m proud to say that my dataset has been down­loaded more than 140 times since its pub­li­ca­tion online! It has been used by stu­dents in the Bachelor’s pro­gramme in Elec­tri­cal Engi­neer­ing and the Master’s pro­gramme in Com­put­er Vision and Bio­met­rics to learn about the use of deep learn­ing mod­els.

Cur­rent­ly, I’m writ­ing a pub­li­ca­tion about this research and I hope that by cit­ing my dataset oth­ers will be able to reuse it for their own ben­e­fit. After all, my dataset is nov­el and pro­vides a more detailed descrip­tion of orchid flower char­ac­ter­is­tics than cur­rent­ly exists. It is not lim­it­ed to orchids but serves as a ‘proof-of-con­cept’ tool that can be adapt­ed to iden­ti­fy oth­er plant species!

Congratulations, Diah!
How will you spend your prize winnings?

I’d like to use my RDNL Dutch Data Prize mon­ey of €1.000 to con­tin­ue devel­op­ing my tool for auto­mat­ed plant iden­ti­fi­ca­tion, and to devel­op a mobile appli­ca­tion so that end-users can upload an image of a plant to iden­ti­fy it’s species and char­ac­ter­is­tics. This would be of huge ben­e­fit to the work of botanists and tax­on­o­mists, and could help save crit­i­cal­ly endan­gered plant species around the world. 

Diah won the RDNL Incen­tive Prize for the ‘Exact and tech­ni­cal sci­ences’ in Novem­ber 2020.
Karel Luy­ben, Nation­al Coor­di­na­tor for Open Sci­ence (left), Diah Apriyan­ti (right).
Watch the record­ing of the RDNL Dutch Data Prize event!

Thank you for shar­ing your sto­ry with us, Diah, and good luck com­plet­ing your PhD. We wish you all the best with these beau­ti­ful flow­ers!.. And, when we stop to admire an orchid in the future per­haps we can use your app to check what species it is!

Co-authors: Mar­it van Eck (Uni­ver­si­ty of Twente) and Con­nie Clare (4TU.ResearchData).
Cov­er image by Myr­i­ams-Fotos from Pix­abay.