1. Create a Bird Recognition AI Application

Create a bird recognition application through the STARSG platform

Project Significance

Firstly, in terms of ecological protection, bird recognition can quickly identify the number and species of birds in images, greatly increasing the efficiency of researchers. By identifying and monitoring birds, we can promptly detect changes in the ecosystem and take appropriate protective measures to maintain biodiversity.

Additionally, during bird-watching activities, bird recognition can provide viewers with introductions to the current birds, allowing bird watchers to gain a more detailed understanding.

Project Practice

First, create a new Agent and enter the Edit Workflow page.

We first attempt to write the simplest bird detection, changing Start to General mode.

Change param Type to Media, add a Media node in the General node, add a Bird Detection node in the Extension, and connect the Start, Media, birdDetection, and End nodes. Write the names and outputs of the Media and Detection elements, then bind the end node to the output value of Bird Detection. The final result is as shown below:

This completes the construction of the simplest bird detection, then we click Trial run to test it. The input image here is:

You can see the result outputs whether there are birds, the type of birds, and the size and coordinates of the birds in the image.

Extracting Recognized Species

The dictionary result output by bird Detection is rather chaotic, so we extract the species information separately to make the program's results clearer.

In the test results, you can see that the output already includes the species of birds in the image. What we need to do now is extract the value corresponding to this species, disconnect the bird Detection and end nodes, and then add a Program node to connect the if condition of the Condition node. We need to process the output from bird Detection through a custom method in the Program node to extract the Species value. In the output, you can see the output value is an array of birds and a Boolean value hasbird. We need to extract the value from the species object in the birds array. By inputting the output of the bird Detection node as Detection in the Program node, the following code can achieve this:

def main(args):
    detection = args["Detection"]
    species_list = [bird['species'] for bird in detection['birds']]
    return {"result": species_list}

Using LLM for Species Introduction

After extracting the species of birds, to allow users of bird recognition to quickly understand the birds in the image, we can use LLM for species introduction.

Then set the input value of the LLM node to the species array extracted from the Program node. Reference the prompt words and let the large language model introduce the species, then connect to the End node. The test results are shown in the figure (the test images are still the images shown above):

Detail Optimization

All the above tests are conducted with bird images as input. When testing with other images (i.e., images without any birds), the program will fail, so we need to add a judgment before the Program node to determine whether it is a bird image. When inputting non-bird images, observe the output value of the bird Detection node, and you can see that it only outputs {"hasbird":false} at this time. We add a new Program node after the bird Detection node, still setting the input as the output of the bird Detection node, and output in the program segment:

def main(args):
    detection = args["Detection"]
    detect=detection["hasBird"]
    return {"detect":detect}

When inputting bird images, hasBird will output True, so after the judgment program, add a new Condition node with the judgment condition as shown below:

If true, connect to the Program program that extracts species; if false, go directly to end.

Final Effect

The final connection diagram is as shown below:

The final running result is as shown below: