You’ve heard plenty of people say, “it’s not the drone, it’s the data,” and we couldn’t agree more. For years now, RIIS has been talking about what options are open to software developers who want to investigate this new market, and the data drones are capturing is what’s driving those options. As a mobile and web development company that’s been in business for 20 years now, we also understand that data isn’t enough.

RIIS stands for Research Into Internet Systems, so as you can imagine, we're always looking for new opportunities to expand the services we can offer in order to see that data turned into something that makes sense for commercial enterprises of all types and sizes. Drones and software development for drones became an obvious opportunity to look at when we saw how Litchi created their own market a while back by creating a niche version of the DJI Go app. There are just so many opportunities that exist with this technology for the creative developer. We’re committed to exploring these opportunities in a safe and secure manner.

Earlier this year, we were working on a couple drone apps for customers and were very encouraged by how easy it was to use DJI's Mobile SDK.  While the SDK isn't perfect yet, there is plenty of functionality available for the iOS or Android developers to create drone apps without having to write a lot of code.

After we understood how to make the DJI drone fly missions, take pictures and plenty more from the app, we started to think if there were ways we could pair other technology with the drone to use Artificial Intelligence or AI to increase the functionality of the drone.

Out of this was born the Sheep Counter app. This app uses TensorFlow's object detection from Google to detect and count sheep in a field. You may have seen Google's object detection in operation if you've downloaded Google Lens on your Android phone. It can detect all sorts of things from people to pets, kites to cups, and much more. Developers can change the inputs to the object detection model to make it recognize other things such as sheep. Finally, we created a backend server and installed OpenCV to allow the drone pilot to upload images from an automated flight and stitch all the images together for a complete count of the sheep.

The Sheep Counter mobile app works as follows:

  1. Pilot creates a map of their field on their phone

  2. Drone flies automated around the field

  3. App uses Tensorflow Object Detection to identify and count sheep in real time

  4. Images are sent to a backend server where they are stitched together

  5. Total sheep count sent back to be displayed on the phone.

You may be asking the question why Counting Sheep and not something else? We were looking for a practical example of something that would work well with object detection and animal husbandry seemed like a good place to start.  Ultimately though, the reason we created the Sheep Counter was to understand how to put AI and drones together in a single app.

TensorFlow and Android play very well together, so making that work with the DJI SDK was relatively straightforward.  I think object detection - the type of AI we used on the app - and drones work very well together.  We're looking for more practical examples now that will help people save time or provide a completely new business model using these technology mashups. We've had inquiries from people who want to count deer, mountain goats.

Those inquiries are what really drive the opportunities with this technology because it’s not about the drone or the data: it’s about how this technology can drive decisions and create efficiencies. After all, we’re mobile developers, we’re not drone hardware developers. We’re simply using the DJI mobile SDK to allow us access to the flying hardware and processing the video feed using TensorFlow to create something new that can help drive decisions and create those efficiencies.

It’s a unique opportunity for users and for us because developers no longer have to build an oven when they're creating an AI application. Today, it's more like baking a cake. Sure, you need to spend a lot of time configuring different systems to get them to work together but someone else has already done the heavy lifting.

Those configurations meant using Google search to get us 5,000 images of sheep from above. After that, we drew boxes around the sheep, which is known as “labeling”.  This labeling helps the object detection model figure out what is and isn't a sheep. We take 70% of these labeled images to train the model and then 30% of the images to test that our model works and does, in fact, recognize sheep.

This isn't a one and done application though. It's much more of an iterative process and we had to redo the labeling and training multiple times. As an example of what it means to go through that process, it turns out it's much more accurate to use images of sheep taken from a drone video than it is to use Google search. The end result makes it all worthwhile though, and in this case, it meant testing and validating the app by simply flying the drone over as many sheep farms as we could.

We're already working on a Cattle Counter which should be completed before the end of the year and we've also been asked to do a Deer version and another version that will count plants.  Future plans are to use Augmented Reality or AR as well as AI on drones. AR will allow us to place an overlay on the drone video to provide the pilot with extra contextual information that wouldn't otherwise be available on the standard DJI apps.

We're looking for sheep farmers to use the app and give us feedback. My guess is that's a much bigger market here in the United States for counting cattle, but again, that’s why we’re looking for an asking the community for feedback about the Sheep Counter app and what should be next for us. We’re not experts in agriculture, we’re mobile developers committed to helping enable inventive ways to use drones and extend a variety of commercial markets. Agriculture is just one of those markets.

The Sheep Counter app is currently available on Google Play if you'd like to try it out.  https://play.google.com/store/apps/details?id=com.riis.sheepcounter