The rising use of AI is clear in every industry; changing spending habits of shoppers, powering self-driving cars and even being used to detect lung cancer earlier in patients. Some AI applications aren’t as ground-breaking as others, but there is no denying its current impact and even more so the potential impact it can make in the near-future.
That being said, in this article the use of AI in drones will be discussed — why we want and need it to operate on a drone, what issues do we face when trying to deploy AI on a drone and finally how can we address those identified issues, to allow reliable AI operation.
With €200m public/private funds being invested into the drone industry, it is apparent that there is a need from organisations to be able to use drones to address business needs and applications — with drones being used in multiple domains; public safety (>20℅), Agriculture (10–20℅), Energy/Oil/Gas (10–20℅) and Delivery/eCommerce (<10℅).
The above percentages are the predicted use of drones in a given domain based on their appearance in academia and media. Source: SESAR European Drone Outlook Study, 2016
As drones will be used in a wide-range of markets, there are a wide range of activities they can perform — some highly complex; requiring trained personnel to operate the drone, ensure final use-cases are met and to be able to perform analysis after the flight has been completed. These activities can take a lot of time which increases costs of deploying drones — making them less and less desirable to operate as time goes on.
The obvious answer to this is the use of AI, which can accomplish:
So, what’s the issue?
We have seen that drones are already extremely useful for use-cases which couldn’t before be realised without them and in fact AI can go even further — to improve the issues which arise when operating drones.
Most AI applications in drones will require some sort of object detection, image recognition or any other similar term — meaning the camera footage will need to be sent to an AI model for processing, so something of value can be derived from it. This type of AI application is computationally expensive.
Most drones used for operations will be small to medium sized— so the drone itself will not be too large. This could be due to hardware costs, legal challenges, or lack of skilled personnel to pilot.
This limits the size of computation hardware which the drone can be equipped with; so, some AI models aren’t viable as the hardware isn’t suitable to execute them — the AI would either be very inaccurate or wouldn’t execute at all.
Additionally, the more powerful hardware becomes the more power it will draw from onboard batteries — in turn reducing flight-times, which further degrades the likelihood of real use-cases being addressed by AI enabled drones.
Finally, there are currently restrictive laws in regard to Beyond Visual Line of Sight (BVLOS) flight operations — which is an attractive prospect for organisations operating drones. Although not much can be done to speed up this process, we can ensure that technically we are ready for BVLOS operations by ensuring safe reliable flight.
Unfortunately, there is no straight-forward answer, there are multiple options which can help to solve the problems mentioned above.
In regard to the computation issues, we actually have started to see some very capable hardware in recent years — such as the NVIDIA Jetson Nano which is a single board computer with a GPU on-board, small enough to be mounted on a drone. Its power draw is also manageable by a large enough battery mounted on the drone — which would power the drone and the Jetson Nano.
The Jetson Nano is capable of handling some complex AI models in real-time, so the drone can receive this information for flight guidance, or it could be sent back to a pilot — if the drone is in human control.
If you are still limited with hardware; you do not have something like the NVIDIA Jetson Nano or it isn’t suitable, you can use a Raspberry Pi. Of course, the Raspberry Pi can still execute some lightweight AI models — but arguably it isn’t as capable at the Jetson Nano.
If you want to use a Raspberry Pi, you can use ‘lighter’ AI models — such as Tensorflow Lite, MobileNet V2 or YOLO Tiny.
There is also an option called edge computing — which can be used to offload the computation responsibility to hardware which is more capable; meaning the drone wouldn’t need to worry about executing AI models at all! — you can read more about this here.
Finally, the issue with flight times can be improved by using a device like above — but of course they will still draw more power as they are processing increasingly complex AI models. We can in some cases run the computers from a separate power source; meaning the drone itself can rely on an exclusive power supply as well as the computer — but then you could argue that it would increase weight too much.
In short — designing a smart drone with AI operations is all about finding a good balance between some trade-offs — do we want to run complex AI models? can we provide the computation power for that? Can we settle for a light AI model on a less capable piece of hardware?
There are many questions and issues which need to be addressed when building, designing or operating a drone — and the right balance needs to be implemented based on the use-case you are trying to achieve.
Drone-as-a-Service (DaaS) is actively participating in trials in industry; to prove the operation of BVLOS drones over 4G/5G networks — showcasing safe reliable flight, with the drone additionally being equipped with complex AI models to achieve complex use-cases.
If you have any questions or would like to understand how Net Reply can help you with this or similar solutions, get in touch with Kye Grundy