Computing device and method for controlling unmanned aerial vehicle in flight space

1. Technical Field

The embodiments of the present disclosure relate to aircraft control systems and methods, and more particularly to a computing device and method for controlling an unmanned aerial vehicle (UAV) in a flight space.

2. Description of related art

An unmanned aerial vehicle (UAV), also known as a Unmanned aircraft System (UAS) or a remotely piloted aircraft (RPA) or unmanned aircraft, is a machine which functions either by a remote control of a navigator or pilot or autonomously, that is, as a self-directing entity. UAVs can fly autonomously or be piloted remotely without carrying a human operator, and are often preferred for missions that are too dull or dangerous for manned aircraft. Some UAVs are controlled to fly autonomously based on a pre-programmed flight plan using a dynamic automation system. However, the dynamic automation system may be more complex, and also cannot effectively control the UAV to fly within a flight space. Therefore, there is a need for a system and method for effectively controlling an UAV to fly within the flight space.

FIG. 1 is a block diagram of one embodiment of a computing device including an unmanned aerial vehicle (UAV) control system.

FIG. 2 is a flowchart of one embodiment of a method for controlling the UAV using the computing device of FIG. 1 .

FIG. 3 is a schematic diagram illustrating one example of creating a 3D sample database for storing various human gesture data.

FIG. 4 is a schematic diagram illustrating one example of capturing a 3D scene image of a 3D scene in front of the user using a depth-sensing camera.

FIG. 5 is a schematic diagram illustrating one example of controlling the UAV to fly in a flight space according to gestures of the user.

The present disclosure, including the accompanying drawings, is illustrated by way of examples and not by way of limitation. It should be noted that references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean “at least one.”

In general, the word “module,” as used herein, refers to logic embodied in hardware or firmware, or to a collection of software instructions, written in a programming language. In one embodiment, the program language may be Java, C, or assembly. One or more software instructions in the modules may be embedded in firmware, such as in an EPROM. The modules described herein may be implemented as either software and/or hardware modules and may be stored in any type of non-transitory computer-readable medium or other storage device. Some non-limiting examples of non-transitory computer-readable media include CDs, DVDs, flash memory, and hard disk drives.

FIG. 1 is a block diagram of one embodiment of a computing device 1 including an unmanned aerial vehicle (UAV) control system 10 . In the embodiment, the computing device 1 may further include a depth-sensing camera 11 , a storage device 12 , and at least one processor 13 . The computing device 1 may be a host computer, a workstation computer, or a server computer. The computing device 1 connects to a UAV 2 through a wireless network 3 , such as an WIFI network, a ZIGBEE network, or other wireless local area network (LAN). In one embodiment, the UAV control system 10 may include computerized instructions in the form of one or more programs that are stored in the storage device 12 and executed by the at least one processor 13 . It should be understood that FIG. 1 illustrates only one example of the computing device 1 that may include more or fewer components than illustrated, or have a different configuration of the various components in other embodiments.

The UAV control system 10 generates one or more flight control commands according to different gestures of a user who controls the UAV 2 to fly in a flight space, and controls or adjusts a flying mode of the UAV 2 in the flight space according to the flight control commands. Each gesture of the user corresponds to a flight control command, which controls or adjusts the flying state of the UAV 2 in the flight space. In one embodiment, the user may move his/her hands upwards, downwards, leftwards, or rightwards, and/or rotate his/her hands clockwise or counterclockwise. The flight control commands may include a first control command for controlling the UAV 2 to climb, to descend, to bank left and right, and a second adjust command for adjusting a flying mode of the of the UAV 2 , such as a flying speed adjustment mode or a flight direction mode of the UAV 2 .

In one embodiment, the depth-sensing camera 11 is a time of flight (TOF) camera device having a 3D image capturing functionality, and can sense a depth distance between a camera lens of the depth-sensing camera 11 and the user. The depth-sensing camera 11 captures one or more 3D images from a scene in front of the user when the user controls the UAV 2 to fly in the flight space. In the embodiment, the user is illuminated by a short light, and the depth-sensing camera 11 measures a time taken from when the light hits the user until the reflected light reaches the depth-sensing camera 11 (called “the reflected time”). This reflected time is directly proportional to a depth distance between the camera lens and the user. Therefore, the depth-sensing camera 11 senses the depth distance between the camera lens and the user according to a pulse time of the short light illuminating at the user and the reflected time of the reflected light from the user.

In one embodiment, the storage device 12 may be an internal storage system, such as a random access memory (RAM) for the temporary storage of information, and/or a read only memory (ROM) for the permanent storage of information. In some embodiments, the storage device 12 may be an external storage system, such as an external hard disk, a storage card, or a data storage medium.

In one embodiment, the UAV control system 10 may include a 3D sample creating module 101 , an image detecting module 102 , and a flight control module 103 . The modules 101 - 103 may comprise computerized codes in the form of one or more programs that are stored in the storage device 12 and executed by the processor 13 to provide functions for implementing the modules. Detailed descriptions of each module will be given with reference to FIG. 2 as described in the following paragraphs.

FIG. 2 is a flowchart of one embodiment of a method for controlling the UAV 2 in a flight space using the computing device 1 of FIG. 1 . In the embodiment, the method can generate one or more flight control commands according to gestures of the user who controls the UAV 2 to fly in a flight space, and controls or adjusts a flying mode of the UAV 2 in the flight space according to each of the flight control commands Depending on the embodiment, additional blocks may be added, others removed, and the ordering of the blocks may be changed.

In block S 21 , the 3D sample creating module 101 creates a 3D sample database that stores human gesture data, and stores the 3D sample database in the storage device 12 . The human gesture data includes data concerning characteristic gestures of humans to control the UAV 2 to fly in the flight space, characteristic data of actions of human for adjusting flying modes of the UAV 2 in the flight space. In one example with respect to FIG. 3 , the human gesture data includes data indicative of a person moving in his/her hands upwards, downwards, leftwards, or rightwards, and/or rotates his/her hands in a clockwise or in a counterclockwise direction. The human gesture data can include movement of one or more hands of the person or other parts such as a leg.

FIG. 3 is a schematic diagram illustrating one example of creating the 3D sample database. In one embodiment, the human gesture data may be captured using the depth-sensing camera 11 , a data relationship is built between each of the human gesture data and each of the flight control commands of the UAV 2 , and the human gesture data are stored in the 3D sample database for determine the gestures and actions of the user in real time. The flying modes may include a climbing mode, a descending mode, a leftward or rightward banking mode, a flying speed adjustment mode, and a flight direction mode of the UAV 2 .

In block S 22 , the image detecting module 102 captures a 3D image of a scene (hereinafter “3D scene image”) in front of the user using the depth-sensing camera 11 when the user controls the UAV 2 to fly the flight space (hereinafter “3D scene image”), and obtains a depth distance between the user and the depth-sensing camera 11 using the depth-sensing camera 11 . In one embodiment, the depth-sensing camera 11 senses the depth distance between the depth-sensing camera 11 and the user, and captures one or more 3D scene images from the scene in front of the user. FIG. 4 is a schematic diagram illustrating one example of capturing a 3D scene image of a scene in front of the user using the depth-sensing camera 11 . In the example, the depth-sensing camera 11 captures a 3D scene image from the scene in front of the user, and senses the depth distance between the depth-sensing camera 11 and the user.

In block S 23 , the image detecting module 102 detects a person image of the user (hereinafter “3D person image”) from the 3D scene image of the scene using a 3D person detection method. In the embodiment, the 3D person detection method comprises steps of: (a) the image detecting module 102 converts a depth distance between each point of the user and the depth-sensing camera 11 into a pixel value, and stores all of the pixel value in a data array; and (b) the image detecting module 102 compares the pixel values of the data array with characteristic data of the user stored in the 3D sample database to obtain the 3D person image from the 3D scene image.

In block S 24 , the image detecting module 102 compares the 3D person image with the human gesture data of the 3D sample database to obtain gesture information of the user from the 3D person image. The gesture information of the user may include gesture and action data relating to the user when the user controls the UAV 2 to fly in the flight space.

In block S 25 , the flight control module 103 converts the gesture information of the user to a flight control command for controlling the UAV 2 in the flight space, and sends the flight control command to the UAV 2 through the wireless network 3 . In one embodiment, the flight control commands may include a first control command for controlling the UAV 2 to fly upwards, downwards, leftwards, and rightwards, and a second adjustment command for adjusting the flying mode of the of the UAV 2 .

In block S 26 , the flight control module 103 drives a driver (e.g. an engine) of the UAV 2 to control the UAV 2 to fly in the flight space or adjust the flying mode of the UAV 2 according to the flight control command In one embodiment, the flight control module 103 can control the UAV 2 to fly upwards, downwards, leftwards, and rightwards, and adjust the flying mode of the UAV 2 including the speed of flight or direction of the UAV 2 .

FIG. 5 is a schematic diagram illustrating one example of controlling the UAV 2 to fly in a flight space according to gestures of the user. In the example, the UAV 2 flies leftwards in the flight space when the user moves his right hand straight out, and flies right when the user moves his left hand straight out. The UAV 2 flies down when the user extends his arms straight down with the palms of the hands inwards, and flies up when the user holds his forearms straight up with the palms inward.

All of the processes described above may be embodied in, and fully automated via, functional code modules executed by one or more general purpose processors of the computing devices. The code modules may be stored in any type of non-transitory readable medium or other storage device. Some or all of the methods may alternatively be embodied in specialized hardware. Depending on the embodiment, the non-transitory readable medium may be a hard disk drive, a compact disc, a digital video disc, a tape drive or other suitable storage medium.

Although certain disclosed embodiments of the present disclosure have been specifically described, the present disclosure is not to be construed as being limited thereto. Various changes or modifications may be made to the present disclosure without departing from the scope and spirit of the present disclosure.