Information processing apparatus, remote indication system, and computer readable medium

This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2008-261106 filed Oct. 7, 2008.

1. Technical Field

This invention relates to an information processing apparatus, a remote indication system and a computer readable medium.

2. Related Art

There has been conventionally known a remote indication system including a server (e.g. a computer) with which a video camera and a projector are connected, and a client (e.g. a computer) of a remote place connected with the server via a network.

According to an aspect of the present invention, there is provided an information processing apparatus including: a reception unit that receives multiple captured images that are captured with a capture device in parallel from an external apparatus to which the capture device that captures an object is connected; a detection unit that detects a priority image that is included in the multiple captured images being received by the reception unit and is assigned priority information that specifies the priority image to be preferentially displayed on a display device; and a controller that controls the reception unit to stop receiving other captured images which are not detected by the detection unit and to receive the priority image detected by the detection unit and controls the display device to display the priority image on the display device when the priority image is detected by the detection unit.

Exemplary embodiments of the present invention will be described in detail based on the following figures, wherein:

FIG. 1 is a block diagram showing the structure of a remote indication system in accordance with an exemplary embodiment of the present invention;

FIG. 2 is a block diagram showing the hardware structure of PC 6 ;

FIG. 3 illustrates the data structure of image data received by PC 6 ;

FIG. 4A illustrates an existing reception condition of image data, and FIG. 4B illustrates a reception condition of image data of this embodiments;

FIG. 5 is a flowchart showing a process executed with PC 6 ;

FIG. 6 is a flowchart showing an image reception process of the step S 3 in FIG. 5 ;

FIG. 7 is a flowchart showing a request process of a second image data;

FIG. 8 is a flowchart showing an interruption process executed during a display process of the step 4 in FIG. 5 ;

FIG. 9 illustrates an example of a message 91 displayed in first window 13 ;

FIG. 10 is a flowchart showing a process executed with a video server 2 ;

FIG. 11 is a flowchart showing a process executed with a distribution server 3 ; and

FIG. 12 is a flowchart showing a process executed with a drawing server 4 .

A description will now be given, with reference to the accompanying drawings, of exemplary embodiments of the present invention.

FIG. 1 is a block diagram showing the structure of a remote indication system in accordance with an exemplary embodiment of the present invention.

The remote indication system in FIG. 1 is provided with a server system 1 corresponding to a first information processing apparatus, and a PC 6 functioning as a second information processing apparatus. These elements are connected to each other via a network 5 functioning as means of communication. The server system 1 is provided with a video server 2 , a distribution server 3 connected to a first camera 7 and a second camera 9 , and a drawing server 4 that is connected to a projector 8 . Based on a control command from the drawing server 4 , the projector 8 emits light beams or projects an annotation image onto an object 11 (e.g. a screen) via a half mirror 10 . It should be noted that the annotation image includes an image of any types such as a line, a character, a symbol, a figure, a color, and a font.

The first camera 7 is composed of a video camera, captures a reflected image of the object 11 via the half mirror 10 , and outputs the captured image to the video server 2 . More specifically, the first camera 7 captures the overview image of the object 11 . It is better that an angle of view and an optical axis of the first camera 7 are identical with those of the projector 8 , because the half mirror 10 can work with the first camera 7 and the projector 8 .

The second camera 9 is composed of a network camera or a camera with a pan tilt zoom function, and is moved freely by an operator. The second camera 2 outputs the captured image to the video server 2 . And the second camera 9 is used for enlarging and capturing a part of the object 11 . More specifically, the second camera 9 captures a part of the object 11 .

The video server 2 stores the captured images by the first camera 7 and the second camera 9 . The distribution server 3 distributes the captured images by the first camera 7 and/or the second camera 9 to the PC 6 based on a distribution request of the captured image from the PC 6 . The drawing server 4 acquires the annotation image from the PC 6 , and outputs it to the projector 8 . It should be noted that the server system 1 may be composed of one server with functions of the video server 2 , the distribution server 3 , and the drawing server 4 although the server system 1 is composed of several servers in this embodiment.

A display device 12 is connected with the PC 2 . The display device 12 displays a first window 13 displaying the captured image by the first camera 7 , a user interface (UI) 14 to draw the annotation image, and a second window 15 displaying the captured image by the second camera 9 . The UI 14 includes a group of buttons such as a pen button, a text button, and an erase button, and icons defined by lines and colors. For example, when the pen button of the UI 14 is pushed down and a figure is drawn on the object 11 in the first window 13 , the figure data is output to the projector 8 from the PC 6 via the drawing server 4 . The projector 8 draws the figure on the object 11 based on the figure data.

In FIG. 1 , the second window 15 displays a display area 17 displaying an object 16 which is a part of the object 11 , an operation button 18 , a zoom in button 19 and a zoom out button 20 . It is unnecessary that the second window 15 is displayed all the time. The second window 15 is displayed on the display device 12 when a predetermined command is input to the PC 6 . When the operation button 18 is pushed down, the second camera 9 changes its direction based on the direction that the button pushed down indicates. At this time, the image displayed in the display area 17 is changed depending on the movement of the second camera 9 . When the zoom in button 19 or the zoom out button 20 is pushed down, the image displayed in the display area 17 is enlarged or reduced.

And the PC 6 outputs a control command to the projector 8 via the drawing server 4 , and controls a brightness of the projected image by the projector 8 . Also the PC 6 outputs the control command to the first camera 7 and the second camera 9 via the video server 2 and the distribution server 3 , and controls the capturing angle and the brightness of the captured image by the first camera 7 and the second camera 9 .

It should be noted that the remote indication system can be provided with several clients (PCs) although only the PC 6 is illustrated as a client in FIG. 1 .

FIG. 2 is a block diagram showing the hardware structure of the PC 6 .

Since the hardware structure of the PC 6 is the same as the each hardware structure of the video server 2 , the distribution server 3 , and the drawing server 4 , a description will now be given of the hardware structure of the PC 6 herein after. It should be noted that, in FIG. 2 , the reference numerals 21 to 29 designate the elements of the video server 2 and the reference numerals 31 to 39 designate the elements of the distribution server. And the reference numerals 41 to 49 designate the elements of the drawing server 4 and the reference numerals 61 to 69 designate the elements of the PC 6 .

The PC 6 includes: a CPU 61 (a reception unit, a detection unit, a controller, a second reception unit) that controls the entire PC 6 ; a ROM 62 that stores control programs; a RAM 63 that functions a working area; a hard disk drive (HDD) 64 that stores various information and programs; a mouse and keyboard 65 ; a network interface 66 that is connected to other computers; a video interface 67 that is connected to the display device; and a USB (Universal Serial Bus) interface 68 that is connected to a USB device, not shown. The CPU 61 is connected to the ROM 62 , the RAM 63 , the HDD 64 , the mouse and keyboard 65 , the network interface 66 , the video interface 67 and the USB interface 68 via a system bus 69 . It should be noted that a CPU of a server combining the video server 2 , the distribution server 3 and the drawing server 4 corresponds to a first reception unit and an addition unit.

FIG. 3 illustrates the data structure of the image data received by the PC 6 .

As illustrated in FIG. 3 , the image data received by the PC 6 are composed of a data header 51 with header information, and a data body 52 . The data header 51 contains attribute information such as a capture time, a type of the image data and a size as needed, and may contains a priority tag (priority information) as described hereinafter. The data body 52 is real data of the image.

FIG. 4A illustrates the reception condition that a known apparatus receives the image data, and FIG. 4B illustrates the reception condition that the remote indication system in accordance to this embodiment receives the image data.

As illustrated in FIG. 4A , when the image data are distributed in order of a first image, a second image and a third image from the distribution server 3 , the PC 6 receives the image data based on that order. In accordance with this embodiment as illustrated in FIG. 4B , the PC 6 requests to receive the data header of new image data whether or not the PC 6 is receiving the image data. And when the data header contains the priority tag, the PC 6 cancels receiving the image data the PC 6 is receiving, receives the image data of the image (priority image) that has the data header containing the priority tag (priority information), and displays the priority image on the display device 12 . When there is not the image data the PC 6 is receiving, the PC 6 receives the image data of the priority image, and displays it on the display device 12 . In the example illustrated in FIG. 4B , the PC 6 receives the second image with the priority tag, and displays the second image on the display device 12 .

FIG. 5 is a flowchart showing a process executed with the PC 6 .

First, the CPU 61 of the PC 6 establishes a connection with the distribution server 3 and the drawing server 4 (step S 1 ).

Then, the CPU 61 receives a command to distribute the image from the distribution server 3 (step S 2 ), and starts an image reception process (step S 3 ). A detail of the image reception process is described hereinafter.

The CPU 61 executes a display process to display the received image data in the first window 13 or the second window 15 , and a drawing process to reflect the annotation image drawn with the UI 14 to the first window 13 (step S 4 ).

The CPU 61 determines whether there is a projection request that requests to project the annotation image drawn in the first window 13 (step S 5 ). When the determination of the step S 5 is YES, the CPU 61 sends the annotation image drawn in the first window 13 to the drawing server 4 as a command (step S 6 ), and moves to the step S 7 described below.

When the determination of the step S 5 is NO, the CPU 61 determines whether there is a termination request (e.g. a request to shut down a power) (step S 7 ). When the determination of the step S 7 is YES, the present process is terminated. Meanwhile, when the determination of the step S 7 is NO, the CPU 61 goes back to the step S 2 .

FIG. 6 is a flowchart showing the image reception process of the step S 3 in FIG. 5 .

Here, the image data to be displayed in the first window 13 are described as the first image data, the second image data, and the third image data in order of the distribution. And 001 , 002 , 003 are added to each of the first image data, the second image data and the third image data as an identifier.

First, the CPU 61 sets the identifier of the first image data (it means 001 ) as the identifier of the image data received from the distribution server 3 (step S 11 ), and sends the request of the first image data to the distribution server 3 (step S 12 ). Then the CPU 61 starts to receive the response that means the first image data from the distribution server 3 (step S 13 ).

The CPU 61 updates the identifier of the image data received from the distribution server 3 from the identifier of the first image data to the identifier of the second image data (step S 14 ), and executes the request process of the second image data (step S 15 ).

FIG. 7 is a flowchart showing the request process of the second image data.

First, the CPU 61 determines whether it is receiving the termination request (e.g. a request to shut down a power) (step S 21 ). When the determination of the step S 21 is YES, the present process is terminated. Meanwhile, when the determination of the step S 21 is NO, the CPU 61 sends the request of the second image data to the distribution server 3 (step S 22 ). It should be noted that the second image data is captured after the first image data, and is newer than the first image data.

The CPU 61 starts to receive the response that means the second image data from the distribution server 3 (step S 23 ). Then, the CPU 61 determines whether the priority tag is included in the data header 51 of the second image data (step S 24 ). The priority tag is the tag to specify the image data to be displayed next after the image data being displayed in the first window 13 .

When the determination of the step S 24 is NO, the CPU 61 updates the identifier of the image data received from the distribution server 3 from the identifier of the second image data to the identifier of the third image data (step S 25 ), and goes back to the step S 21 . In the loop processing from the step S 21 to the step S 23 , the CPU 61 receives the data header 51 of the new image data after the data header 51 of the new image data, and determines whether its data header 51 includes the priority tag.

When the determination of the step S 24 is YES, the CPU 61 stops receiving the first image data, the process of the step S 13 in FIG. 6 (step S 26 ). And the CPU 61 registers the second image data to the HDD 64 as the image data to be displayed in the first window 13 next (step S 27 ), and moves to the step S 16 in FIG. 6 .

The CPU 61 determines whether it is receiving the request which requests to cancel the image reception process in FIG. 6 from the mouse and keyboard 65 (step S 16 ).

When the determination of the step S 16 is YES, the present process is terminated. Meanwhile, when the determination of the step S 16 is NO, the CPU 61 determines whether the reception of the image data with the priority tag has been finished (step S 17 ). When the determination of the step S 17 is NO, the CPU 61 goes back to the step S 16 . Meanwhile, when the determination of the step S 17 is YES, the CPU 61 stops the request process of the second image data (step S 18 ).

Lastly, the CPU 61 determines whether it is receiving the termination request (e.g. a request to shut down a power) (step S 19 ). When the determination of the step S 19 is YES, the present process is terminated. Meanwhile, when the determination of the step S 19 is NO, the CPU 61 goes back to the step S 11 .

FIG. 8 is a flowchart showing the interruption process executed during the display process of the step S 4 in FIG. 5 .

First, the CPU 61 of the PC 6 determines whether it has received the image data with the priority tag (step S 31 ). When the determination of the step S 31 is NO, the CPU 61 repeats this determination. Meanwhile, when the determination of the step S 31 is YES, the CPU 61 displays the message 91 showing that the CPU 61 has received the image data with the priority tag in the first window 13 (step S 32 ). As illustrated in FIG. 9 , this message 91 has a button 92 to stop updating the image data displayed in the first window 13 , and a button 93 to direct the CPU 61 to update the image data displayed in the first window 13 .

Then, the CPU 61 determines whether the direction of updating the image data displayed in the first window 13 is being input (step S 33 ). When the determination of the step S 33 is YES, the CPU 61 displays the image data with the priority tag in the first window 13 (step S 34 ), and goes back to the step S 31 . Meanwhile, when the determination of the step S 33 is NO, the CPU 61 keeps displaying the image data displayed in the first window 13 (step S 35 ), and goes back to the step S 31 .

FIG. 10 is a flowchart showing a process executed with the video server 2 .

First, the CPU 21 of the video server 2 determines whether it is receiving the termination request (e.g. a request to shut down a power) (step S 41 ). When the determination of the step S 41 is YES, the present process is terminated. Meanwhile, when the determination of the step S 41 is NO, the CPU 21 determines whether it is acquiring the new image data from the first camera 1 or the second camera 9 (step S 42 ).

When the determination of the step S 42 is NO, the CPU 21 goes back to the step S 41 . Meanwhile, when the determination of the step S 42 is YES, the CPU 21 determines whether it is receiving the request which requests to add the priority tag to the acquired image data (step S 43 ). The request which requests to add the priority tag to the image data is specified with the mouse and keyboard 25 that are connected to the video server 2 . Or it is specified by the access to the video server 2 from the PC 6 . Also it should be noted that the request which requests to add the priority tag to the image data may be specified automatically at predetermined intervals (e.g. ten seconds) based on the order of the program stored in HDD 24 .

When the determination of the step S 43 is YES, the CPU 21 adds the priority tag to the acquired image data (step S 44 ), and temporarily stores the image data with the priority tag in the HDD 24 (step S 45 ). Meanwhile, when the determination of the step S 43 is NO, the CPU 21 moves to the step S 45 . In this case, the image data temporarily stored in HDD 24 is not being added the priority tag to.

Then, the CPU 21 determines whether it is receiving the request of the image data temporarily stored in the HDD 24 from the distribution server 3 (step S 46 ). When the determination of the step S 46 is YES, the CPU 21 supplies the data image temporarily stored in the HDD 24 to the distribution server 3 (step S 47 ), and goes back to the step S 41 . Meanwhile when the determination of the step S 46 is NO, the CPU 21 goes back to the step S 41 .

FIG. 11 is a flowchart showing the process executed with the distribution server 3 .

First, the CPU 31 of the distribution server 3 determines whether it is receiving the termination request (e.g. a request to shut down a power) (step S 51 ). When the determination of the step S 51 is YES, the present process is terminated. Meanwhile when the determination of the step S 51 is NO, the CPU 31 determines whether it is receiving the request of the image data from the PC 6 (step S 52 ).

When the determination of the step S 52 NO, the CPU 31 goes back to the step S 51 . Meanwhile, when the determination of the step S 52 is YES, the CPU 31 requests the image data temporarily stored in the HDD 24 to the video server 2 (step S 53 ), supplies the image data received from the video server 2 to the PC 6 (step S 54 ), and goes back to the step S 51 .

FIG. 12 is a flowchart showing the process executed with the drawing server 4 .

The CPU 41 of the drawing server 4 waits until it receives a command from the PC 6 (step S 61 ). When the CPU 41 receives the command presenting the annotation image from the PC 6 (step S 62 ), the CPU 41 converts the command to the data for the projector 8 , and outputs it (step S 63 ). The projector 8 projects the annotation image onto the object 11 based on the converted data.

Then, the CPU 41 determines whether it is receiving the termination request (e.g. a request to shut down a power) (step S 64 ). When the determination of the step S 64 is YES, the present process is terminated. Meanwhile, when the determination of the step S 64 is NO, the CPU 41 goes back to the step S 61 .

As described in detail above, according to the exemplary embodiment, the CPU 61 starts to receive the several image data in parallel from the server system 1 to which the first camera 7 and the second camera 9 that capture the object are connected, detects the image data with the priority tag to be preferentially displayed on the display device 12 from the several image data being received, stops receiving the image data from which the priority tag hasn't been detected, and displays the image data from which the priority tag has been detected on the display device 12 . Therefore, the specified image data is preferentially displayed regardless of the reception condition of the image data. Also a communication band of the network is used efficiently because the reception of the image data from which the priority tag is not detected is stopped.

It should be noted that the CPU 61 may display the buttons 92 and 93 to choose whether a user stops updating the image data displayed on the display device 12 when the image data with the priority tag has been received. In addition it should be noted that before that the CPU 61 may display information showing that the CPU 61 has received the image data with the priority tag. In these cases, the user can figure out the display of the priority image, and choose whether or not to stop updating the captured image displayed on the display device 12 .

A recording medium on which the software program for realizing the functions of the PC 6 is recorded may be supplied to the PC 6 , and the CPU 61 of the PC 6 may read and execute the program recorded on the recording medium. In this manner, the same effects as those of the above-described exemplary embodiment can be achieved. The recording medium for providing the program may be a CD-ROM, a DVD, or a SD card, for example.

Alternatively, the CPU 61 of the PC 6 may execute a software program for realizing the functions of each PC, so as to achieve the same effects as those of the above-described exemplary embodiment.

It should be noted that the present invention is not limited to those exemplary embodiments, and various modifications may be made to them without departing from the scope of the invention.