Methods and systems for interacting with an imaging device

Embodiments of the present invention comprise methods and systems for interacting with an imaging device, in particular, for interacting with an imaging device from a remote computing device.

It may be desirable to control an imaging device remotely from a remote computing device. Methods and systems for replicating the imaging device user interface at the remote computing device may desirable. In particular, replicating the imaging device user interface at the remote computing device without requiring specialized imaging-device-associated software may be desirable.

Embodiments of the present invention comprise methods and systems for interacting with an imaging device.

In some embodiments, an imaging device may receive a request from a remote computing device for an interface image associated with the imaging device. The interface image may comprise an image of those portions of the imaging device required for effective user interface with the imaging device. In some embodiments of the present invention, an interface image may be a functional replica of an interface associated with the imaging device. In some embodiments, the functional replica may be visually similar to the interface. In alternative embodiments, the functional replica may not be visually similar to the interface, while still providing the functionality of the interface. In some embodiments, the interface image may comprise an image of the UI panel of the imaging device. The imaging device may obtain an interface image in response to the request and may send the interface image to the remote computing device. In some embodiments of the present invention, the request may comprise an HTTP request, and the interface image may be sent to the remote computing device in an HTTP response.

In some embodiments, an imaging device may receive, from a remote computing device, a change request associated with an interface at the imaging device and an interface image displayed at the remote computing device. The imaging device may send a change indicator to the remote computing device indicating whether or not the interface image currently available at the remote computing device is a functional replica of the current interface at the imaging device. In some embodiments, the change request may comprise an HTTP request, and the change indicator may be sent to the remote computing device in an HTTP response.

In some embodiments, an imaging device may receive, from a remote computing device, user-input information associated with the interface image obtained at the remote computing device, and the imaging device may be controlled in accordance with the received user-input information. In some embodiments, the user-input information may be received through a query string in the request line of an HTTP request.

In alternative embodiments of the present invention, a remote computing device may send a request to an imaging device for an interface image associated with the imaging device. The interface image may comprise an image of those portions of the imaging device required for effective user interface with the imaging device. In some embodiments of the present invention, an interface image may be a functional replica of an interface associated with the imaging device. In some embodiments, the functional replica may be visually similar to the interface. In alternative embodiments, the functional replica may not be visually similar to the interface, while still providing the functionality of the interface. In some embodiments, the interface image may comprise an image of the UI panel of the imaging device. The remote computing device may receive an interface image in response to the request and may display the interface image in a web page at the remote computing device. In some embodiments of the present invention, the request may comprise an HTTP request, and the interface image may be received in an HTTP response.

In some embodiments, a remote computing device may send, to an imaging device, a change request associated with an interface at the imaging device and an interface image displayed at the remote computing device. The remote computing device may receive from the imaging device a change indicator indicating whether or not the interface image currently available at the remote computing device is a functional replica of the current interface at the imaging device. In some embodiments, the change request may comprise an HTTP request, and the change indicator may be received in an HTTP response. In some embodiments, the remote computing device may request an updated interface image in response to a positive change indicator.

In some embodiments, a remote computing device may send, to an imaging device, user-input information received at the computing device in association with a display interface image. In some embodiments, the user-input information may be sent in a query string in the request line of an HTTP request.

In some embodiments of the present invention, a remote computing device and an imaging device may be connected through a gateway computing device, wherein the gateway computing device may receive requests from the remote computing device related to an interface at the imaging device. The gateway computing device may obtain interface-related information from the imaging device in response to requests from the remote computing device, and the gateway computing device may send the interface-related information to the remote computing device.

The foregoing and other objectives, features, and advantages of the invention will be more readily understood upon consideration of the following detailed description of the invention taken in conjunction with the accompanying drawings.

FIG. 1 is a picture showing embodiments of the present invention comprising an imaging device connected via a network to a remote computing device;

FIG. 2 is a chart showing exemplary embodiments of the present invention comprising a remote computing device requesting an interface image from an imaging device, wherein the interface image is a functional replica of an interface on the imaging device;

FIG. 3 is a chart showing exemplary embodiments of the present invention comprising an imaging device receiving, from a remote computing device, a request for an interface image and the imaging device obtaining and sending an interface image to the remote computing device;

FIG. 4 is a chart showing exemplary embodiments of the present invention comprising a remote computing device polling an imaging device for interface changes;

FIG. 5 is a chart showing exemplary embodiments of the present invention comprising sending, from an imaging device to a remote computing device, a status indicator indicating the status of an interface image currently displayed at the remote computing device relative to the interface at the imaging device;

FIG. 6 is a chart showing exemplary embodiments of the present invention comprising receiving user input at a remote computing device and sending user-input information associated with the received user input from the remote computing device to an imaging device;

FIG. 7 is a chart showing exemplary embodiments of the present invention comprising receiving user-input information at an imaging device from a remote computing device and controlling the imaging device in accordance with the user-input information;

FIG. 8 is a chart showing exemplary embodiments of the present invention comprising a remote computing device requesting front panel information from an imaging device and sending user-input information from the remote computing device to the imaging device;

FIG. 9 is a picture of exemplary embodiments of the present invention comprising an imaging device and a remote computing device communicating via a gateway computing device;

FIG. 10 is a chart showing exemplary embodiments of the present invention comprising a gateway computing device receiving a request from a remote computing device for an interface image associated with an imaging device and the gateway computing device obtaining an interface image and sending it to the remote computing device;

FIG. 11 is a chart showing exemplary embodiments of the present invention comprising a remote computing device polling for interface changes on an imaging device through a gateway computing device; and

FIG. 12 is a chart showing exemplary embodiments of the present invention comprising a gateway computing device transmitting user-input information received from a remote computing device to an imaging device.

Embodiments of the present invention will be best understood by reference to the drawings, wherein like parts are designated by like numerals throughout. The figures listed above are expressly incorporated as part of this detailed description.

It will be readily understood that the components of the present invention, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of the embodiments of the methods and systems of the present invention is not intended to limit the scope of the invention but it is merely representative of the presently preferred embodiments of the invention.

Elements of embodiments of the present invention may be embodied in hardware, firmware and/or software. While exemplary embodiments revealed herein may only describe one of these forms, it is to be understood that one skilled in the art would be able to effectuate these elements in any of these forms while resting within the scope of the present invention.

It may be desirable to control an imaging device remotely from a remote computing device. Methods and systems for replicating the imaging device user interface at the remote computing device may desirable. In particular, replicating the imaging device user interface at the remote computing device without requiring specialized imaging-device-associated software may be desirable.

Exemplary embodiments of the present invention may comprise an imaging device and a remote computing device. In some embodiments of the present invention shown in FIG. 1 , an imaging device 10 and a remote computing device 12 may be communicatively coupled through a network 14 . Exemplary networks include local area networks (LANs), wide area networks (WANs), the Internet, cellular networks, satellite networks, wireless proximity networks (for example, Bluetooth, Wi-Fi, WiMAX and other wireless proximity networks) and other networks. In alternative embodiments, the imaging device 10 and the remote computing device 12 may be communicatively coupled via a communication link, for example, a telephone line, a serial cable, an IrDA (Infrared Data Association) link, and other, wired or wireless, communications links.

In some embodiments, the imaging device 10 may comprise a multi-function peripheral (MFP) device that combines the functions of two or more traditionally separated imaging devices. An MFP may combine any number of imaging devices, but typically comprises the functions of one or more of the following exemplary imaging devices: a printer, a scanner, a copier, a filing device, a document-management device, a publishing device, a media-duplication device, a display device and a fax machine. In alternate embodiments, the imaging device 10 may comprise a single-function imaging device. Exemplary single-function imaging devices may be a printer, a scanner, a copier, a filing device, a document-management device, a publishing device, a media-duplication device, a display device and a fax machine.

The imaging device 10 may comprise a user interface (UI) panel 16 , which may comprise input buttons 18 and/or a display device 20 . In some embodiments, the display device 20 may comprise a touch panel system with or without input buttons. In some embodiments of the present invention, the imaging device 10 may comprise a user interface, wherein the components of the user interface may be proximately located. In alternative embodiments of the present invention, the components of the user interface may not be proximately located. An exemplary user interface panel 16 may be the front panel on an imaging device 10 .

Exemplary remote computing devices include personal computers, for example, desktop computers, laptop computers, tablet computers, handheld computers and other personal computers, cellular telephones, personal digital assistant (PDA) devices and other computing devices. The remote computing device 12 may comprise a display 22 and a device 24 , or devices, by which a user may provide user input. Exemplary input devices may include a mouse, a keyboard, a pointing device, a joystick, a touch screen and other input devices. The remote computing device 12 may comprise a web browser (not shown).

Communication and transport of data between the imaging device 10 and the remote computing device 12 may be by any protocol or combination of protocols, of which exemplary protocols may comprise WS/SOAP (Web Services/Simple Object Access Protocol), SOAP/XML (Simple Object Access Protocol/eXtensible Markup Language), DIME (Direct Internet Message Encapsulation), FTP (File Transfer Protocol), NFS (Network File System), SMTP (Simple Mail Transfer Protocol), HTTP/HTML (HyperText Transfer Protocol/HyperText Markup Language), Email, a proprietary protocol over TCP/IP (Transmission Control Protocol/Internet Protocol) and AppleTalk®.

Some embodiments of the present invention may be described in relation to FIG. 1 and FIG. 2 . In these embodiments, a web page associated with an imaging device 10 may be loaded 30 at a remote computing device 12 . A JavaScript inside the web browser at the imaging device 10 may connect 32 to the imaging device 10 and may send 34 a request for an interface image associated with the imaging device 10 . In some embodiments of the present invention, the request for the interface image may comprise an HTTP request.

An interface image associated with an imaging device 10 may comprise an image of those portions of the imaging device 10 required for effective user interface with the imaging device 10 . In some embodiments of the present invention, an interface image may be a functional replica of an interface associated with the imaging device 10 . In some embodiments, the functional replica may be visually similar to the interface. In alternative embodiments, the functional replica may not be visually similar to the interface, while still providing the functionality of the interface. In some embodiments, the interface image may comprise an image of the UI panel 16 of the imaging device.

The remote computing device 12 may receive 36 an interface image from the imaging device 10 . In some embodiments of the present invention, the interface image may be received 36 in an HTTP response. In some embodiments of the present invention, the interface image may be a JPEG image. The interface image may be in any image format known in the art. Exemplary image formats may include JPEG, JPEG-2000, PNG, BMP, TIFF, GIF and SVG (Scalable Vector Graphics). The remote computing device 12 may display 38 the interface image in the web page from which the request for the interface image was initiated.

The JavaScript associated with the web page from which the interface image was requested may poll 40 the imaging device 10 for interface changes relative to the currently displayed interface image. In some embodiments, the polling may occur approximately once a second. In some embodiments, the polling frequency may be configurable. In some of these embodiments, the polling frequency may be adjusted based on network 14 traffic loads. In some embodiments, the polling may invoke an HTTP script which may check to see if an interface image associated with the imaging device 10 interface differs from the currently displayed interface image at the remote computing device 12 . In some embodiments of the present invention, the polling request may comprise an HTTP request.

Some embodiments of the present invention may be described in relation to FIG. 1 and FIG. 3 . In these embodiments, an imaging device 10 may receive 42 a request for an interface image from a remote computing device 12 . In some embodiments of the present invention, the request for the interface image may comprise an HTTP request. The imaging device 10 may obtain 44 an interface image corresponding to the state of the imaging-device interface at a time associated with the receipt of the request, and the imaging device 10 may send the interface image to the remote computing device 46 . In some embodiments of the present invention, the interface image may be a JPEG image. The interface image may be in any image format known in the art. Exemplary image formats may include JPEG, JPEG-2000, PNG, BMP, TIFF, GIF and SVG. In some embodiments of the present invention, the interface image may be sent 46 to the remote computing device 12 in an HTTP response.

In some embodiments of the present invention, obtaining 48 an interface image may comprise obtaining a “screen shot” of the imaging device 10 interface. In alternative embodiments of the present invention, images corresponding to interface states may be pre-stored in imaging device 10 storage, which may be located within the imaging device 10 or remote to the imaging device 10 .

Some embodiments of the present invention may be described in relation to FIG. 1 and FIG. 4 . In these embodiments, a remote computing device 12 may send 50 a status-change request to an imaging device 10 . In some embodiments, the status-change request may comprise an HTTP request. The status-change request may be associated with an interface image displayed at the remote computing device 12 . The status-change request may request to receive a change indicator that indicates if the interface image currently displayed at the remote computing device 12 is a functional replica of the current interface at the imaging device 10 at a time associated with the receipt of the request. The remote computing device 12 may receive 52 a change-request response from the imaging device 10 . In some embodiments, the change-status response may comprise an HTTP response, wherein a status-change indicator may be communicated in the response body. In some embodiments, the status-change indicator may be a “0” indicating no change and that the currently displayed interface image is a functional replica of the imaging device 10 interface. In some embodiments, the status-change indicator may be a “1” indicating that the currently displayed interface image is no longer a functional replica of the imaging device interface.

The remote computing device 12 may examine 54 the received status-change indicator, and if the status-change indicator indicates “no change” 56 , then another status-change request may be sent 50 to the imaging device 10 at the next-scheduled polling time. If the status-change indicator indicates “change” 58 , then a request may be sent 60 to the imaging device 10 for an interface image. In some embodiments, the request for an interface image may comprise an HTTP request. A current interface image may be received 62 at the remote computing device 12 from the imaging device 10 , and the current interface image may be displayed 64 . In some embodiments, the new interface image may be received 62 in an HTTP response. In some embodiments of the present invention, the interface image may be a JPEG image. The interface image may be in any image format known in the art. Exemplary image formats may include JPEG, JPEG-2000, PNG, BMP, TIFF, GIF and SVG. Polling of the imaging device 10 may continue 50 .

Some embodiments of the present invention may be described in relation to FIG. 1 and FIG. 5 . In these embodiments, an imaging device 10 may receive 70 , from a remote computing device 12 , a status-change request. The status-change request may be associated with an interface image. In some embodiments, the status-change request may comprise an HTTP request. The imaging device 10 may compare 72 the interface image with the current interface state to determine 74 if the interface image is a functional replica of the imaging device 10 interface at a time associated with the receipt of the status-change request. If the current interface is not 76 substantially different from the interface image, a status-change indicator indicating “no change” may be sent 78 from the imaging device 10 to the remote computing device 12 . If there are substantial differences 80 , then a status-change indicator indicating that the interface image is no longer a functional replica of the interface may be sent 82 from the imaging device 10 to the remote computing device 12 . In some embodiments of the present invention, the status-change indicator may be sent in an HTTP response in the response body.

Some embodiments of the present invention may be described in relation to FIG. 1 and FIG. 6 . In these embodiments, user input may be received 90 at a remote computing device 12 . The user input may be related to an interface image, displayed at the remote computing device 12 , associated with an imaging device 10 . Information associated with the user input may be sent 92 from the remote computing device 12 to the imaging device 10 . In some embodiments of the present invention, the user-input information may be encoded as standard query string parameters in a request line of an HTTP request. Exemplary user input may comprise a spatial location associated with a mouse click, a key code associated with a keyboard interaction, mouse-button status associated with a mouse click, a spatial location associated with a touch-panel touch, a joystick position associated with a joystick click and other user input.

In some embodiments of the present invention, after user-input information is sent 92 to the imaging device 10 from the remote computing device 12 , a request may be sent to the imaging device 10 for an interface image irrespective of the polling cycle.

Some embodiments of the present invention may be described in relation to FIG. 1 and FIG. 7 . In these embodiments, user-input information may be received 100 at an imaging device 10 from a remote computing device 12 . The imaging device 10 may be controlled 102 in accordance with the received 100 user-input information. In some embodiments of the present invention, the interface at the imaging device 10 may be updated in accordance with the received 100 user-input information. The update may reflect the user-input information in a manner consistent with corresponding, direct user input at the imaging device 10 interface.

In some embodiments of the present invention, a remote computing device 12 may send a connection-status request to an imaging device 10 . The connection-status request may be sent through an HTTP request in some embodiments. The connection-status request may request the status of an interface associated with the imaging device 10 , for example, the front panel. The imaging device 10 may receive the request and send a response indicating the connection status. In some embodiments, the response may be sent in the body of an HTTP response.

Some embodiments of the present invention may be understood in relation to FIG. 1 and FIG. 8 . In these embodiments, a web browser 110 on a remote computing device may request 114 , from an imaging device 112 , an imaging of the imaging device front panel. The request may comprise an HTTP request. The imaging device 112 may return 116 an image of the imaging device 112 front panel to the browser 110 . In some embodiments, the image may be returned 116 to the browser 110 in the body of an HTTP response. The browser 10 may poll 118 the imaging device 112 for changes in the front panel. The polling may comprise an HTTP request. If the front panel has changed, the imaging device 112 may return a “true” value 120 . In some embodiments, the “true” response may be received at the browser 110 in the body of an HTTP response. When a “true” value is returned 120 , the browser 110 may be instructed 122 to reload the image, thereby initiating another request 114 for an image of the front panel. If the value returned is “false” 124 , polling 118 may continue. In some embodiments, the “false” response may be received at the browser 110 in the body of an HTTP response. When mouse events are received at the browser 110 , the mouse events may be sent 126 to the imaging device 112 . In some embodiments, the mouse events may be sent 126 through a query string in the request line of an HTTP request. When keyboard events are received at the browser 100 , the keyboard events may be sent 128 to the imaging device 112 . In some embodiments, the keyboard events may be sent 128 through a query string in the request line of an HTTP request.

In some embodiments of the present invention, wherein communication between an imaging device and a remote computing device comprises standard web-based protocols, validation and authentication of a user may be performed at the imaging device using any of the known-in-the-art methods of validation and authentication. The imaging device may further comprise security mechanisms that may secure embedded web pages, and in some embodiments, these mechanisms may be used to ensure authorized and authenticated access to an HTML page that may host the interface image.

Alternative embodiments of the present invention may be described in relation to FIG. 9 . A gateway computing device 134 may be communicatively coupled with an imaging device 130 and a remote computing device 132 . In some embodiments, the gateway computing device 134 , the imaging device 130 and the remote computing device 132 may be communicatively coupled through a network connection. Exemplary networks include local area networks (LANs), wide area networks (WANs), the Internet, cellular networks, satellite networks, wireless proximity networks (for example, Bluetooth, Wi-Fi, WiMAX and other wireless proximity networks) and other networks. In alternative embodiments, the gateway computing device 134 , the imaging device 130 and the remote computing device 132 may be communicatively coupled via a communication link, for example, a telephone line, a serial cable, an IrDA (Infrared Data Association) link, and other, wired or wireless, communications links.

In some embodiments of the present invention, the imaging device 130 may host a Virtual Network Computing (VNC) server for a remote-front-panel application. The gateway computing device 134 may run a remote-front-panel web service application.

In some embodiments, the imaging device 130 may comprise a multi-function peripheral (MFP) device that combines the functions of two or more traditionally separated imaging devices. An MFP may combine any number of imaging devices, but typically comprises the functions of one or more of the following exemplary imaging devices: a printer, a scanner, a copier, a filing device, a document-management device, a publishing device, a media-duplication device, a display device and a fax machine. In alternate embodiments, the imaging device 10 may comprise a single-function imaging device. Exemplary single-function imaging devices may be a printer, a scanner, a copier, a filing device, a document-management device, a publishing device, a media-duplication device, a display device and a fax machine.

The imaging device 130 may comprise a user interface (UI) panel 136 , which may comprise input buttons 138 and/or a display device 140 . In some embodiments, the display device 140 may comprise a touch panel system with or without input buttons. In some embodiments of the present invention, the imaging device 130 may comprise a user interface, wherein the components of the user interface may be proximately located. In alternative embodiments of the present invention, the components of the user interface may not be proximately located. An exemplary user interface panel 136 may be the front panel on an imaging device 130 .

Exemplary remote computing devices include personal computers, for example, desktop computers, laptop computers, tablet computers, handheld computers and other personal computers, cellular telephones, personal digital assistant (PDA) devices and other computing devices. The remote computing device 132 may comprise a display 142 and a device 144 , or devices, by which a user may provide user input. Exemplary input devices may include a mouse, a keyboard, a pointing device, a joystick, a touch screen and other input devices. The remote computing device 132 may comprise a web browser (not shown).

Communication and transport of data between the imaging device 130 and the gateway computing device 134 may be by any protocol or combination of protocols, of which exemplary protocols may comprise WS/SOAP (Web Services/Simple Object Access Protocol), SOAP/XML (Simple Object Access Protocol/eXtensible Markup Language), DIME (Direct Internet Message Encapsulation), FTP (File Transfer Protocol), NFS (Network File System), SMTP (Simple Mail Transfer Protocol), HTTP/HTML (HyperText Transfer Protocol/HyperText Markup Language), Email, a proprietary protocol over TCP/IP (Transmission Control Protocol/Internet Protocol), VNC (Virtual Network Computing) protocol and AppleTalk®.

Communication and transport of data between the remote computing device 132 and the gateway computing device 134 may be by any protocol or combination of protocols, of which exemplary protocols may comprise WS/SOAP (Web Services/Simple Object Access Protocol), SOAP/XML (Simple Object Access Protocol/eXtensible Markup Language), DIME (Direct Internet Message Encapsulation), FTP (File Transfer Protocol), NFS (Network File System), SMTP (Simple Mail Transfer Protocol), HTTP/HTML (HyperText Transfer Protocol/HyperText Markup Language), Email, a proprietary protocol over TCP/IP (Transmission Control Protocol/Internet Protocol) and AppleTalk®.

In some embodiments of the present invention described in relation to FIG. 9 and FIG. 10 , a gateway computing device 134 may receive 150 a request from a remote computing device 132 for an interface image associated with an imaging device 130 . In some embodiments, the request may comprise an HTTP request. The gateway computing device 134 may send 152 , to the imaging device 130 , an interface-data-update request requesting update information related to an interface associated with the imaging device 130 . In some embodiments, the interface-data-update request may be sent to the imaging device 130 from the gateway computing device 134 using a VNC/RFB (Remote FrameBuffer) protocol. The gateway computing device 134 may receive 154 , from the imaging device 130 , interface update information. The gateway computing device 134 may convert 156 interface data associated with the imaging device to an interface image and may send 158 the interface image to the remote computing device 132 . In some embodiments, the gateway computing device 134 may convert interface data from RFB to SOAP compliant binary data.

In some embodiments of the present invention described in relation to FIG. 9 and FIG. 11 , a gateway computing device 134 may receive 160 , from a remote computing device 132 , a status-change request associated with an interface image. In some embodiments, the status-change request may comprise an HTTP request. The gateway computing device 134 may send 162 , to an imaging device 130 , an interface-data-update request requesting update information related to an interface associated with the imaging device 130 . In some embodiments, the interface-data-update request may be sent to the imaging device 130 from the gateway computing device 134 using a VNC/RFB (Remote FrameBuffer) protocol. The gateway computing device 134 may receive 164 , from the imaging device 130 , interface update information. The gateway computing device 134 may compare 166 the interface image currently associated with the remote computing device 132 with the current state of the interface at the imaging device 130 . If the gateway computing device 134 determines 168 that there is not 170 a substantial difference between the interface image currently associated with the remote computing device 132 with the current state of the interface at the imaging device 130 , then the gateway computing device may send 172 a “no change” indicator to the remote computing device 132 . If the gateway computing device 134 determines 168 that there is 174 a substantial difference between the interface image currently associated with the remote computing device 132 with the current state of the interface at the imaging device 130 , then the gateway computing device may send 176 a “changed” indicator to the remote computing device 132 . In some embodiments of the present invention, the status indicator may be sent in an HTTP response.

In some embodiments of the present invention described in relation to FIG. 9 and FIG. 12 , a gateway computing device 134 may receive 180 , from a remote computing device 132 , user interaction data. In some embodiments, the user interaction data may be received through a query string in a request line of an HTTP request. The gateway computing device 134 may send 182 the user interaction data to an imaging device 130 .

The user interaction data may be related to an interface image, displayed at the remote computing device 132 , associated with an imaging device 130 . Exemplary user input may comprise a spatial location associated with a mouse click, a key code associated with a keyboard interaction, mouse-button status associated with a mouse click, a spatial location associated with a touch-panel touch, a joystick position associated with a joystick click and other user input.

The terms and expressions which have been employed in the foregoing specification are used therein as terms of description and not of limitation, and there is no intention in the use of such terms and expressions of excluding equivalence of the features shown and described or portions thereof, it being recognized that the scope of the invention is defined and limited only by the claims which follow.