Contextual keyboard located on a remote server for implementation on any content delivery and interaction application

This application is related to and claims priority from U.S. Provisional Application No. 61/943,291 filed on Feb. 21, 2014, entitled “A Contextual Keyboard Located On A Remote Server For Implementation On Any Website,” which is expressly incorporated by reference herein.

This disclosure relates to a contextual keyboard located on a remote server for implementation on any website or device capable of supporting an on-screen keyboard, such as a smartphone, tablet, TV, laptop, smart eyewear, smartwatch, etc.

The typical implementation of a keyboard to enter information on a content delivery and interaction application, such as a website, is either through a physical keyboard or a virtual keyboard that is installed on a device such as a smartphone, tablet, etc.

Historically, users have worked with physical keyboards connected to computers to input information into a content delivery and interaction application. With the advent of touch-based devices such as smartphones and tablets, the physical keyboard has been replaced by a virtual keyboard, with several options available to the user to install and use on their device. Both these foregoing methods are limited in the sense that the contextual element of what is being input on the content delivery and interaction application is limited to non-existent. Furthermore, in the case of certain virtual devices, such as those based on Apple Inc.'s current iOS system, only certain types of interactions with virtual keyboards are allowed, which limit the user experience. For example, tracing a sequence of letters to form a word is not allowed in any current virtual keyboard implementations; users can only enter one character/number/letter at a time.

Described are apparatus and methods for providing a contextual keyboard service located on a remote server for implementation on any content delivery and interaction application such as a website or device capable of supporting an on-screen keyboard.

FIG. 1 illustrates an overview of the system according to embodiments described herein.

FIG. 2 illustrates a flowchart of the process according to embodiments described herein.

Described are methods and apparatus to implement a virtual keyboard, which address conventional limitations, while enhancing the user experience.

By virtual keyboard is meant that on a display screen of a smartphone, tablet, laptop, smart TV or other computer device with a display screen, that when the user opens a content delivery and interaction application, which can be an application (such as virtual reality or augmented reality or other application) or an interface on the device that requires text input or connects to a particular website at a particular website address, that either the application/interface or the website itself is configured with the virtual keyboard as described herein, which allows configuration that is specific to specific features of that particular device or website, and known characteristics of users of such a device or website.

It is understood that the smartphone, tablet, laptop or other computer device includes, in addition to the display screen mentioned, a processor, memory, software that controls the device and allows opening of applications, connection to websites and the like, in a conventional manner. In addition, the computer device allows for the implementation of the capabilities described herein, which include communication with a remote server on which the content delivery and interaction application, such as the accessed website, that contains the functionality herein is hosted.

FIG. 1 illustrates an overview of the system 100 , in which a computer device 110 communicates with a server 120 over a network 130 , using a communication protocol, such as a TCP/IP protocol.

The methodology for implementing the virtual and contextual keyboard system implemented on a website in the following manner, and it is understood that other content delivery and interaction application can be similarly implemented:

a) An SDK, in particular an image and an API endpoint,

written in a special programming language, such as Javascript is provided to the website administrator, or device/application developer as described hereinafter.

b) The website administrator/application developer incorporates this SDK into their code for the website/application, such that when a user selects a search button or a text input field or equivalent on this particular website/application, it results in a visual of the virtual keyboard on the display screen of the computer device of the user.

c) The user can enter text, numbers, special characters, etc., from their computer device by interacting with the virtual keyboard that appears on the display screen, such as in a drop down menu when the user selects a search button or a text input field or equivalent on this particular website. This interaction with the virtual keyboard could be programmed in a number of different ways, such as one keystroke (single mouse click or a touch input on a character on the virtual keyboard) at a time or by drawing a trail through a sequence of keys in a continuous manner. This latter implementation enhances the user experience for those on an iOS device.

d) The SDK stores the coordinates of the inputs relative to the layout of the virtual keyboard and sends this information over the network to the host server, where the contextual keyboard system interprets the coordinates, maps them to actual letters, characters, numbers and/or words and sends the information back over the network to the display screen of the computer device of the user, where it is displayed in real-time.

e) Further, the system also offers suggestions for additional words in the sequence of input text being entered by the user. These suggestions are based on context-awareness and can, if desired, incorporate trending information culled from the world wide web and/or information specific to the website (either provided by the website or obtained from elsewhere) or the application where the virtual keyboard is displayed, which makes for a better user experience.

FIG. 2 illustrates a flowchart of the server process 200 that occurs at the server, once the SDK is installed. These initially include step 210 in which the page is delivered to the user in the format corresponding to the computer device and which includes the API. Step 210 is followed by step 220 , in which once the user selects a search button or a text input field or equivalent on this particular website and the API is called to deliver to the display screen of the user the virtual keyboard, as well as the functionality to interact with the virtual keyboard caused inputs, the API registered coordinates of the inputs relative to the layout of the virtual keyboard are received at the server from over the network. In step 230 , the delivered inputs are operated upon by the website server, which then delivers back to the user computer the actual letters, characters, numbers and/or words.

In a particular aspect, it is apparent that the API can be customized to each different particular website/application, thus allowing, for example, the programming differences to be content based. Thus, for example, a user who virtually selects “bb” on a sports website using the embodiments as described would be provided a baseball sports page link, whereas a user who virtually selects “bb” on a site relating to different types of mobile devices would be provided a link to go to a webpage associated with a Blackberry device. In another use case, a search conducted on a set-top box connected to a TV, would bring up choices for TV shows and movies. In a different use case where the user is searching in a mapping application, the results will correspond to geographical location prediction.

One particular implementation of the SDK is described hereinafter:

Predictions REST API

The Predictions SDK is a REST API that can be used by clients to retrieve word predictions corresponding to a trail on the keyboard. The key elements of using the API are:

The keyboard image The Rest API call

The Keyboard Image

The Predictions API depends on the coordinates of the trail, hence the dimensions of the keyboard image are important.

The REST API Call

Once the keyboard image has been displayed and the trail recorded (typically through one of the onMouse* events in Javascript), a call can be made to the API endpoint to get the prediction.

The expected call is a HTTP POST request with the arguments listed below.

Arguments

The arguments to the call are:

coords

This is a list of coordinate objects each of which contains two fields: x and y. This could be an Object in javascript, a list of dictionaries in Python or any equivalent structure in other programming languages. This parameter MUST be JSON encoded before it is sent to the server.

For example, the coordinate list [(1, 1), (2, 2), (3, 3)] would be encoded as:

[{“y”: 1, “x”: 1}, {“y”: 2, “x”: 2}, {“y”: 3, “x”: 3}]

cookie

The cookie is an opaque value that is handed to clients by the server. The cookie is the server's way of keeping track of a “typing session”. The first request is made without a cookie. The cookie from the response is used for the next request and so on. The response to every call will result in an updated cookie for use with the next call. While passing the cookie hack on each request isn't required, it is recommended since it will increase prediction accuracy.

A typical encoded parameter list looks like:

coords=[{“y”: 1, “x”: 1}, {“y”: 2, “x”: 2}, {“y”: 3, “x”: 3}]&cookie=[ ]

Response

The response is json encoded and consists of two elements

predction

This is the prediction of the word corresponding to the trail.

cookie

This is the cookie to be used for the next request.

A typical response looks like:

{“cookie”: [5882], “prediction”: “cat”}

Code samples

Sending a request

The following function saves the coordinates to a list:

The following function sends the list of coordinates to the server for a prediction, decodes the response and logs it:

Although the present inventions are described with respect to certain preferred embodiments, modifications thereto will be apparent to those skilled in the art.