Adjunct processing of multi-media functions in a messaging system

The present invention relates generally to media processing in messaging systems and, more particularly, to extending processing abilities of messaging systems to efficiently perform a variety of multi-media processing functions.

Messaging systems that provide voice and fax messaging capabilities are well known. One example of such a messaging system is the Network Applications Platform (NAP) commercially available from UNISYS Corporation (“the NAP system”). The NAP is a configuration of hardware and software that provides data and voice processing capabilities through applications running on a host computer system. The NAP, in combination with a network interface unit (NIU), provides the interface between these applications, called network applications, and a telephone network. The NAP is implemented on selected Unisys A Series and ClearPath HMP NX computer systems running the Unisys MCP operating system. Further details of the structure and function of the NAP are provided in the following issued patents and pending applications, all of which are hereby incorporated by reference in their entireties:

U.S. Pat. No. 5,133,004, issued Jul. 21, 1992, entitled “Digital Computer Platform for Supporting Telephone Network Applications”;

U.S. Pat. No. 5,323,450, issued Jun. 21, 1994, entitled “Telephone Network Applications Platform for Supporting Facsimile Applications”;

U.S. Pat. No. 5,384,829, issued Jan. 24, 1995, entitled “Digital Computer Platform for Supporting Telephone Network Applications”;

U.S. Pat. No. 5,493,606, issued Feb. 20, 1996, entitled “Multi-Lingual Prompt Management System for a Network Applications Platform”;

U.S. Pat. No. 6,058,166, issued May 2, 2000, entitled “Enhanced Multi-Lingual Prompt Management in a Voice Messaging System With Support for Speech Recognition”;

U.S. patent application Ser. No. 09/161,214, filed Sep. 25, 1998, entitled “Multiple Node Messaging System Wherein Nodes Have Shared Access To Message Stores Of Other Nodes”;

U.S. patent application Ser. No. 09/307,014, filed May 7, 1999, entitled “Inter-System Call Transfer”; and

U.S. patent application Ser. No. 09/451,077, filed Nov. 30, 1999, entitled “Method and Apparatus for Preventing Hung Calls During Protocol Violations in a Voice Messaging System”.

Network applications running on the NAP 20 can provide enhanced services for telephone networks, such as Call Answer, Call Forward, Voice Store and Forward, Voice Mail, Bank By Phone, Emergency Number Facilities, Directory Assistance, Operator Assistance, Call Screening, Automatic Wake-up Services, and the like.

So-called “Universal Messaging” systems are an attempt to combine traditional voice and fax messaging capabilities with electronic mail (“e-mail”) capabilities. One approach to providing universal messaging system functionality is to, in effect, integrate an existing voice messaging system, such as the Unisys NAP, with an existing e-mail system. Each platform retains its own message store, but the system integrates those stores so that a subscriber can access all of the subscriber's voice, fax and e-mail messages from a single interface. Universal messaging systems typically provide access to messages from either a Web client, such as a Web browser, or a traditional telephone handset.

With the advent of multi-media processing, both traditional voice and fax messaging systems, such as the Unisys NAP, and universal messaging systems that combine voice, fax, and e-mail messaging capabilities, are likely to experience a need for a variety of multi-media processing capabilities, such as text-to-speech conversion (so that an e-mail message can be “played” over a telephone handset in a universal messaging system) and speech recognition (to enable voice commands to be interpreted or to enable voice messages to be converted to text). Also, both traditional messaging systems and newer universal messaging systems may in the future incorporate natural language understanding capabilities, which again requires specialized processing.

Unfortunately, traditional voice and fax messaging systems, whether integrated with other messaging platforms in a universal messaging solution or not, are limited in the amount and type of multi-media processing they can perform. These systems are not currently capable of handling the large volumes of multi-media processing that today's messaging services may require. Consequently, there is a need for improved systems and methods for efficiently performing various kinds of multi-media processing, such as text-to-speech (TTS), automated speech recognition (ASR), and natural language understanding (NLU) in a messaging system. The present invention satisfies this need.

The present invention is directed to systems and methods that enable existing messaging systems to efficiently perform multi-media processing. The present invention is particularly useful in an integrated universal messaging system in order to provide an improved method for performing a variety of multi-media processing functions in such a system, including, for example, text-to-speech processing, automated speech recognition processing, and natural language understanding processing.

A messaging system in accordance with the present invention comprises (i) a voice messaging platform for receiving, processing, and storing messages; (ii) at least one network interface unit (NIU) having a first interface to the voice messaging platform for communicating between the NIU and the voice messaging platform and a second interface to a telephone network for receiving calls from the telephone network; (iii) at least one embedded services platform (ESP) operatively coupled to the first and second interfaces of the NIU, the ESP comprising a processor, a memory, and an operating system executing on the processor for executing software applications that are otherwise incapable of executing within the NIU, the ESP further comprising a network interface for communicating between the ESP and a network external to the messaging system; and (iv) at least one media resource server connected to the ESP via the network interface of the ESP, the ESP offloading selected multi-media processing functions to the media resource server. This eliminates the need for such processing functions to be performed by the voice messaging platform and thus results in improved efficiency.

A universal messaging system in accordance with the present invention comprises (i) a voice messaging platform for receiving, processing, and storing voice messages; (ii) an e-mail messaging platform interfaced to the voice messaging platform for receiving, processing, and storing e-mail messages; (iii) at least one network interface unit (NIU) having a first interface to the voice messaging platform for communicating between the NIU and the voice messaging platform and a second interface to a telephone network for receiving calls from the telephone network; (iv) at least one embedded services platform (ESP) operatively coupled to the first and second interfaces of the NIU, the ESP comprising a processor, a memory, and an operating system executing on the processor for executing software applications that are otherwise incapable of executing within the NIU, the ESP further comprising a network interface for communicating between the ESP and a network external to the messaging system; and (iv) at least one media resource server connected to the ESP via the network interface of the ESP, the ESP offloading selected multi-media processing functions to the media resource server.

Additional features and advantages of the present invention will become evident hereinafter.

The foregoing summary, as well as the following detailed description of the preferred embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the invention, there is shown in the drawings an embodiment that is presently preferred, it being understood, however, that the invention is not limited to the specific methods and instrumentalities disclosed. In the drawings:

FIG. 1 is a block diagram of an exemplary messaging system in accordance with an embodiment of the present invention;

FIG. 2 is block diagram of the software components employed by the messaging system of FIG. 1 in accordance with the present invention;

FIG. 3 provides further details of the software components represented in FIG. 2 ;

FIG. 4 is a block diagram of an exemplary universal messaging system in accordance with an embodiment of the present invention; and

FIG. 5 is a flowchart of the processing performed by the universal messaging system when processing a request for Text-To-Speech (TTS) conversion in accordance with present invention.

As used herein and in the claims, the following terms and phrases have the following meanings:

“Voice messaging platform” means a combination of hardware and/or software that provides voice messaging capability. The voice messaging platform interfaces to a telephone network, such as the Public Switched Telephone Network (PSTN), to enable subscribers to establish a call to the voice messaging platform from a telephone handset connected to the telephone network.

“E-mail messaging platform” means a combination of hardware and/or software that provides electronic mail messaging capabilities.

“Telephony session” means the time from when a call is received on a voice messaging platform to the time the call is terminated by the caller.

“E-mail session” means the collection of interactions with an e-mail messaging platform that occur during and that correspond to a particular telephony session.

“Call flow” means any program, procedure, script, state machine, or other series of instructions that define how a call is processed on a voice messaging platform during a telephony session.

Referring now to the drawings wherein like numerals indicate like elements throughout, there is shown in FIG. 1 a block diagram of an exemplary messaging system 100 in accordance with one embodiment of the present invention. Messaging system 100 comprises host computer 110 electronically coupled to network interface unit (NIU) 115 . In turn, NIU 115 is electronically coupled to public switched telephone network (PSTN) 180 that supports telephone-based subscribers 190 . As shown, host computer 110 comprises a messaging platform 135 that executes network applications 120 and 125 , respectively. In addition, messaging platform may allow network applications 120 and 125 to cooperate with message store 130 . Message store 130 may be used by messaging system 100 to store data for telephone-based subscribers 190 . In the preferred embodiment, messaging platform 135 comprises the aforementioned Network Applications Platform (NAP) commercially available from Unisys Corporation, and the host computer 110 on which the NAP runs comprises a selected Unisys A Series or ClearPath HMP NX computer system.

The NIU 115 contains a series of interfaces, interface 1 (INT 1 ), interface 2 (INT 2 ), and interface 3 (INT 3 ). One interface, such as INT 1 , interfaces the NTU 115 to the messaging platform 135 on the host computer 110 . Communication between INT 1 and messaging platform 115 is via a Small Computer Systems Interface (SCSI) bus 140 . Others of the interfaces, such as INT 2 and INT 3 (of which there can be many), interface NIU 115 to PSTN 180 . Preferably, interfaces such as INT 1 , INT 2 , and INT 3 are implemented on printed circuit boards housed within the NIU that can communicate with each other via a common bus 170 . In a preferred embodiment, bus 170 implements the Multibus II (IEEE 1296) open bus standard.

According to the present invention, the NTU further comprises an embedded services platform (ESP) 150 . In the preferred embodiment, the ESP 150 comprises a single board computer having its own processor, memory, and operating system executing on the processor, which together provide a general purpose computing capability within the NIU 115 for executing software applications that are otherwise incapable of executing within the NIU. Additionally, the ESP 150 further comprises a network interface for connection of the NIU 115 to other communications networks external to the messaging system, such as communications network 160 . In the preferred embodiment, the NIU 115 comprises a modified Telephony Services Processor (TSP) commercially available from Unisys Corporation. Additional details concerning this NIU implementation are provided in co-pending, commonly assigned, patent application Ser. No. 09/636,656, filed herewith, entitled “Network Interface Unit Having an Embedded Services Processor”, which is hereby incorporated by reference in its entirety.

As described more fully in the co-pending application, interfaces INT 2 , INT 3 are each implemented in the TSP by a Primary Rate Interface Module (PRIM), of which there can be many in any given TSP. Interface INT 1 is implemented by a PDP Card. Each PRIM interfaces a set of (e.g., 24 or 32 ) telephone circuits to the PDP card. In addition, one PRIM can be dedicated to signaling. The ESP 150 preferably comprises an EWSIII SBCP5200 single board computer commercially available from RadiSys Corporation. This board includes an Intel Pentium processor and executes the Microsoft Windows NT 4.0 operating system. A pair of 10/100 BaseT Ethernet controllers provide the network interface for connecting to and communicating across communications network 160 . This enables the ESP 150 to support connection to 100 Megabit Ethernet communications networks. The ESP 150 connects to the bus 170 within the NIU 115 , enabling it to communicate with the other interface boards INT 1 , INT 2 , INT 3 , etc. within the NIU 115 . Thus, the ESP 150 provides a general purpose computing capability within NIU 115 , and more particularly, provides such capability at the point in the architecture of messaging system 100 where the messaging platform 135 interfaces to the telephone network 180 .

Further according to the present invention, a media resource server MRS 165 is connected to the messaging system via communication network 160 and the network interface of the ESP 150 of the NIU 115 . Software executing on the host computer 110 , the ESP 150 , and the MRS 165 , enables the ESP 150 to offload selected multi-media processing functions to the MRS 165 , thereby eliminating the need for such processing functions to be performed by the voice messaging platform. For example, in the present embodiment, the ESP 150 is capable of offloading multi-media functions including but not limited to Text-to-Speech (TTS) conversion, Automatic Speech Recognition (ASR), and Natural Language Understanding (NLU). Any number of MRS units 165 may be attached to an ESP 150 up to the throughput capacity of the network connection 160 . Further, multiple ESP's 150 (not shown) may reside in the NIU 115 to further expand the multi-media processing capacity of the present invention or to provide redundancy.

FIG. 2 is a block diagram illustrating one exemplary software architecture for offloading a multi-media processing function to the MRS 165 and MRS 167 of the messaging system 100 of FIG. 1 . In this example, an automated speech recognition function is being offloaded to the MRS 165 . Also in this example, the messaging platform 135 comprises the Unisys NAP. Two subcomponents of the NAP architecture are shown, a voice network management system component (VNMS) and an application interface module component (AIM). The VNMS component is the host-based software of the NAP that routes incoming calls to the appropriate applications (e.g., application 125 ), routes outgoing calls to the appropriate ports, manages the recording and transmission of voice messages, and tracks information about each call. The AIM component provides the interface between the VNMS and the network application 125 . The AIM translates high-level application commands from the application 125 to functions specific to the VNMS. The AIM reports results of command execution by the VNMS back to the application by way of a response. Commands and responses issued by the AIM carry with them unique dialog IDs assigned by the AIM that associate them with a particular subscriber telephony session on the platform (i.e., a particular call received from a subscriber via the telephone network).

Software on the ESP 150 of the TSP 115 may comprise a Multibus driver that allows communication between the ESP 150 and the PRIM 162 and PDP 145 interfaces of the TSP 115 , an Adjunct Services Manager (ASM) that cooperates with the Multibus driver and an external communications stack comprised of a COM/DCOM layer and a TCP/IP protocol layer communication stack. The communications stack (COM/DCOM and TCP/IP) handles communications between the ASM 230 of the ESP 150 and cooperating components on the MRS 165 via the network 160 using instructions 225 and response tokens 220 .

In the present embodiment, the MRS 165 is an Intel-based server running the Microsoft Windows NT operating system. As shown in FIG. 2 , the MRS 165 has a corresponding communication stack (COM/DCOM and TCP/IP) similar to that found in the NIU 115 for communicating with the ESP 150 via the network 160 . Communication stack comprises a LAN Network Interface Card (NIC) 205 supporting TCP/IP stack 210 and COM/DCOM libraries 215 . In addition, the ASR functionality of the MRS 165 is implemented by Automatic Speech Recognition (ASR) Resources 233 having ASR Manager 235 and one or more Integrated Language Objects 240 and 245 , each of which further comprises an ASR Object 250 and an NL Interpreter 255 . Each ASR Object 250 represents a particular speech recognizer from a particular vendor for a particular language. One example of a suitable ASR engine that can be used in the present embodiment is the ASR 1500 available from Lernout & Hauspie, Burlington, Mass. Thus, in a multi-language application, there may be a number of ASR Objects on the MRS 165 . This architecture allows different speech recognizers from different vendors to be used, as desired, without having to change the underlying software architecture of the MRS 165 and ESP 150 . The functions and operations of the NL Interpreter are better described in U.S. Pat. No. 6,094,635, entitled “System and Method For Speech Enabled Application”, which is hereby incorporated by reference in its entirety.

The ASR Manager 235 on the MRS 165 is responsible for managing all interactions between the ASM 235 on the ESP 150 and any of the ASR Objects (e.g. 250 ). When multiple MRS servers (e.g. 165 and 167 ) are attached to the network 160 , the ASM 230 of the ESP 150 may choose the least busy ASR Manager 235 among all of the cooperating MRS servers (e.g. 165 and 167 ) to process a given request for speech recognition processing. The ASM 235 then routes recognition parameters, audio data, and recognition results to and from the ASR Manager 235 selected to handle the particular request. In turn, the ASR Manager 235 selects the appropriate ASR Object (speech recognizer) 250 to process the request.

In greater detail, at system startup, the ASR Manager 235 on the MRS 165 automatically initializes as a Microsoft Windows NT COM service. After initialization, the ASR Manager creates the array of Integrated Language Objects (ILOs) 240 —the number of these objects may be specified in the initialization. As mentioned, each ILO 240 may contain one instance of speech recognizer (in the form of an ASR object 250 ) and one instance of a Natural Language Interpreter 255 . The ASR Manager then calls an initialize function on each ILO 240 .

Once the ASR Manager 235 creates the ILOs (e.g. 240 and 245 ), it keeps track of how many ILOs are available for each application, application version, language, and recognizer vendor. Each turn of the dialog with a caller is handled as a separate speech recognition session. A request from the NAP 135 for speech recognition will specify the grammar to use for a given speech recognition session, along with all parameters and data needed to carry out the ASR processing. In turn, the ASR Manager 235 will initiate an object that serves as a FIFO cache for receiving data to be processed. When audio data arrives, the ASR manager 235 feeds the data into the cache object. The ASR Manager 235 then determines which ILO (i.e. ILO 240 or ILO 245 ) is available to process the data and then connects the cache object to the ILO. If no ILO is available, the cache object is queued to wait for resources when they become available. The ASR Manager 235 then coordinates responses to the ASM 230 on the ESP 150 regarding the behavior of the ILO and/or cache objects. When the ASR Manager 235 gets a result from the ILO 240 , it formats the result into a NAP message and returns the result in a response to the ASM 230 . If a request to cancel a particular ASR request is received, the ASR Manager 235 closes the cache object and notifies the ILO 240 to abort further processing.

Further, as shown in FIG. 2 , messaging system 100 may support multiple MRS such as MRS 167 . Similar to MRS 165 , MRS 167 is capable of operating various media processing applications. In the example shown, MRS 167 supports Text-To-Speech (TTS) resource 257 having a TTS Manager 260 that cooperates with one or more respective TTS Call Objects 265 , TTS Engines 270 , and Audio Destination Objects 275 . In a preferred embodiment, the TTS engines are preferably Microsoft Speech API (SAPI) compliant, which facilitates interchangeability. One example of such a TTS engine is the Speech Cube available from Elan Informatique, a Lernout & Hauspie company.

FIG. 3 is a block diagram illustrating an expanded software architecture for the ESP 150 and MRS 165 in which additional media resource processing is provided on the MRS 165 in addition to the ASR processing described in FIG. 3 . Specifically, as shown, MRS 165 have co-located various media processing resources, such as, text-to-speech conversion (TTS) resource 257 . Other resources may also be available. Similar to the ASR resource 233 having an ASR Manager, the TTS resource 257 includes a TTS Manager 260 . The TTS Manager 260 is responsible for assigning a request for TTS processing to one or more TTS engines 270 on the MRS. The TTS Manager 260 functions in much the same manager as the ASR Manager 235 , handling requests for TTS processing and managing the communication of requests and responses between the TTS engines 270 and the requesting ESP 150 .

To manage requests for different types of multi-media processing (e.g., TTS or ASR), the software on the ESP 150 includes an Adjunct Services component. Upon initialization, the Adjunct Services component initializes communications with an adjunct application program interface (Adjunct API) (which is implemented in the form of a Windows NT DLL) that provides a communication interface between the messaging system 135 and the adjunct services or resources on the MRS 165 , such as TTS and ASR. The Adjunct Services component serves to route specific classes of media processing requests (e.g., TTS, ASR, etc.) from the Multibus driver to the appropriate media resource manager on the MRS. The Adjunct Services component also provides a means to log information to the NAP for logging and tracing functions.

In operation, the ESP 150 of a NIU 115 will communicate a variety of instructions to the MRS to realize TTS or ASR processing. Included in these instructions are initialization parameters and valuation to initialize a processing session between the NIU 115 and the MRS 165 and 167 . This initialization process includes the communication of data indicative of the text or voice to be processed by the MRS 165 . Additionally, the NIU 115 communicates a request instruction and data to indicate to the MRS 165 to process the attached data. In the case of TTS processing, the NIU 115 sends initialization parameters and submits requests the MRS 167 to process chunks of data until all of the data has been processed (i.e. has been played back to the subscriber through the telephone network interface of the NIU). Comparatively, when processing ASR, the NIU 115 initializes the MRS 165 to process voice and the MRS 165 submits requests for chunks of voice data for processing until the voice data is deciphered and correlated to a universal messaging application function or all of the voice data has been processed. In the case the voice data is deciphered the resulting function is passed by the MRS 165 to the universal messaging application running on the host computer of the universal messaging system through the ESP 150 of the NIU 115 for execution.

FIG. 4 is a block diagram of an exemplary universal messaging system 200 in which the voice messaging system of FIG. 1 is modified and integrated with an e-mail messaging platform 207 . The voice messaging system in FIG. 4 is denoted generally at 105 . The universal messaging system 200 is described herein for the purpose of further illustrating the present invention, in particular, the ability to perform text-to-speech processing in such a system.

Referring to the FIG. 4 , the universal messaging system 400 comprises a voice messaging platform 105 integrated with an electronic mail messaging platform 407 . A network interface unit (NIU) 115 provides a physical connection between the voice messaging platform 105 and the Public Switched Telephone Network (PSTN) 180 . One or more servers 435 , on which a Handset E-mail Access Service (HEAS) software component 430 executes, assists in the integration between the voice messaging platform 105 and the e-mail messaging platform 407 . The HEAS 430 provides access to a subscriber's e-mails managed by an e-mail server 440 of the e-mail messaging platform 407 . The voice messaging platform 105 uses the HEAS 430 to provide telephone handset access to e-mails. In the present embodiment, the e-mail server 440 implements the POP3 electronic mail messaging protocol, and the HEAS 430 uses the POP3 protocol to access a subscriber's e-mails, as described more fully below. Additional details concerning the exemplary universal messaging system 400 are provided in co-pending, commonly assigned, patent application Ser. No. 09/636,668, filed herewith, entitled “Coordination of a Telephony Handset Session with an E-mail Session in a Universal Messaging System”.

In the present embodiment, the voice messaging platform comprises the Unisys Network Applications Platform (NAP) 401 . As mentioned above, the NAP 401 provides data and voice processing capabilities through network applications running on a host computer system 110 . In the exemplary system 400 , a Universal Messaging application 413 (hereinafter “Universal Messaging Application” or “UMsg Application”) executes on the NAP 401 and provides universal messaging services to subscribers of the voice messaging system 105 .

A network application, like the Universal Messaging Application 413 , comprises one or more call flows that determine the functionality of the application. A call flow defines how a call is processed by the application from the time it is connected until it is disconnected. It determines how a network application will react to various telephony-related events. In particular, a call flow specifies each function or processing step to be executed during a telephone call, the possible results for each function, the decision path to be followed based on the result, and each prompt or voice message to be played during the course of the telephone call.

A call flow is comprised of a sequence of call states. As used herein, the phrase “to play a prompt” and variations thereof means to output the digitized voice of a prompt over a telephone connection or the like so that it can be heard by a caller. A call state is one state, or point, in a call flow. At any given time, each call coming into a network application has a finite call state. At a given call state, a given function is performed and the results of the function point either to a next state, or to an exit state. The function can lead to one or many results.

On the NAP 401 , a call flow can execute two kinds of functions—core functions and custom functions. Core functions are pre-coded functions provided with the NAP 401 . These functions provide basic telephony services such as initiating a call, playing a voice prompt, collecting digits, and the like. Custom functions are application-specific functions that the application developer creates to perform functions not available using the core functions. For example, the application may require the retrieval of information from an application-specific database. Custom functions can be written in any programming language that the underlying computer system 22 (e.g. UNISYS A Series or ClearPath HMP NX) on which the NAP 401 is implemented can execute, such as, for example, LINC, ALGOL, COBOL, C or the like.

A network application executes in a runtime environment on the NAP 401 . Multiple network applications can be installed and run on a single NAP 401 , with each network application running in its own runtime environment. The runtime environment for a given network application includes a core function library that contains the executable code for each core function, and a custom function library that contains the executable code for each custom function written for the network application. Another library may be provided for custom functions written in the LINC programming language.

A primary component of the runtime environment is called the Agent. The Agent is a software component that (i) performs and controls call flow execution (the next function to be executed is based on the result of the previous function), (ii) passes parameters to various functions, (iii) translates new dialog activity into call states, (iv) saves information from NAP responses for use during call flow execution, and (v) enters call flow information into memory-based state tables. Execution of a call flow by the Agent is interpretive; the Agent interprets a representation of the call flow contained in a CALLFLOWS file. At each call state, the Agent invokes the specified core function or custom function from the appropriate library.

Referring still to FIG. 4 , the UMsg Application 413 executing on the NAP 401 comprises a set of Main Telephony User Interface (TUI) Call Flows 405 and a second set of E-mail Access Call Flows 410 . Hereinafter, the terms “Main TUI Call Flows 405 ” and “E-mail Access Call Flows 410 ” may be used in the singular, when referring to one call flow of the respective set, or in the plural, when referring to the collection of call flows of the respective set, as the case may require. When the Universal Messaging Application 413 is installed on the NAP 401 , the NAP 401 initiates an Agent 415 for the UMsg Application 413 .

A Universal Messaging Client Services library software component (hereinafter “UMsgCS Library”) 425 , which comprises a set of executable software routines that execute on the host computer 115 , interfaces the host computer system 110 and NAP 401 to the HEAS 430 . The UMsgCS Library 425 communicates with the HEAS 430 using a sockets protocol and is responsible for socket management. The UMsgCS Library 425 also performs data translation between the host computer system 110 and the NT server 435 .

A Call Flow Library 420 is used to create client-server relationships among the sets of call flows 405 and 410 of the UMsg Application 413 and the UMsgCS Library 425 . The UMsgCS Library 425 serves as a server to the set of Email Access Call Flows 410 of the UMsg Application 413 . The E-mail Access Call Flows 410 , in addition to serving as a client of the UMsgCS Library 425 , also serve as a server to the Main TUI Call Flows 405 of the UMsg Application 413 . Thus, the E-mail Access Call Flows 410 serve a dual role as both a client and a server.

As described in co-pending application Ser. No. 09/636,668, the UMsg Application 413 , the UMsgCS Library 425 , and the HEAS 430 , work together to coordinate a telephony session (i.e., a call from a subscriber's telephone handset 190 into the NAP 401 ) with a corresponding e-mail session on the e-mail server 440 that is established in response to a request by the subscriber for access from the telephone handset 190 to the subscriber's e-mail inbox on the server 440 . Through the use of various state tables, these components maintain an association between the telephony session and the e-mail session. As described more fully in the co-pending application, the Main TUI Call Flow 405 of the UMsg Application 413 will, on behalf of the subscriber, request via the UMsgCS Library 425 that the HEAS 430 logon to the subscriber's account on the e-mail server 440 and obtain information concerning the number of e-mail messages in the subscriber's e-mail inbox, as well as the text of at least the header fields of each e-mail message (To:, From:, Subject:, etc.). The UMsg Application 413 maintains a table for each telephony session that stores in appropriate fields of the table, the IP Address of the server 435 that hosts the HEAS 430 that is managing the corresponding e-mail session for the subscriber, a HEAS Session Index that is used by the HEAS 430 to identify the subscriber's session with the e-mail server 440 , and an e-mail ID that identifies a particular e-mail message in the subscriber's inbox on the e-mail server 440 . As described hereinafter, this information is used whenever a subscriber subsequently chooses to have the body of that particular e-mail message converted from text-to-speech and played back to the subscriber over the telephone handset 190 .

Text-to-speech processing of the body text of an e-mail is illustrated in the flow diagram of FIG. 5 . As shown, processing begins at block 500 and proceeds to block 510 where a check is performed to determined if a subscriber has made a request to have the text of a particular e-mail message converted to speech by, for example, pressing a particular key on the telephone handset 190 that corresponds to a menu choice for such a feature (e.g., “Press 4 to hear the text of the current e-mail message”). If a request has not been made, processing reverts to block 500 . However, if the alternative proves to be true, processing proceeds to block 520 where the DTMF digit for that selection is passed from the PRIM 162 of the NIU 115 to the PDP 145 and on to the NAP 201 . The NAP 201 passes the received data at block 530 to the UMsg Application, which recognizes the selection as a request to have the text of the body of the current e-mail message converted to speech and then played back to the subscriber. In response to the request, the UMsg Application 213 sends a corresponding request via the NAP 201 to the ESP 150 at block 540 . As part of the request, the UMsg Application 213 passes the IP Address of the HEAS server 235 , the HEAS Session Index for the subscriber's e-mail session on the HEAS 230 , and the e-mail ID of the message to be converted from text-to-speech. The ESP 150 then forwards a corresponding request to the TTS Manager on the MRS 165 at block 550 . At block 560 , the TTS Manager on the MRS 165 then makes a request to the HEAS 230 , via network 160 , to obtain the text of the requested e-mail. The HEAS 230 returns the requested text to the TTS Manager on the MRS 165 at block 570 . The TTS Manager on the MRS 165 then invokes the services of a TTS engine on the MRS 165 to perform the requested conversion at block 580 . The resulting voice data is then transmitted to the ESP 150 as a response to the request at block 585 . The ESP 150 then causes the voice data to be transmitted to the requesting subscriber via PDP 145 , then to PRIM 162 at block 590 . The subscriber can now hear a reading of the text of the message on the telephone handset 190 . Processing then terminates at block 595 . Note that the TTS processing of the e-mail text and the playback of the resulting voice data to the subscriber are completely offloaded from the NAP 201 , resulting in improved processing efficiency and faster response times for the subscriber. In addition, this architecture allows commercially available media resource engines (e.g. TTS, ASR, NLU) to be used without any accommodation for the native TSP environment.

In a preferred embodiment, rather than having the entire body text of an e-mail message converted form text to speech and played back to the subscriber, the subscriber may only request that the short text segments in the header of the message (e.g., To:, From:, Subject:, etc.) be played back. In this case, the UMsg Application 213 may have cached those portions of a message (but not the body text). In response to a request to have these shorter text segments converted to speech, the UMsg Application 213 may forward the text to the ESP 150 with the request, thereby eliminating the requirement of the MRS to request those segments from the HEAS 230 . In this embodiment, therefore, the MRS 165 need only access the HEAS 230 when a request for playback of the body text of a message is made.

At least portions of the present invention may be embodied in the form of program code (i.e., instructions) stored on a computer-readable medium, such as a magnetic, electrical, or optical storage medium, including without limitation a floppy diskette, CD-ROM, CD-RW, DVD-ROM, DVD-RAM, magnetic tape, flash memory, hard disk drive, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention. The present invention may also be embodied in the form of program code that is transmitted over some transmission medium, such as over electrical wiring or cabling, through fiber optics, over a network, including the Internet or an intranet, or via any other form of transmission, wherein, when the program code is received and loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention. When implemented on a general-purpose processor, the program code combines with the processor to provide a unique apparatus that operates analogously to specific logic circuits.

In sum, the present invention provides a system and methods that allow messaging systems to adapt to and efficiently perform a variety of multi-media processing functions. It is understood that changes may be made to the embodiments described above without departing from the broad inventive concepts thereof. Accordingly, the present invention is not limited to the particular embodiments disclosed, but is intended to cover all modifications that are within the spirit and scope of the invention as defined by the appended claims.