Patent US10853717B2 - Creating a conversational chat bot of a specific person
0 likes7,171 views
The patent describes methods for creating a conversational chat bot that represents a specific person. Social data about the specific person, such as images, voice recordings, social media posts, and messages, is accessed. This social data is used to create a personalized chat index reflecting the specific person's personality. The chat index is then used to train a chat bot or language model to converse in the manner and style of the specific person. During conversations, additional data sources may be accessed to respond to questions for which the social data does not provide answers.
Patent US10853717B2 - Creating a conversational chat bot of a specific person
- 1. US010853717B2 (12) United States Patent Abramson et al. (10) Patent No.: US 10,853,717 B2 (45) Date of Patent: Dec. 1, 2020 (54) CREATING A CONVERSATIONAL CHAT BOT OF A SPECIFIC PERSON 8,819,549 B2 9,514,748 B2 2002/0010584 Al 2009/0254417 A1 * 8/2014 Nageswaram et al. 12/2016 Reddy et al. 1/2002 Schultz et al. 10/2009 Beilby (71 ) Applicant: Microsoft Technology Licensing, LLC, Redmond, WA (US) GOON 3/004 706/45 2013/0257877 Al 10/2013 Davis (Continued) (72) Inventors: Dustin I Abramson, Bellevue, WA (US); Joseph Johnson, Jr., Seattle, WA (US) FOREIGN PATENT DOCUMENTS WO 2003073417 A2 9/2003 (73) Assignee: Microsoft Technology Licensing, LLC, Redmond, WA (US) OTHER PUBLICATIONS ( * ) Notice: Subject to any disclaimer, theterm ofthis patent is extended or adjusted under 35 U.S.C. 154(b) by 873 days. Wang, et al., “High Quality Lip -Sync Animation for 3d Photo Realistic Talking Head”, In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Mar. 25, 2012, pp. 4529-4532. (Continued) (21) Appl. No.: 15 /484,470 (22) Filed: Apr. 11, 2017 (65) Prior Publication Data Primary Examiner — David R Vincent (74) Attorney, Agent, or Firm Merchant & Gould P.C. US 2018/0293483 A1 Oct. 11, 2018 (57) ABSTRACT (51) Int. Cl. G06N 3/00 (2006.01) H04L 12/58 (2006.01) GO6N 20/00 (2019.01 ) (52) U.S. CI. CPC GOON3/006 (2013.01); GO6N 20/00 (2019.01); H04L 51/02 (2013.01); H04L 51/04 (2013.01); H04L 51/32 (2013.01) (58) Field of Classification Search CPC G06F 16/3329 USPC 706/15, 45 See application file for complete search history. Examples of the present disclosure describe systems and methods of creating a conversational chat bot of a specific person. In aspects, social data (e.g., images, voice data, social media posts, electronic messages, written letters, etc.) about the specific person may be accessed. The social data may be usedto create ormodify a special index inthe theme ofthe specific person's personality. The special index may be used to train a chat bot to converse in the personality of the specific person. During such conversations, one or more conversational data stores and/orAPIs may be usedto reply to user dialogue and/or questions for which the social data does not provide data. In some aspects, a 2D or 3D model of a specific person may be generated using images, depth information, and/or video data associated with the specific person. (56) References Cited U.S. PATENT DOCUMENTS 8,433,344 B1 * 4/2013 Virga G09B 29/106 455/457 8,719,200 B2 5/2014 Beilby et al. 20 Claims, 8 Drawing Sheets Start ReceiveRequestAssociatedwithSpecific 5302 Person 304 s Access Social Data for Specific Person 5 306 Create Personality Index Using Social Data 5308 Train ChatBotUsing Personality Index End 300
- 2. US 10,853,717 B2 Page 2 (56) References Cited U.S. PATENT DOCUMENTS 2014/0118140 A1 * 5/2014 Amis 2015/0365395 A1 * 12/2015 Enriquez GO8B 25/08 340/539.13 H04L 63/083 726/5 HO4L 67/2804 G06F 3/0488 G06K 9/00248 2018/0102947 A1 * 2018/0188905 A1 * 2019/0035149 A1 * 4/2018 Bhaya 7/2018 Tran 1/2019 Chen OTHER PUBLICATIONS Wolchover, Natalie, “ How the Cleverbot Computer Chats Like a Human”, http://www.livescience.com/15940-cleverbot-computer chats-human.html, Published on: Sep. 7, 211, 3 pages. "SelenaBot”,https://www.producthunt.com/posts/selenabot,Retrieved on: Feb. 27, 2017, 2 pages. Lasecki, et al., “Real-Time Conversational Crowd Assistants”, In Proceedings ofExtendedAbstracts on Human Factors in Computing Systems, Apr. 27, 2013, 6 pages. * cited by examiner
- 3. 1 CLIENTDEVICE U.S. Patent 102A SERVERDEVICE 1 106B CLIENTDEVICE Dec. 1, 2020 NETWORK104 SERVERDEVICE 102B 106A I SERVERDEVICE Sheet 1 of 8 I CLIENTDEVICE 106C 1 102C 100 US 10,853,717 B2 FIG . 1
- 4. INPUTPROCESSINGUNIT U.S. Patent USERINTERFACE 202 DATA STORE ( S ) 204 Dec. 1, 2020 INDEXENGINE 206 CHAT BOTENGINE Sheet 2 of 8 208 200 US 10,853,717 B2 FIG . 2
- 5. U.S. Patent Dec. 1, 2020 Sheet 3 of 8 US 10,853,717 B2 Start 302 ReceiveRequestAssociatedwithSpecific 53 Person 304 Access Social Data for Specific Person 306 Create Personality Index Using Social Data 308 s Train ChatBot Using Personality Index End 300 FIG. 3
- 6. U.S. Patent Dec. 1, 2020 Sheet 4 of 8 US 10,853,717 B2 COMPUTING DEVICE SYSTEM MEMORY OPERATING SYSTEM 405 REMOVABLE STORAGE PROGRAM MODULES 409 APPLICATION INSTRUCTIONS/ DATA NON-REMOVABLE STORAGE 410 INPUT DEVICE(S) PROCESSING UNIT 412 OUTPUT DEVICE(S) 414 402 COMMUNICATION CONNECTIONS 416 420 406 404 408! 400 FIG. 4 OTHER COMPUTING DEVICES 450
- 7. U.S. Patent Dec. 1, 2020 Sheet 5 of 8 US 10,853,717 B2 530 500 525 o 520 515 505 o ?Q 510 510 535 FIG. 5A
- 8. U.S. Patent Dec. 1, 2020 Sheet 6 of 8 US 10,853,717 B2 502 S 561 Special-Purpose Processor 562 560 Memory Processor 566 Apps 505 Display 564 OS 530 Peripheral Device Port 567 Storage 535 Keypad 550 Power Supply Video Interface Audio Interface LED Radio Interface Layer 556 554 520 552 FIG. 5B
- 9. U.S. Patent Dec. 1, 2020 Sheet 7 of 8 US 10,853,717 B2 GENERAL COMPUTING DEVICE TABLET COMPUTING DEVICE MOBILE COMPUTING DEVICE CHAT BOT CREATION APPLICATION 621 CHAT BOT CREATION APPLICATION 621 CHAT BOT CREATION APPLICATION 621 604 606 608 NETWORK 615 SERVER CHAT BOT CREATION APPLICATION 621 602 STORE 616 INSTANT MESSAGING STORES MAILBOX SERVICES SOCIAL NETWORKING SERVICES WEB PORTAL DIRECTORY SERVICES 622 624 626 628 630 FIG. 6
- 10. U.S. Patent Dec. 1, 2020 Sheet 8 of 8 US 10,853,717 B2 700 FIG. 7
- 11. 10 15 sense . 30 US 10,853,717 B2 1 2 CREATING A CONVERSATIONAL CHAT FIG. 4 is a block diagram illustrating example physical BOT OF A SPECIFIC PERSON components ofa computing device withwhich aspects ofthe disclosure may be practiced. BACKGROUND FIGS. 5A and 5B are simplified block diagrams of a 5 mobile computing device with which aspects ofthe present A chat robot (chat bot) is a conversational computer disclosure may be practiced. program that simulates human conversation using textual FIG. 6 is a simplified block diagram of a distributed and/or auditory input channels. Typically, chat bots are computing system inwhich aspects ofthe present disclosure implemented in dialogue systems and natural language may be practiced. processing systems to perform various practical tasks (e.g., FIG. 7 illustrates a tablet computing device for executing customer support, information acquisition, etc.). In such one or more aspects of the present disclosure. implementations, chat bots are trained using data conversa DETAILED DESCRIPTION tional dialogue samples from various users and user ses sions. As such, the chat bots in these implementations Various aspects ofthe disclosure are described more fully represent a generic, normalized version ofthe personalities below with reference to the accompanying drawings, which and attributes of the entire sampled user base. form a part hereof, and which show specific exemplary It is with respect to these and other general considerations aspects. However, different aspects ofthe disclosure may be that the aspects disclosed herein have been made. Also, implemented in many different forms and should not be although relatively specific problems may be discussed, it 20 construed as limited to the aspects set forth herein; rather, should be understood that the examples should not be these aspects are provided so that this disclosure will be limited to solving the specific problems identified in the thorough and complete, and will fully convey the scope of background elsewhere in this disclosure. the aspects to those skilled in the art. Aspects may be practiced as methods, systems or devices. Accordingly, SUMMARY 25 aspects may takethe form ofahardware implementation, an entirely software implementation or an implementation Examples ofthe present disclosure describe systems and combining software and hardware aspects. The following methods of creating a conversational chat bot of a specific detaileddescriptionis, therefore, notto be takenina limiting person (or specific entity). In aspects, social data (e.g., images, voice data, social media posts, electronic messages, The present disclosure provides systems and methods of written letters, etc.) relating to the specific person may be creating a conversational chat bot of a specific person (or accessed. The social data may be used to create or modify a specific entity). In aspects, social datarelatingto the specific special index in the theme ofthe specific person's person person may be accessed. In examples, the specific person ality. The special index may be used to train a chat bot to may correspond to a past or present entity (or a version converse and interact in the personality of the specific 35 thereof), such as a friend, a relative, an acquaintance, a person. During such conversations, one or more conversa celebrity, a fictional character, a historical figure, a random tional data stores and/or APIs may be used to reply to user entity, etc. The specific person may also correspond to dialogue and/or questions for which the social data does not oneself (e.g., the user creating/training the chat bot), or a provide data. In some aspects, a voice font of the specific version of oneself (e.g., oneself at a particular age or stage person may be generated using recordings and sound data 40 of life). Social data, as used herein, may refer to images, related to the specific person. In some aspects, a 2D or 3D image data, voice data, emails, text messages, dialogue model ofthe specific personmaybegeneratedusing images, data/commands, social media posts, written letters, user depth information, and/or video data associated with the profile information, behavioral data, transactional data, geo specific person. location data, and other forms of data about a specific This Summary is provided to introduce a selection of 45 person. In examples, social data may be stored by, and/or concepts in a simplified form that are further described collected from , various data sources. The social data (or below in the Detailed Description. This Summary is not portions thereof) may be used to create or modify a person intended to identify key features or essential features ofthe alized chat index in the theme of the specific person's claimed subject matter, nor is it intended to be used to limit personality. A chat index, as used herein, may refer to a the scope ofthe claimed subject matter.Additional aspects, 50 repository of conversational data. In examples, creating/ features, and/or advantages ofexamples will be set forth in modifying the personalizedchat indexmay comprise apply part in the description which follows and, in part, will be ing one or more rule sets or machine learning to the social apparent fromthe description, ormay be learnedbypractice data of a specific person. of the disclosure. In aspects, a personalized chat index may be usedto train 55 a chat bot or language understanding (LU ) model to con BRIEF DESCRIPTION OF THE DRAWINGS verse and/orinteractinthepersonality ofthe specificperson. A model, as used herein, may refer to a predictive or Non -limiting and non -exhaustive examples are described statistical language model that may be used to determine a with reference to the following figures. probability distribution over one or more word, character FIG. 1 illustrates an overview ofan example system for 60 sequences orevents, and/ortopredictaresponsevaluefrom creating a conversational chat bot of a specific person as one or more predictors. In examples, a model may be a described herein. rule-based model, amachine learningregressor, amachine FIG. 2 illustrates an example input processing unit for learning classifier, a neural network, or the like. In some creating a conversational chat bot of a specific person as aspects, conversing in the personality of a specific person described herein . 65 may include determining and/or using conversational attri FIG. 3 illustrates an example method of creating a con butes of the specific person, such as style, diction, tone, versational chat bot ofa specific person as described herein voice, intent, sentence/dialogue length and complexity,
- 12. US 10,853,717 B2 3 4 topic, and consistency. Conversing in the personality of a aspects, system 100 may provide an environment for soft specific personmay additionally include determining and/or ware components to execute, evaluate operational constraint using behavioral attributes (e.g., user interests, opinions, sets, and utilize resources or facilities ofthe system 100. In etc.) and demographic information (e.g., age, gender, edu such aspects, the environment may include, or be installed cation, profession, income level, relationship status, etc.) of 5 on, one or more processing devices. For instance, software the specific person and/or persons determined to be similar (e.g., applications, operational instructions, modules, etc.) to the specific person. In some aspects, during conversations may be run on a processing device such as a computer, with the chat bot or LU model, one or more conversational mobile device (e.g., smartphone/phone, tablet, laptop, per data stores and/orAPIs maybeusedto replyto userdialogue sonal digital assistant (PDA), etc.) and/or any other elec and/or questions for which the social data does not provide 10 tronic device. As an example of a processing device oper data. The conversational data stores may comprise, for ating environment, refer to the exemplary operating example, crowd-sourced conversational data from dia environments depicted in FIGS. 4-7. In other instances, the logues, interactions, and/or transactions collected from vari components of systems disclosed herein may be distributed ous data sources. The crowd-sourced conversational data across and executable by multiple devices. For example, may comprise labeled and/or unlabeled data, training data, 15 inputmay be entered on a client device andinformationmay and the like. In examples, various learning techniques (e.g., be processed or accessed from other devices in a network deep learning, heuristics, etc.) may be applied to the con (e.g. server devices, network appliances, other client versational data to determine the conversational data source devices, etc.). to use during a particular portion of a dialogue associated As presented, system 100 comprises client devices 102A with a chat bot. 20 C, distributed network 104, and a distributed server envi In aspects, a voice font of a specific person may be ronment comprising one or more servers, such as server generated by applying speech recognition and/or a speech devices 106A-C. One of skill in the art will appreciate that synthesis algorithm to one or more voice recordings in the the scale ofsystems such as system 100 may vary and may social data. The voice recordings may be collected from the include additional orfewercomponents thanthose described social data, one or more Internet of Things (“ IoT”) data 25 in FIG. 1. In some aspects, interfacing between components sources (such as personal digital assistants, natural language ofthe system 100 may occur remotely, for example, where understanding systems, etc.), and the like. The voice font components ofsystem 100 may be distributed across one or may be applied to the chat bot of a specific person. In more devices of a distributed network. aspects, a two-dimensional (“2D ”) image of a specific In aspects, client devices 102A-C may be configured to person may be generated by applying a facial recognition/ 30 receive input via a user interface component or other input detection algorithm to one or more photos in the social data means. Examples ofinput may include voice, visual, touch, and/or collected from one or more other data sources. The and text input. In examples, one or more portions of the 2D image may be applied to the chat bot ofa specific person input may correspond to social data associated with one or to create a more realistic, human -like chat experience. In more people/entities. Client devices 102A-C may store the some aspects, a 2D model (or a portion ofthe data used to 35 social data and/orprovide access to data sources comprising create the 2D model) of a specific person may be used to social data for the one or more people/entities. The data generate a three-dimensional (“3D ”) model of the specific sources may be located on, or accessible to, server devices person. Forexample, one ormore images, depth information 106A-C via network 104. As an example, client devices and/orcolorinformationmaybeprovidedas input(s)to a 3D 102A-C may provide access to social media data, user modelling algorithm . The 3D modelling algorithm may 40 profile data, and image data for one ormore people/entities. generate a 3D model and facilitate the application ofthe 3D Such data may be locally stored on client devices 102A-C, model to the chat bot of a specific person. The 3D model or on one or more of server devices 106A-C. In some may provide for a more immersive and interactive experi aspects, client devices 102A-C may have access to a per ence (e.g., in a virtual reality, augmented reality, or mixed sonalityindex (oraninstancethereof).Thepersonalityindex reality context) for a user interacting with the chat bot. 45 may be a generic personality index or a personalized per Accordingly, thepresent disclosureprovides aplurality of sonality index. A generic personality index, as used herein, technical benefits including but not limited to: creating a may comprise social data corresponding to a set oftraining conversational experience with a specific person; analyzing data, a generic user, or a multitude of anonymous users. A social data of a specific person to create a personalized personalized personality index, as used hereon, may com personality index; supplementing a personalizedpersonality 50 prise social dataforoneormorepeople/entities,one ormore index using data from a conversational repository; using a algorithms for processing social data and events (e.g., tex personalized personality index totrain a chat bot; creatingan tual data, handwritten data, images, voice data, historical accurate voice font using recordings of a specific person; events, etc.), and processed data (e.g., dialogue slots and applying avoice font ofa specific personto a chatbot ofthe corresponding data, event and dialogue hypotheses, time specific person; generating a3D model ofa specific person; 55 period information, image tags and descriptions, voice font and applying a 3D of a specific person to a chat bot ofthe data, 2D/3D information, etc.). Client devices 102A-C may specific person, among other examples. configurea generic personality indexby applying social data FIG. 1 illustrates an overview ofan example system for to the genericpersonality index.Forexample, social data for creating a conversational chat bot of a specific person as a specific person may be applied to a generic personality described herein. Example system 100 may be a combina- 60 index, thereby creating a personalizedpersonalityindex for tion of interdependent components that interact to form an the specific person. In some aspects, the personalized per integrated whole for performing task management. In sonality index may change or evolve overtime as social data aspects, system 100 may include hardware components and similar information is altered (e.g., added, modified, or (e.g., usedto execute/run an operating system (OS)), and/or removed) in the personalized personality index. software components (e.g., applications, application pro- 65 In aspects, client devices 102A-C may provide a person gramming interfaces (APIs), modules, virtual machines, ality index (or portions thereof) and/or a personalized per runtime libraries, etc.) running on hardware. In particular sonality index (or portions thereof) to a chat bot or LU
- 13. US 10,853,717 B2 5 6 model. The chat bot/LU model may be located locally, on a ing unit 200 may comprise the techniques and input server device, or some combination thereof. The chat bot/ described in FIG. 1. In alternative examples, a single system LUmodel may use the personality index as input to train the (comprising one or more components such as processor chat bot to interact in accordance with one or more person and/or memory) may perform the methods and processes alities in the personality index. For example, client devices 5 described in systems 100 and 200, respectively. 102A-C may provide a personalized personality index to a With respect to FIG. 2, input processing unit 200 may chat bot. The chat bot may be trained using the personalized comprise user interface 202, data store(s) 204, index engine personality index to interact conversationally in the person 206, and chat bot engine 208. Interface 202 may be config ality ofthe specific person associated with the personalized uredto receive, store and provide access to content, such as personality index. An instance ofthe trained, personalized 10 social data for one or more people or entities. In aspects, chat bot may be transmitted to one or more client devices interface 202 may access various data sources comprising and/or server devices. In some aspects, client devices social data relating to one or more people or entities. Such 102A-C may have access to a one or more chat indexes. A data sources may include social media websites, search chat index, as used herein, may refer to a repository of engines, content/resource providers, user profiles/accounts, conversational data comprising social data and/or conver- 15 image/photo repositories, voice recordings,musical record sational algorithms associated with a plurality of users, ings, print sources (e.g., books, magazines, newspapers, events and conversational scenarios. As an example, a chat etc.), handwritten letters, narrative accounts ofevents, a chat index may comprise question and answer information from index ofconversational data, labeled/unlabeledtraining data a specific person, question and answer information from a sets, etc. Interface 202 may collect social data from one or person or entity determined to be similar to the specific 20 more data sources inresponse to a query associatedwith one person, crowd-based question and answer information from ormore specific people. The collecteddatamaybe storedby a group ofusers or a portion ofan overall community (e.g., a data store accessible to interface 202, such as data store(s) crowd-sourced data), general information related to a spe 204. Data store(s) 204 may be configured to store and/or cificperson, generic informationrelatingto aparticulartopic organize data according to various criteria. For instance, or time period (but unrelated to a specific person), scripted 25 data store(s) 204 may store social data by a user identifica and/or pre-generated automated questions/replies, labeled tion of a specific person associated with the social data, data, voice data, image data, etc. In examples, the client date/time, social data subject/topic, social data type, or the devices 102A-C may use a chat index to supplement infor like. mational gaps and/ordiscrepancies inthe chatbot/LUmodel Index engine 206 may be configured to create a person knowledge base. For instance, client devices 102A-C may 30 ality index. In aspects, index engine 206 may receive a enable a chat bot to directly access or query a chat index (or request to generate a personality index. The request may be an associated service) to determine an answer or an appro associated with one or more specific people or entities. In priate response for a specific person personified by the chat examples, a request may be transmittedto index engine 206 bot. via interface 202, or received directly via an interface In aspects, client devices 102A-C may provide for creat- 35 component accessible by a client or client device. In ing and/or applying a voice font to a chat bot. For example, response to receiving the request, index engine 206 may client devices 102A-C may access voice data (e.g., voice access social data collectedby interface 202 and/or storedby recordings, musical recordings, etc.) comprised in social data store(s) 204. Index engine 206 may search for and data, a personality index or other data sources. Speech collect social data associated with the one or more specific recognition and/or speech synthesis techniques may be 40 people or entities identified in the request. The social data applied to the voice data to create a voice font ofa specific associated with the one or more specific people or entities person. The models and/or algorithms for implementing ("personalized data") may be combined with a personality suchtechniques may be providedby client devices 102A-C, index (or a generic personality index) and processed to server devices 106A-C, or a separate device/service. The facilitate the creation of a personalized personality index voice font may then be applied to a chat bot to enable the 45 (e.g., a personality index corresponding to the personalized chat bot to converse in the voice of a specific person. In data for the specific person/entity). In some aspects, pro some aspects, clientdevices 102A-Cmayfurtherprovide for cessing the personalized data may comprise identifying and creating and/or applying a 2D or 3D model of a specific categorizing conversation data (e.g., explicit question/an person to a chat bot. For example, client devices 102A - C swer data, inferred question/answer data, historical event may access image data to create a 2D model ofthe specific 50 facts and hypotheses, etc.) collected for a specific person/ person. Additionally or alternatively, client devices 102A-C entity. may access image data and/or3D data (e.g., photos, images, Processing the personalized data may further comprise depth information, color information, mapping information, determining and categorizing conversation data associated etc.) comprised in social data, a personality index or other with people/entities similar to the specific person/entity datasources.Theimagedata and/or 3D datamay beapplied 55 identified in the request. In examples, determining similari to a 3D modelling algorithm or serviceto create a 3D model ties between a specific person/entity and another person/ ofapersonorentity.Alternately, client devices 102A-Cmay entity (e.g., the " other person") may include using machine access a 3D modelling device. The 3D modelling device learned techniques and/ornatural language processing tech may be configured to perform a 3D scan of a person or niques to analyze and compare the social data ofthe other entity, and/or access one or more previous 3D scans. In 60 person. Such an analysis/comparison may include the use of examples, a 2D or 3D model may be applied to a chat bot latent semantic indexing, latent Dirichlet processing, word to enable immersive interactions with the likeness of a and/or sentence embedding models, collaborative filtering specific person/entity. techniques, entitygraphs, Jaccard similarity, cosine similar FIG. 2 illustrates an overview of an exemplary input ity and/or translation models. Such an analysis/comparison processing unit 200 for creating a conversational chat bot of 65 may further include the use of approval indicators (e.g., a specific person, as described herein. The conversational “ likes”(“dislikes,” display screen swipes, ratings, reviews, chat bot creation techniques implemented by input process comments, watch lists, etc.) for social media data, music
- 14. US 10,853,717 B2 7 8 data, image data, etc. In at least one example, the analysis topics. Forexample, ifthe specific person is/was not a public may include comparing one or more characteristics (e.g., figure or otherwise well-known, there may be little or no traits, attributes, events, etc.) of the specific person /entity publically-available information regarding the specific per with the other person. Such characteristics may include son. Ageneric chat index, however, may comprise a generic demographic data (e.g., age, gender, income, employment, 5 or pre-scripted answer to a topic or subject identified in a education, time period of lifetime, etc.), behavioral data corresponding dialoguerequest.Alternately, thegeneric chat (e.g., access dates/times, transaction trends, purchase his index may comprise logic for composing one or more tory, frequented sites, dwell times, click data, etc.), stylistic questions directed to soliciting information from a user or content of data (e.g., style, diction, tone, voice, intent, the specific person/entity. Information obtained as a result of sentence/dialogue length and complexity, etc.), psycho- 10 posing the one or more questions to users or the specific graphic data (e.g., user interests, opinions, likes/dislikes, person /entity may be provided to one or more chat indexes values, attitudes, habits, etc.), and the like. In such an and processed accordingly. example, at least a subset of the characteristics may be Chatbot engine 208 may be configuredto generate a chat provided to a scoring or comparison algorithm /model for bot or LU model. In aspects, input processing unit 200 may evaluation.Thescoringorcomparisonalgorithm /modelmay 15 cause chatbotengine 208 to generate one ormore chatbots generate and/or assign scores or labels to the evaluated (or instances thereof). Input processing unit 200 may then characteristics. The scoring or comparison algorithm /model cause or facilitate the application ofdata from a personality mayusethe generated scores/labels to determine a similarity index to the one or more generated chat bots. In examples, score or metric for the other person. The similarity score/ applying personalized data to a chat bot may generate a metric may represent theestimated similarity between a 20 personalizedchatbot configured tointeractconversationally specific person/entity andtheotherperson/entity. In aspects, in the personality of a specific person/entity. Applying the processed personalized data may be used to create, personalized data to a chat bot may also cause a voice font, organize, populate or update a personalized personality a 2D image, or a 3D model ofa specific person /entity to be index forthe specific person/entity identified in the request. applied to the chat bot. Chat bot engine 208 may be further Index engine 206 may be further configuredto access one 25 configuredto establisha set ofinteractionrules forachatbot ormore conversational data sources and/orAPIs. In aspects, or LU model. In aspects, the set of interaction rules may index engine 206 may have access to one or more data provide for determining when (and in what order) to utilize sources comprising crowd -sourced conversation data. The the data and various data sources available to index engine crowd-sourcedconversation data maybeusedto supplement 206.As an example, chat bot engine 208 may establish arule the data in a personality index. The crowd-sourced conver- 30 set dictating that, in response to receiving dialogue input, a sation data may comprise social data collected /derived from specific chat bot may attempt to provide a response using a plurality of users and relating to one or more specific data from the following data sets (in order): 1) social data people/enti events, time periods, and/or conversational from a specific person /entity, 2) social data from users scenarios. The crowd-sourced conversation data may addi similar to the specific person /entity, 3) social data from a tionally comprise conversational algorithms/models forpro- 35 global user base (such as the internet at large) that may or cessing the social data comprised in the crowd-sourced may not be similar to the specific person/entity, and 4) conversation data. In examples, the crowd-sourced conver generic, catch all phrases/questions that are not specific to sation data may be collected from various online and offline the specific person/entity. As another example, in response sources, and stored in, for example, a crowd-sourced chat to receiving dialogue input, chat bot engine 208 may provide index. The crowd-sourced chat index may include crowd- 40 the received dialogue input to a machine learning model for based perceptions, opinions and knowledge regarding the processing dialogue. The machine learning model may then actions, communications and/or events relating to one or apply decision logic to determine a hierarchal data traversal more specific people/entities, a period of time, or one or process for collecting reply data. In such aspects, chat bot more events. For example, if the specific person is/was a engine 208 may associate one or more established rule sets public figure, there may be publically-available information 45 (ormodels) with a corresponding personalized chat bot, and (e.g., Wikipedia articles, biographies/autobiographies, print/ facilitate the deployment and/or implementation ofthe chat audio/video news stories, podcasts, etc.) regarding the spe bot and rule set (ormodel) to one ormore computing device, cific person. The publically-available information (or por services or user accounts. tions thereof) maybeusedto populate a crowd-sourced chat FIG. 3 illustrates an example method of creating a con index. 50 versational chat bot ofa specific person as described herein. In aspects, index engine 206 may additionally or alter In aspects, method 300 may be executed by an exemplary nately have access to one or more data sources comprising system such as system 100 of FIG. 1. In examples, method generic conversation data. The generic conversation data 300 may be executed on a device comprising at least one may be used to address dialogue input for which a person processor configured to store and execute operations, pro ality index and a data source comprising crowd-sourced 55 grams or instructions. However, method 300 is not limited conversationdataisunabletoprovideananswerordata.The to such examples. In other examples, method 300 may be generic conversation data may comprise scripted and/or performed on an application or service for creating and/or pre-generated automatic questions/replies, generic conver implementing a conversational chat bot or LU model. In at sational and time period-based algorithms/models, and per least one example, method 300 may be executed (e.g., sonality-neutralsocialdata.Thatis, thegenericconversation 60 computer-implemented operations) by one or more compo data may not be associated (or associable) with a specific nents of a distributed network, such as a web service ! person/entity. In examples, the generic conversation data distributed network service (e.g. cloud service). may be manually or automatically generated, selected or Example method 300 begins at operation 302 where a storedin, forexample, a generic chat index. The generic chat request associated with a specific person or entity is index may include generic or theoretical opinions and 65 received. In aspects, a computing device, such as input knowledge regarding the actions, communications and/or processing unit 200, may receive arequest to generate, train events relating to one or more generic people/entities or or modify a chat bot or LU model. The request may
- 15. US 10,853,717 B2 9 10 comprise information associated with a specific person or photo data, and/or correlate the detected tags with the entity, such as a name, a nickname, an occupation, an processedphotodata. Theanalysis ofsocial datamay further associated time period (e.g., lifetime, period in office, play include evaluating voice data in (or associated with) the ing career, etc.), a description, etc. The information may be social data. Such an evaluation may include using speech used to identify one or more data sources comprising (or 5 recognition and/or speech syntheses techniques to generate potentially comprising) data relatedto the specific person or a voice font corresponding to a specific person/entity. Fur entity.At operation 304, social data for the specific person) ther still, the analysis of social data may include generating entitymay be accessed. Inaspects, one ormore queries may and/or evaluating 2D/3D data in (or associated with) the be generatedand submittedto the one ormore identifieddata social data. Such an evaluation may include using 2D/3D sources. Generating queries may comprise identifying key- 10 modelling techniques (and associated data) to generate a 2D words or terms in a request, and formulating queries based or 3D model corresponding to a specific person/entity. thereon. In response to submitting a query, one or more In some aspects, a personality index may comprise (orbe result sets comprising social data may be generated and associated with) one or more data processing algorithms or received by the computing device. In examples, the social models for processing data, such as social data and event datamay comprise information relating to one or more 15 data.As an example, the data processing algorithms/models specific people or entities. Such information may include may correspondto a set ofinteraction rules for using one or images, image data, voice data, social media posts, written more datasets associated with a specific person/entity. Such letters, user profile information, behavioral data, transac interaction rules may include criteria for accessing one or tional data, geolocation data, and other forms ofdata. As an more datasets, a preferred order of accessing one or more example, the social data for a current celebrity may include 20 datasets (e.g., first preference: social data from a specific social media posts from (and about) the celebrity, voice and person/entity; second preference: social data from users image data (e.g., recordings of interviews, performances, similar to the specific person/entity; third preference: social etc.), movies/televisions shows, electronic news and articles data from a global user base; etc.), criteria for determining about the celebrity, web chatter relating to the celebrity, etc. whether the specific person/entity is a present or historical As another example, the social data for a historical figure 25 figure, etc. As another example, the data processing algo (such as Abraham Lincoln) may include handwritten letters rithms/models may correspondto image classification rules/ and similar correspondences authored by the historical fig algorithms. Such image classification rules/algorithms may ure, books authored or about the historical figure, informa dictate the processing and analysis of image data. For tion related to the relevant time period associated with the instance, a personalized personality index may comprise an historical figure, physical media comprising audio data 30 unlabeledphoto ofa person surfing.An image classification and/or video data, photos, etc. In such aspects, the social algorithm may be applied to the photo to determine the data (or portions thereof) may be stored in a data store subject ofthe photo or an action associated with the photo accessible to the computing ce, such as data store(s) (e.g., " person surfing"). A facial recognition algorithm may 204. beappliedto thepersoninthephoto. The dataresultingfrom At operation 306, a personality index may be created 35 the facial recognition analysis may be compared to labeled using social data. In aspects, a computing device may have image data accessible to the personalized personality index. access to index-generation component, such as index engine A label may be applied to the person in the photo based on 206. The index -generation component may have access to the comparison (e.g., “John surfing"). An image analysis one ormore sources ofsocial data, suchas data store(s) 204. technology may also be applied to the photo. The image In examples, the computing device may cause the index- 40 analysis technology may analyze the metadata ofthe photo generation component to generate apersonality index (or an to identify, for example, an associated geotag. The associ instance thereof) as part ofreceiving the request received at ated geotag may then be applied to the photo (e.g., "John operation 302. The computing device may provide the surfing in Hawaii”). In such an example, additional data index-generation component information identifying a spe froma device associatedwiththe photo (e.g., the originating cific person or entity. As a result, the index-generation 45 device) may be stored in the personalized personality index component may identify and/or collect data related to the and used in the analysis ofthe photo. Such additional data identified specific person/entity from the one or more may include data from one or more sensors of the device. sources ofsocial data. The identified/collecteddatamaythen Examples ofsensors may include a GPS sensor, a proximity be processed and applied to the personality index (e.g., a sensor, an accelerometer sensor, a gyroscopic sensor, a force generic personality index ); thereby, creating a personalized 50 sensor, an acoustic sensor, a touchscreen sensor, an optical personality index in the theme ofthe specific person/entity. sensor, and a localization sensor. One of skill in the art will Forexample, amachine learningmodelmay analyzea setof appreciate that other types of sensors may also be used. As social data to identify and categorize content, content attri yet another example,the data processing algorithms/models butes, content authors/contributors, data sources, etc. Such may correspond to data acquisition rules/algorithms. Such an analysis may include categorizing the social data by type 55 data acquisition rules may provide for soliciting/acquiring (e.g., textual data, audio data, image data, etc.), determining data (e.g., in the form ofquestions to the user) from a user the source/author(s) of the social data (e.g., a specific (e.g., the specific person/entity, the user interacting withthe person/entity, one ormore otherpersons similarto a specific chat bot, etc.) or from data sources identified by a user. For person/entity, subject matter experts, random users, etc.), instance, a personalized personality index may comprise determining the degree of similarity betweena specific 60 social datarelatingto a deceased relative ofauser.Although person/entity and alternate sources/authors, identifying the social data may comprise information from the lifetime question andanswerpairs, identifying dialogue expressions, ofthe deceased relative, the social data may not comprise etc. The analysis of social data may also include evaluating information related to a time period after the lifetime ofthe photo data in (or associated with) the social data. Such an deceased relative. As a result, a set ofdata acquisition rules evaluationmay include using, for example, deep learning to 65 may be generated for (or assigned to) the personalized detect tags in (and/or attributes of) the photo data, process personality index. The set of data acquisition rules may (e.g., identify, annotate, summarize, etc.) the events in the provide instructions for acquiring data related to various
- 16. US 10,853,717 B2 11 12 time periods ofthe deceasedrelative's lifetime (e.g., before, ries. The system memory 404 may include an operating during and/or after the lifetime). Such instruction may system 405 and one or more program modules 406 suitable include asking a user questions about a time period, one or for running software application 420, such as one or more more events and/or people, or asking a user where such components supported by the systems described herein. As information may be obtained. In such an example, such 5 anexample, system memory 404 may store social data (e.g., questions may indicate the specific person represented by images, image data, voice data, emails, text messages, the personalized personality index (e.g., the deceased rela dialogue data/commands, social mediaposts, writtenletters, tive) possesses a perceived awareness that he/she is, in fact, user profile information, behavioral data, transactional data, deceased. geolocation data, etc.), personality index data and instruc At operation 308, a chat bot or LU model may be trained 10 tions for creating a conversational chat bot of a specific using a personality index (or personalized personality entity. The operating system 405, for example, may be index). In aspects, a computing device may have access to suitable for controlling the operation of the computing a conversational computer program , such as chat bot engine device 400. Furthermore, embodiments of the disclosure 208. The conversational computerprogrammayhave access may be practiced in conjunction with a graphics library, to one or more personality indexes. Inexamples, the com- 15 other operating systems, or any other application program puting device may cause the conversational computer pro and is not limited to any particular application or system. gram to generate a chat bot/LU model (or an instance This basic configuration is illustrated in FIG. 4 by those thereof) as part ofreceiving the request at operation 302. A components withinadashedline408. Thecomputingdevice personality index (or a portion ofthe data therein) may be 400 may have additional features or functionality. For provided as input to a generated chat bot/LU model to train 20 example, the computing device 400 may also include addi the chat bot/LU model. For example, a chat bot may be tional data storage devices (removable and/or non-remov trained using processed social data and one or more data able) such as, for example, magnetic disks, optical disks, or processing algorithms or rule sets. The trained chat bot/LU tape. Such additional storage is illustrated in FIG. 4 by a model may be operable to interact conversationally in the removable storage device 409 and a non-removable storage personality of a specific person/entity associated with the 25 device 410. personalized personality index. Interacting conversationally As stated above, a number ofprogram modules and data may include determining the a subject and/or intent for one files may be stored in the system memory 404. While ormore expressions ofa dialogue, identifying a data source executing on the processing unit 402, the program modules comprising response data, determining whether response 406 (e.g., application 420) may perform processes including, data is present in accessible data sources, generating and 30 but not limited to, the aspects, as described herein. Other posing questions to supplement gaps and/or verify data in program modules that may be used in accordance with the data source data, etc. In at least one example, the trained aspects ofthepresent disclosuremay include electronic mail chat bot/LU model may be additionally or alternatively and contacts applications, word processing applications, operable to provide additional functions, such as replying to spreadsheet applications, database applications, slide pre emails and social media posts, answering voice calls and 35 sentation applications, drawing or computer-aided applica providing voicemails, serving as a personal digital assistant, tion programs, etc. storingreminders ormessages, etc. In some aspects, training Furthermore, embodiments of the disclosure may be a chat bot/LU model may additionally include applying one practiced in an electrical circuit comprising discrete elec or more visual or auditory characteristics or attributes to a tronic elements, packaged or integrated electronic chips chat bot/LU model. For example, a personality index may 40 containing logicgates, a circuit utilizing a microprocessor, include (or have access to) a voice font, a 2D image and/or or on a single chip containing electronic elements ormicro a 3D model of a specific person/entity associated with the processors. For example, embodiments of the disclosure personalized personality index. The voice font, a 2D image may be practicedvia a system-on-a-chip (SOC) where each and /or a 3D model may be applied to the chat bot/LU model or many of the components illustrated in FIG. 4 may be to provide a more immersive user experience for users 45 integrated onto a single integrated circuit. Such an SOC interacting with the chat bot/LU model. device may include one or more processing units, graphics FIGS. 4-7 and the associated descriptions provide a units, communications units, system virtualization units and discussion of a variety of operating environments in which various application functionality all ofwhich are integrated aspects of the disclosure may be practiced. However, the (or “burned”) onto the chip substrate as a single integrated devices and systems illustratedanddiscussedwithrespectto 50 circuit. When operating via an SOC, the functionality, FIGS. 4-7 are for purposes ofexample and illustration and described herein, with respect to the capability of client to are not limiting of a vast number of computing device switch protocols may be operated via application -specific configurations that may be utilized for practicing aspects of logic integrated with other components of the computing the disclosure, described herein. device 400 on the single integrated circuit (chip). Embodi FIG. 4 is a block diagram illustrating physical compo- 55 ments of the disclosure may also be practiced using other nents (e.g., hardware) ofacomputingdevice400withwhich technologies capable ofperforming logical operations such aspects ofthe disclosure may be practiced. The computing as, for example, AND, OR, and NOT, including but not device components described below may be suitable for the limited to mechanical, optical, fluidic, and quantum tech computing devices described above, including the client nologies. In addition, embodiments ofthe disclosure may be computing devices 102A-C and the server computing 60 practicedwithina general purpose computer or inany other devices 106A-C. In a basic configuration, the computing circuits or systems. device 400 may include at least one processing unit 402 and The computing device 400 may also have one or more a system memory 404. Depending on the configuration and input device(s) 412 such as a keyboard, a mouse, a pen, a type of computing device, the system memory 404 may sound or voice input device, a touch or swipe input device, comprise, but is not limitedto, volatile storage (e.g., random 65 etc. The output device(s) 414 such as a display, speakers, a access memory), non -volatile storage (e.g., read-only printer, etc. may also be included. The aforementioned memory), flash memory, or any combination ofsuchmemo devices are examples and others may be used. The comput
- 17. US 10,853,717 B2 13 14 ing device 400 may include one or more communication graphical user interface (GUI), a visual indicator 520 (e.g., connections 416 allowing communications with other com a light emitting diode), and/or an audio transducer525 (e.g., puting devices 450. Examples of suitable communication a speaker). In some aspects, the mobile computing device connections 416 include, but are not limited to, radio fre 500 incorporates a vibration transducer for providing the quency (RF) transmitter, receiver, and/or transceiver cir- 5 user with tactile feedback. In yet another aspect, the mobile cuitry; universal serial bus (USB), parallel, and/or serial computing device 500 incorporates input and/or output ports. ports, such as an audio input (e.g., a microphone jack), an The term computer readable media as used herein may audio output (e.g., a headphone jack ), and a video output include computer storage media. Computer storage media (e.g., a HDMI port) for sending signals to or receiving may include volatile and nonvolatile, removable and non- 10 signals from an external device. removable media implemented in any method ortechnology FIG. 5B is a block diagram illustrating the architecture of for storage of information, such as computer readable one aspect ofa mobile computing device. That is, themobile instructions, data structures, or program modules. The sys computing device 500 can incorporate a system (e.g., an tem memory 404, theremovable storage device 409, andthe architecture) 502 to implement some aspects. In one non -removable storage device 410 are all computer storage 15 embodiment, the system 502 is implemented as a “smart media examples (e.g., memory storage). Computer storage phone" capable of running one or more applications (e.g., media may include RAM , ROM, electrically erasable read browser, e-mail, calendaring, contact managers, messaging only memory (EEPROM), flash memory or other memory clients, games, and media clients/players). In some aspects, technology, CD-ROM, digital versatile disks (DVD) orother the system 502 is integrated as a computing device, such as optical storage, magnetic cassettes,magnetic tape, magnetic 20 an integrated personal digital assistant (PDA) and wireless disk storage or other magnetic storage devices, or any other phone. article ofmanufacture which can be used to store informa One ormore applicationprograms 566may be loadedinto tion and which can be accessed by the computing device the memory 562 and run on or in association with the 400. Any such computer storage media may be part of the operatingsystem 564. Examples ofthe application programs computing device 400. Computer storage media does not 25 include phone dialer programs, e-mail programs, personal include a carrierwave orotherpropagatedormodulateddata information management (PIM) programs, word processing signal. programs, spreadsheet programs, Internet browser pro Communication media may be embodied by computer grams, messaging programs, and so forth. The system 502 readable instructions, data structures, program modules, or also includes a non-volatile storage area 567 within the otherdata in a modulated data signal, such as a carrierwave 30 memory 562. Thenon-volatile storage area 567maybeused or other transport mechanism , and includes any information to store persistent information that should not be lost ifthe delivery media. The term “modulated data signal” may system 502 is powereddown. The applicationprograms 566 describe a signal that has one or more characteristics set or may use and store information in the non -volatile storage changed in such a manner as to encode information in the area 567, such as e-mail or othermessages usedby an e-mail signal. By way of example, and not limitation, communi- 35 application, andthe like. A synchronization application (not cation media may include wired media such as a wired shown) also resides on the system 502 and is programmed network or direct-wired connection, and wireless media to interact with a corresponding synchronization application such as acoustic, radio frequency (RF), infrared, and other resident on a host computer to keep the information stored wireless media. in the non -volatile storage area 567 synchronized with FIGS. 5A and 5B illustrate a mobile computing device 40 corresponding information stored at the host computer. As 500, for example, a mobile telephone, a smart phone, should beappreciated, otherapplications may be loaded into wearable computer (such as a smart watch ), a tablet com the memory 562 and run on the mobile computing device puter, a laptop computer, and the like, with which embodi 500 described herein . ments ofthe disclosure may be practiced. In some aspects, The system 502 has a power supply 550, which may be the clientmaybe amobilecomputing device. Withreference 45 implemented as one or more batteries. The power supply to FIG. 5A, one aspectofamobile computing device 500 for 550 might further include an external power source, such as implementing the aspects is illustrated. In a basic configu anAC adapterorapowereddocking cradlethat supplements ration, the mobile computing device 500 is a handheld or recharges the batteries. computer having both input elements and output elements. The system 502 may also include a radio interface layer The mobile computing device 500 typically includes a 50 552 that performs the function oftransmitting andreceiving display 505 and one ormore inputbuttons 510 that allow the radio frequency communications. The radio interface layer user to enter information into the mobile computing device 552 facilitates wireless connectivity betweenthe system 502 500. The display 505 of the mobile computing device 500 and the " outside world,” via a communications carrier or may also function as an input device (e.g., a touch screen service provider. Transmissions to and from the radio inter display ). If included, an optional side input element 515 55 face layer 552 are conducted under control ofthe operating allows furtheruserinput. The side input element 515 may be system 564. In otherwords, communications receivedbythe a rotary switch, a button, or any other type ofmanual input radio interface layer 552 may be disseminated to the appli element. In alternative aspects, mobile computing device cationprograms 566 via the operating system 564, and vice 500 may incorporate more or less input elements. For example, the display 505 may not be a touch screen in some 60 The visual indicator 520 may be used to provide visual embodiments. In yet another alternative embodiment, the notifications, and/or an audio interface 554 may be used for mobile computing device 500 is a portable phone system, producing audiblenotifications viathe audio transducer525. such as a cellular phone. The mobile computing device 500 In the illustrated embodiment, the visual indicator 520 is a may also include an optional keypad 535. Optional keypad light emitting diode (LED) and the audio transducer 525 is 535 may be a physical keypad or a “ soft” keypad generated 65 a speaker. These devices may be directly coupled to the on the touch screen display. In various embodiments, the power supply 550 so that when activated, they remain on for output elements include the display 505 for showing a a duration dictated by the notification mechanism even versa .
- 18. US 10,853,717 B2 15 16 though the processor(s) (e.g., processor 560 and/or special In addition, the aspects and functionalities described herein purpose processor 561) and other components might shut may operate over distributed systems (e.g., cloud-based down for conserving battery power. The LED may be computing systems), where application functionality, programmed to remain on indefinitely until the user takes memory, data storage and retrieval and various processing action to indicate the powered -on status ofthe device. The 5 functions may be operated remotely from each other over a audio interface 554 is usedto provide audible signals to and distributed computing network, such as the Internet or an receive audible signals from the user. For example, in intranet. User interfaces and information of various types addition to being coupled to the audio transducer 525, the maybedisplayedviaon-boardcomputing devicedisplays or audio interface 554 may also be coupled to a microphone to via remote display units associated with one or more com receive audible input, such as to facilitate a telephone 10 puting devices. Forexample, user interfaces and information conversation. In accordance with embodiments ofthe pres ofvarious types may be displayed and interacted with on a ent disclosure, the microphone may also serve as an audio wall surface onto which user interfaces and information of sensor to facilitate control of notifications, as will be various types areprojected. Interactionwiththe multitude of described below. The system 502 may further include a computing systems with which embodiments ofthe inven video interface556thatenables anoperationofanon-board 15 tionmaybepracticedinclude, keystroke entry,touch screen camera 530 to record still images, video stream , andthe like. entry, voice or other audio entry, gesture entry where an Amobile computing device 500 implementingthe system associated computing device is equipped with detection 502 may have additional features or functionality. For (e.g., camera) functionality for capturing and interpreting example, the mobile computing device 500 may also include user gestures for controlling the functionality of the com additional data storage devices (removable and/or non- 20 puting device, and the like. removable) such as, magnetic disks, optical disks, or tape. Aspects ofthe present disclosure provide a system com Such additional storage is illustrated in FIG. 5B by the prising: at least one processor, and memory coupled to the non -volatile storage area 567. at least one processor, the memory comprising computer Data/information generated or captured by the mobile executable instructions that, when executed by the at least computing device 500 and storedvia the system 502 may be 25 one processor, performs a method for creating a conversa stored locally on the mobile computing device 500, as tional chat bot of a specific entity, the method comprising: described above, or the data may be stored on any number receiving arequest associatedwith a specific entity; access ofstorage media that may be accessed by the device via the ing social data associated with the specific entity, the social radio interface layer 552 or via a wired connection between data comprising at least one ofimages ofthe specific entity, the mobile computing device 500 and a separate computing 30 voice data for the specific entity, conversational data asso device associatedwiththemobile computing device 500, for ciated with the specific entity, and publicly available infor example, a server computer in a distributed computing mation about the specific entity; using the social data to network, such as the Internet.As shouldbe appreciated such create a personality index, wherein the personality index data/informationmay be accessedvia themobile computing comprises personality information for the specific entity; device 500 via the radio interface layer 552 or via a 35 andusing the personality index to train a chatbot to interact distributed computing network. Similarly, such data/infor conversationally using the personality ofthe specific entity. mation may be readily transferred between computing In some examples, the method further comprises using devices for storage and use according to well-known data/ information in the request to identify one or more data informationtransferand storagemeans, including electronic sources, wherein the one ormore data sources comprise the mail and collaborative data/information sharing systems. 40 social data. In some examples, the social data is further FIG. 6 illustrates one aspect of the architecture of a based on at least one of social media posts, written letters, system for processing data received at a computing system userprofile information, behavioral data,transactional data, from a remote source, such as a personal computer 604, and geolocation data. In some examples, the method further tablet computing device 606, or mobile computing device comprises: collecting the accessed social data; storing the 608, as described above. Content displayed at server device 45 accessed social data in a data store; and providing an index 602 may be stored in different communication channels or generation engine access to the stored social data. In some otherstoragetypes. Forexample, various documents may be examples, the method further comprises: processing the stored using a directory service 622, a web portal 624, a social datausing at least one ofmachine learningtechniques mailbox service 626, an instant messaging store 628, or a and one or more rule sets; and applying the processed social social networking site 630. A chat bot creation application 50 data to the personality index to generate a personalized 621 may be employed by a client that communicates with personality index. In some examples, the personality index server device 602, and/or the chat bot creation application is associated with one or more data processing algorithms 620 may be employed by server device 602. The server for processing the social data, wherein the one or more data device 602 may provide data to and from a client computing processing algorithms correspond to at least one ofchat bot device such as a personal computer 604, a tablet computing 55 interaction rules, image classification rules, and data acqui device 606 and/or a mobile computing device 608 (e.g., a sition rules. In some examples, training the chat bot com smart phone) through a network 615. By way of example, prises applying to the chat bot at least one ofa voice font of the computer system described above may be embodied in the specific entity, a 2D image ofthe specific entity, anda 3D a personal computer 604, a tablet computing device 606 image ofthe specific entity. In some examples, the method and/or amobile computing device 608 (e.g., a smartphone). 60 further comprises: submitting dialogue to the chat bot;and Any of these embodiments of the computing devices may generating, by the chat bot, a response to the submitted obtain content from the store 616, in addition to receiving dialogue, wherein generating the response comprises utiliz graphical data useable to be either pre-processed at a ing a hierarchical data traversal process to collect response graphic -originating system , or post-processed at a receiving data from one or more data sources accessible to the computing system . 65 personality index. In some examples, the hierarchical data FIG. 7 illustrates an exemplary tablet computing device traversal process comprises evaluating social data from the 700 that may execute one or more aspects disclosed herein. specific entity, evaluating social data from entities similarto
- 19. US 10,853,717 B2 17 18 the specific entity, evaluating social data from a global user storage device; and generating, by the chat bot, a response base, and evaluating generic response options. In some to the received dialogue, wherein generating the response examples, collecting response data comprises: determining, comprises utilizing a data traversal process to collect by the chat bot, the personality index does not comprise data response data from one or more data sources accessible to for addressing one or more parts ofthe submitted dialogue; 5 the personality index. composing, bythechatbot, one ormorequestionsto address Aspects ofthepresentdisclosurearedescribed abovewith the data not comprised in the personality index; and posing, reference to block diagrams and/or operational illustrations to a user interacting with the chat bot, the one or more of methods, systems, and computer program products questions. according to aspects of the disclosure. The functions/acts Aspects ofthepresentdisclosure furtherprovide amethod 10 noted in the blocks may occur out ofthe order as shown in for creating a conversational chat bot ofa specific entity, the any flowchart. For example, two blocks shown in succession method comprising: receiving a request associated with a may in fact be executed substantially concurrently or the specific entity; accessing social data associated with the blocks may sometimes be executed in the reverse order, specific entity, the social data comprising at least one of depending upon the functionality/acts involved. images of the specific entity, voice data for the specific 15 The description and illustration of one or more aspects entity, conversationaldata associatedwiththe specific entity, provided in this application are not intended to limit or and publicly available information about the specific entity; restrict the scope of the disclosure as claimed in any way. using the social data to create a personality index, wherein The aspects, examples, and details provided in this applica the personality index comprises personality information for tion are considered sufficient to convey possession and the specific entity; and using the personality index to train a 20 enable others to make and use the best mode of claimed chat bot to interact conversationally using the personality of disclosure. The claimed disclosure should not be construed the specific entity. In some examples, the specific entity as being limited to any aspect, example, or detail provided corresponds to at least one of a friend, a relative, an in this application. Regardless of whether shown and acquaintance, a celebrity, a fictional character and a histori described in combination or separately, the various features cal figure. In some examples, the personality indexprovides 25 (both structural and methodological) are intended to be access to data from the specific entity and to a generalized selectively included or omitted to produce an embodiment chat index. In some examples, the method further comprises with a particular set offeatures. Having been provided with processing the social data using at least one of machine the description and illustration of the present application, learning techniques and one or more rule sets, wherein one skilledinthe artmay envisionvariations, modifications, processing the social data comprises identifying conversa- 30 and alternate aspects falling within the spirit ofthe broader tion data collected for the specific entity and identifying aspects of the general inventive concept embodied in this conversation data collected for one or more entities similar application that do not depart from the broader scope ofthe to the specific entity. In some examples, identifying conver claimed disclosure. sation data collected for one or more entities similar to the What is claimed is: specific entity comprises determining similarities between 35 1. A system comprising: the one or more entities and the specific entity using at least at least one processor; and one of expression analysis techniques, approval indicators, memory coupledto the atleast one processor, thememory and characteristics comparisons. In some examples, the comprising computer executable instructions that, compared characteristics comprise at least one of demo when executed by the at least one processor, performs graphic data, behavioral data, content style, and psycho- 40 a method for creating and interacting with a conversa graphic data. tional chat bot of a specific entity, the method com Aspects ofthe present disclosure further provide a com prising: puter-readable storage device storing computer executable receiving a request associated with a specific entity; instructions thatwhenexecutedcauseacomputing systemto accessing social data associatedwiththe specific entity, perform a method for creating a conversational chat bot of 45 the social data comprising at least one of: images of a specific entity, the method comprising: receiving a request the specific entity, voice data for the specific entity, associated with a specific entity; accessing social data asso conversational data associated with the specific ciated with the specific entity, the social data comprising at entity, and publicly available information about the least one ofimages ofthe specific entity, voice data for the specific entity; specific entity, conversational data associated with the spe- 50 using the social data to create a personality index, cific entity, and publicly available information about the wherein the personality index comprises personality specific entity; using the social data to create a personality information for the specific entity; index, wherein the personality index comprises personality usingthepersonality index to train achat botto interact information for the specific entity; and using the personality conversationallyusingthepersonalityinformationof index to traina chatbot to interact conversationallyusing the 55 the specific entity; personality of the specific entity. In some examples, the receiving, by the chat bot, dialogue; personality index is associated with one or more data generating, by the chat bot, a response to the dialogue processing algorithms for processing the social data, using a hierarchical data traversal process to collect wherein the one or more data processing algorithms corre response data from one or more data sources acces spond to at least one of chat bot interaction rules, image 60 sible to the personality index, wherein collecting the classification rules, and data acquisition rules. In some response data comprises: examples, training the chat bot comprises applying to the determining, by the chat bot, the personality index chat bot at least one of a voice font ofthe specific entity, a does not comprise data foraddressing one ormore 2D image of the specific entity, and a 3D image of the parts ofthe dialogue; specific entity. In some examples, the method further com- 65 composing, bythe chatbot, one ormore questions to prises: receiving, by the trained chat bot, dialogue from a address the data not comprised in the personality user via an interface accessible to the computer-readable index; and
- 20. 10 15 20 30 US 10,853,717 B2 19 20 providing, to a user interacting withthe chat bot, the 12. Themethodofclaim9, themethodfurthercomprising one or more questions. processing the social data using at least one of machine 2. The system ofclaim 1, the method further comprising learning techniques and one or more rule sets, wherein using informationinthe request to identify one ormore data processing the social data comprises identifying conversa sources, whereinthe one or more data sources comprise the 5 tion data collected for the specific entity and identifying social data. conversation data collected for one or more entities similar 3. The system ofclaim 1, wherein the social data is further to the specific entity. based on at least one of social media posts, written letters, 13. The method of claim 9, wherein the compared char userprofile information, behavioral data, transactional data, acteristics comprise atleast oneofdemographic data, behav and geolocation data. ioral data, content style, and psychographic data. 4. The system ofclaim 1, the method further comprising: collecting the accessed social data; 14. Acomputer-readable storage device storing computer executable instructions that when executed cause a comput storing the accessed social data in a data store; and providing an index generation engine access to the stored ing systemto performamethodforcreating aconversational social data. chat bot of a specific entity, the method comprising: 5. The system ofclaim 1, the method further comprising: receiving a request associated with a specific entity; processing the social data using at least one of machine accessing social data associated with the specific entity, learning techniques and one or more rule sets; and the social data comprising one or more images ofthe applyingthe processed social data to thepersonality index specific entity and at least one of voice data for the to generate a personalized personality index. specific entity, conversational data associated with the 6. The systemofclaim 1, whereinthepersonality index is specific entity, andpubliclyavailableinformationabout associated with one or more data processing algorithms for the specific entity; processing the social data, wherein the one or more data usingthe social datato create apersonality index, wherein processing algorithms correspond to at least one ofchat bot the personality index comprises personality informa interaction rules, image classification rules, and data acqui- 25 tion for the specific entity; sition rules. using the personality index to train a chat bot to interact 7. The system of claim 1, wherein training the chat bot conversationally using the personality of the specific comprises applyingto thechatbot atleastoneofavoicefont entity; ofthe specific entity, a 2D image ofthe specific entity, and receiving, by the chat bot, dialogue; a 3D image ofthe specific entity. generating, by the chat bot, a response to the dialogue 8. The system of claim 1, wherein the hierarchical data using a hierarchical data traversal process to collect traversal process comprises evaluating social data from the response data from one ormore data sources accessible specific entity, evaluating social data from entities similarto to the personality index, wherein collecting the the specific entity, evaluating social data from a global user base, and evaluating generic response options. response data comprises: 9. A method for creating a conversational chat bot of a determining,bythe chatbot, thepersonality index does specific entity, the method comprising: not comprise data foraddressing one ormore parts of receiving a request associated with a specific entity; the dialogue; accessing social data associated with the specific entity, composing, by the chat bot, one or more questions to the social data comprising at least one of: images ofthe 40 address the data not comprised in the personality specific entity, voice data for the specific entity, con index; and versational data associatedwith the specific entity, and providing, to a user interacting with the chat bot, the publicly available informationaboutthe specific entity; one or more questions. processing the social data using at least one of machine 15. The computer-readable storage device of claim 14, learning techniques and one ormore rule sets, wherein 45 whereinthepersonality index is associatedwithone ormore processing the social data comprises: data processing algorithms for processing the social data, identifying conversation data collected for the specific wherein the one or more data processing algorithms corre entity; spond to at least one of chat bot interaction rules, image identifying conversation data collected for one ormore classification rules, and data acquisition rules. entities similar to the specific entity; and 16. The computer-readable storage device of claim 14, determining similarities between the one or more enti wherein training the chat bot comprises applying to the chat ties and the specific entity using at least one of bot at least one of a voice font of the specific entity, a 2D expression analysis techniques, approval indicators, image ofthe specific entity, and a 3D image ofthe specific and characteristics comparisons; entity. usingthe social datato createapersonality index,wherein 55 17. The computer-readable storage deviceofclaim 14, the the personality index comprises personality informa method further comprising: tion for the specific entity; and receiving, by the chat bot, dialogue from a user via an using the personality index to train a chat bot to interact interface accessible to the computer-readable storage conversationally using the personality information of device; and the specific entity. generating, by the chat bot, a response to the received 10. The method of claim 9, wherein the specific entity dialogue, wherein generating the response comprises corresponds to at least one of a friend, a relative, an utilizing a data traversal process to collect response acquaintance, a celebrity, a fictional character and a histori data from one or more data sources accessible to the cal figure. personality index. 11. The method ofclaim 9, wherein the personality index 65 18. The system ofclaim 1, whereinthe request comprises provides access to data from the specific entity and to a at least one of: a name, a nickname, an occupation, or a time generalized chat index. period. 35 50 60
- 21. 22 US 10,853,717 B2 21 19. The system ofclaim 6, whereinthe at least one ofchat bot interaction rules, image classification rules, and data acquisition rules comprises criteria for accessing one or more datasets. 20. The system of claim 19, wherein the image classifi- 5 cation rules are configured to compare facial recognition data ofthe specific entity to labeled image data to determine a label for the specific entity.