No Intelligence without Knowledge Representation

Yes, that was an attempt to invoke the famous slogan of the American revolutionary war "No taxation without representation", although the circumstances are very different here. No one is demanding that knowledge representation be a prerequisite for intelligence. Rather the argument here is that no "intelligent" applications that the market is craving for - self-driving cars, conversational chatbots and storefront robots for example - can be developed sidestepping thorny areas of Artificial Intelligence (AI) that are full of difficult questions, most notably Knowledge Representation and Reasoning. This assertion has profound effects on what the market should expect and how companies should proceed in utilizing the significant powers of Machine Learning (ML) and AI that are already available to us.

What is Knowledge Representation?

Simply put, Knowledge Representation is about storing what we understand - about a particular topic or about the universe in general - in a systematic manner that supports reasoning. We humans do this almost every moment. Looking out of the window right now, I see dark clouds and that makes me realize that it will probably rain today. This is based on my understanding of weather, my ability to reason on what I know about weather and what I see right now.

If we want a machine - an intelligent agent - to do this, what we should do based on the technology available right now is to develop an ontology - a formal and explicit specification on a shared understanding - that represents what we understand about weather and develop a system that performs reasoning on that ontology as well as sensory input. OWL (Web Ontology Language) has become the de facto standard for defining ontologies and since it is based on Description Logic, it is possible to carry out reasoning on it. The technology works for simple problems - we can develop a reasoning system for weather based on an ontology which would function reasonably well - but it is hard. Very hard. If we generalize the problem into developing a general Knowledge Representation and Reasoning (KR&R) system, well it basically cannot be solved right now.

There are different aspects of the problem that makes it very hard to solve. One is that human knowledge is not a single gigantic ontology which is internally consistent. We have multiple viewpoints about about the same thing that are at times contradictory. Knowledge Representation (KR) as a field attempts to account for this by developing multiple ontologies, which may not be fully consistent of each other. Moreover, the efforts to develop an all-encompassing single ontology that encapsulates all the knowledge collectively held by the mankind have largely been abandoned now. While we are moving in the right direction, progress has been slow in terms of making systems that can handle knowledge representation.

Another problem with knowledge representation and reasoning is that it is very difficult to develop systems that can construct ontologies - or understandings - on their own. Humans are experts on ontology creation. We effortlessly and instantly create new concepts and related them to what we already know as in "oh, there are vehicles that have only three wheels but are kind of like cars". It should also be noted that our understanding is not limited to physical things. We develop abstract concepts all the time - tell someone who has never heard of the electoral college and how it operates and they would get it - and enrich our understanding with them. All this is very hard for machines or "intelligent systems" as they are often called. There are techniques for developing ontologies based on inputs, mostly for text, but they are not as robust or as sophisticated as the ontology development processes of humans.

Reasoning on ontologies is also a related hard problem. Logic works fine on well defined ontologies but when ontologies themselves are evolving, incomplete and sometimes inconsistent with each other, the problem becomes harder and harder to tackle.

In summary, knowledge representation and reasoning (KR&R) are hard problems. I am not saying they are unsolvable problems - just very hard and we haven't solved them to a satisfactory level as of now. Researchers are spending a lot of time on these problems but progress has been slow. Very slow.

This rather depressing description about the status of intelligent systems or artificial intelligence would appear to be at odds with the recent developments in the industry and what is being reported. After all, we have seen a large number of applications that make use of artificial intelligence and machine learning - navigation apps, search engines, chatbots, translators and "almost" self-driving cars just to name a few. As a matter of fact, these two pictures are not contradictory. The simple truth is that we have learnt how to develop "intelligent systems" while largely sidestepping the hard problems of knowledge representation and reasoning. Each and every one of the above applications has found ways to sidestep knowledge representation and reasoning while still appearing to be intelligent. It works, generally speaking, at least up to a certain level. I think we have reached the limit of this method for at least a few applications and they will be stuck there until and unless the hard problems of knowledge representation and reasoning are solved.

This discussion is also related to the distinction between "narrow (weak) AI" and "general (strong) AI". All the currently available AI products are narrow AI products in that they handle a specific task - identifying letters in handwriting, playing chess or selecting a document matching a search query for example. They have not achieved general intelligence which can be described as the ability to handle any task in an "intelligent" manner as if a human would. The argument here is that some of the products that many people think are right around the corner - conversation agents and self-driving cars for example - are not narrow enough to be achieved simply using the narrow AI technologies which have worked for us so far and require at least some aspects of general AI - primarily Knowledge Representation and Reasoning.

Intelligence without Knowledge Representation

It is interesting to look at some "intelligent" applications and dissect how they operate while sidestepping knowledge representation. The easiest to understand is probably navigation systems. Google Maps has a very sophisticated database of locations and providing instructions on how to go from your home to the Texas capital building does not require the system to understand the difference between small towns and cities that you pass on the way or the political leaning of a particular city. The concepts that the system has to understand - the difference between toll roads and normal roads for example - have been carefully encoded into the system by its creators. And it works fine, most of the time. In situations such as a rural road being closed without the app not knowing, we know better than to go ahead.

Translators such as Google Translate are more advanced than navigation systems. A lot of the success of early translating systems was due to statistical NLP, which analyzes patterns of word occurrences with various statistical tools often using a bilingual corpora (Canadian Hansard records that are published both in English and French is a resource cited often) which have nothing to do with knowledge representation and reasoning. Google Translate and similar systems are reported to be using a different system, which has an intermediate representation that does not depend on a particular language and relies on this intermediate representation for translating. Presumably, they still make heavy use of statistical NLP to generate a translation that sounds syntactically correct in the target language. While this method is closer to knowledge representation than pure statistical translation, it is still a statistical system where the intermediate representation is heavily guided by the the text that has been processed. Based on the publicly available information on how companies such as Google use knowledge representation, these systems most probably do not have a deep ontology that connects the concepts on the underlying representation and is incapable of performing reasoning on them. If they were, they would be much more successful in hard translating tasks such as translating poetry.

Chatbots are probably the technology that comes closest to looking like having some form of knowledge representation and reasoning built in to them. There are various types of chatbots ranging from chatbots used in commercial websites to assist customers to personal assistants such as Google Assistant and Siri. Some are even reported to be more advanced than personal assistants - Sophia the Robot for example. Such systems and even personal assistants are not even referred to as chatbots on many occasions since they are seen as systems more advanced than regular chatbots used in websites. However, the underlying technology is pretty much the same for all these systems.

The more advanced chatbots such as Google Assistant actually do make use of knowledge representation. But their use of knowledge representation is very narrow, basically limited to identifying instances of a limited number of concepts (artists for example) and a limited number of relationships between instances. For Google Assistant, the underlying knowledge base is a system named Google Knowledge Graph, which is discussed in some detail in the following section. The bottom line is that while Google Knowledge Graph contains a large number of facts, its underlying ontology is very shallow and it is not capable of performing the more advanced functionalities of knowledge representation and reasoning such as automatically updating the ontologies, supporting multiple ontologies and carrying out reasoning on the information contained in the ontology. This severely hampers the conversation ability of these conversation agents.

Moreover, the type of conversation that these systems can engage in is very limited. They get activated when a predefined word is heard and depending on what you say they would do one of the limited number of tasks or answer the question that you asked. The knowledge representation systems powering them support this type of conversations. But this is a very limited type of conversation that comes nowhere close to how a human would converse, where the response can be marginally related to what your hear but then expands on it or moves to a completely different topic just to name two possibilities. Engaging in these types of conversations - in the manner close to how HAL 9000 engages in conversations in Arthur C. Clarke's 2001 - requires a much more sophisticated from of knowledge representation and reasoning.

Lacking such sophisticated knowledge representation and reasoning capabilities, these systems sometimes rely on an archaic technique first used in 1960s to get the conversation going - to provide the syntactic sugar between answering question and carrying out tasks. This technique was pioneered by a program named ELIZA which is largely based on Rogerian psychotherapy where the psychotherapist encourages the patient to talk about themselves by asking them to expand on what they say which can be achieved using a surprisingly limited vocabulary and conversation ability. Some versions of the earlier ELIZA programs would occasionally interject statements like "Do you have a mental disorder?" which would be awkward to say the least if used in current chatbots. But statements such as this are effective in convincing the human that they are talking with an intelligent agent so much so that there have been reports that some people chatting with earlier versions of ELIZA, without knowing that it was a bot, were convinced that it was a rude human. Although it looks like this means ELIZA has passed the Turing Test, this is not the case: Under the settings of the Turing Test, the human knows that one entity he or she is conversing with is a machine and just has to distinguish it from a human. The humans fooled by ELIZA would have easily identified it as the machine.

It should also be noted that personal assistants such as Siri, Alexa and Google Home are different from regular chatbots in the websites in that they are integrated with speech recognition and speech synthesis systems, which have been solved to a satisfactory level using largely statistical techniques (and recently deep neural networks). This does not change their utility. Users know that these are not really conversation systems, that would just talk with humans, although they are sometimes portrayed as such.

The same models is being used by more advanced chatbots that have sometimes been labelled as intelligent beings such as Sophia the Robot, which has even been awarded citizenship of Saudi Arabia. She (!) has an artificial face that shows emotions which makes it easy to believe that she has achieved artificial general intelligence, in other words to believe that she is an actual intelligent agent. But behind the scene, she uses the same model used by personal assistants: decrypt the question to identify key concepts, find matching records from a database of facts built on a limited ontology and synthesize an answer.

The bottom line is that even the most sophisticated chatbots or conversational agents have not solved knowledge representation and reasoning. That's why it's fairly easy to get these fairly systems to do ridiculous things such as getting Amazon Alexa and Google Home to talk in a never ending loop. This wouldn't happen if these systems had powerful knowledge representation and reasoning systems. Moreover, we would have actual artificial intelligent agents carrying out tasks such as handling all customer interactions in a fast food restaurant or answering your questions and taking actions in a customer help line. The technology is not yet there. Not even close. In essence, these conversational agents have sidestepped the hard problems of knowledge representation and reasoning just like navigation systems and machine translation systems although they are doing it in a clever manner that creates an illusion of solving knowledge representation and reasoning under some conditions.

I don't think there is anything wrong with developing "intelligent" applications without trying to attack the problems of knowledge representation and reasoning head on. Most of the success of AI system after the so called AI winter from 1960s to 1980s were achieved by building on what is possible in narrow focused areas rather than trying to solve problems such as knowledge representation which fall under the general (strong) AI or GOFAI (Good Old Fashioned AI). A lot of progress of what has been achieved was due to Machine Learning - systems can learn things and improve their performance - which falls well short of the grand expectations of AI. But it does not mean that we can develop all the products that the markets are demanding while continuing to sidestep the hard problems. We appear to have reached the limits now in terms of some products.

Current State of Knowledge Representation

It's not like tech companies have completely ignored the field of knowledge representation. Products such as Google Assistant, Siri and "knowledge panels" that appear in Google search results would have been impossible without some use of knowledge representation. It has been reported that Google uses a product named "Google Knowledge Graph" and it would be reasonably to assume that the technologies used by other tech companies are similar in nature considering that Google is often seen as the market leader in handling information. So let's look at Google Knowledge Graph in some detail.

Google Knowledge Graph had been launched in 2012. In the context of knowledge representation, it can be considered a simple ontology with a few concepts and up to millions of instances for each concept. In addition, it keeps track of relationships between the instances. For example, it "knows" that Leonardo da Vinci is an renaissance era artist and that Mona Lisa was a painting created by him. It also knows that Raphael and Michelangelo are artists from the same era. This is presumably achieved by having a set of concepts (e.g, person, period, artwork), assigning instances to those concepts (e.g, Leonardo da Vinci is a person, Mona Lisa is an artwork) and storing connections between these instances (e.g., Mona Lisa painting is an artwork created by Leonardo da Vinci). The following introductory video provided by Google in 2012 at the launch of Knowledge Graph provides an overview of how it operates.

As mentioned earlier, Google Knowledge Graph is used by Google Assistant in providing answers to questions posed by users. It is also used to create "Knowledge Panels" that appear in the right hand side of Google search results that provides the information the user is looking for instead of asking her to follow a link to a webpage and find the information herself. This is "information extraction" in action as described in the next section.

Not much information is publicly available on how Google Knowledge Graph works beyond what is described above but there are lot of reasons to believe that the ontology used to store the knowledge is not that sophisticated - in terms of the number of concepts supported, formalizing the types of properties that exist between instances of concepts, the ability to automatically add concepts and properties based on the available information and more advanced functionalities such as carrying out logical reasoning on concepts and instances of the ontology or supporting multiple ontologies that might not be fully consistent with each other. For example, the documentation Google provides on the API for Knowledge Graph lists about a dozen of types of entities which probably indicates that the number of concepts are at most in the hundreds. These concepts are taken from schema.org, which is an ontology that many tech companies are promoting the websites to use to organize the information they provide. It has showed some success but has still not shown the widespread adoption that would make it easier to organize the information using this fairly limited schema/ontology.

In essence, my critique about Google Knowledge Graph and similar products is that they come nowhere near to what is needed to support the intelligent applications such as self-driving cars and intelligent chatbots that many people expect to be operational in the near future. That would require much more sophisticated and extensive ontologies than what schema.org currently provides, such as the ability to automatically update the ontologies by adding more concepts based on the information are processed and the ability to carry out logical reasoning on the information contained in the knowledge base selecting appropriate concepts and instances in real time. These are not simple asks and would require years if not decades of research work which would not fit on to the optimistic timelines envisioned in launching the above mentioned intelligent applications. Moreover, I think that attempting to develop such intelligent applications without significantly improving the current technology on knowledge representation is an attempt to sidestep knowledge representation in this endeavor.

Limitations in Sidestepping Knowledge Representation

Let's return to the central question raised in previous sections - Where have we reached the limits in terms of what is possible to achieve without knowledge representation and reasoning? I think we have already reached this limit for self-driving vehicles and conversational chatbots. Machine translation is another product that may well have been stalled although it is closer to the final outcomes than others. Search engines and question answering systems are probably an area where the products can reach a new level of sophistication if KR&R is solved.

On the first glance, demanding that self-driving cars possess sophisticated knowledge representation and reasoning capabilities appears to be preposterous. After all, aren't we just expecting these devices to drive from point A to point B? They are often portrayed as the next logical step for ride hailing services such as Uber or Lyft where the app charges your credit card and sends a vehicle that doesn't have a driver to pick you up and takes you to your destination. You can track your path using the familiar navigation app. Everything sounds reasonable.

But when you think about all the intricate details of driving a vehicle, things get complicated. Very complicated. Let's start with a simple question. What if the passenger has a heart attack while riding a self-driving cab? What is the cab supposed to do? Is it supposed to not do anything and just take the passenger to the destination? Will it even "know" that the passenger had a heart attack? A human driver certainly would and either call 911 or take the passenger to the nearest hospital. If a self-driving car is to handle situations like this, it would have to continuously process the sensory information collected from the vehicle and identify triggers that require special actions. Some processing can be transferred to servers outside the vehicle but they would still have to account for a wide range of triggers and the actions required by each trigger. Even if Machine Learning techniques can handle the tasks of identifying all the triggers with an accuracy level comparable with humans we are still talking about a wide range of triggers and actions which would keep on evolving. For a human, it would be easy and natural, but for a system that has limited knowledge representation and reasoning capabilities, these would be very hard to handle. In other words, we can basically see narrow AI failing in this situation and requiring something that is closer to general AI.

Does that seem like that far-fetched scenario which doesn't even apply for all instances of self-driving vehicles, only fully automated taxi services? After all your vehicle is not expected to take you to the hospital or call 911 if you get sick. Ok, let's take a look at more mundane examples directly involved with driving. On a sunny day, a self-driving car driving on a country road facing the bright white sky can't distinguish any difference in the color on the road ahead from the color of the sky. But there's a pickup truck moving slowly, perpendicular to the road, just to the right off the road. A human driver would immediately slow down and take a better look since they know that pickup trucks with trailers could make a left turn across the road, especially in a country road, a rancher entering his ranch for example. A self-driving car does not have such a rich understanding about peculiarities of rural roads and would just rely on the sensory input. As such, unless the sensors can detect the difference in color of the trailer from the color of the background white sky, it would just move forward at the speed set for the road. Unfortunately, this hypothetical scenario is very similar to what appears to be what has happened in a fatal crash involving a Tesla and a truck. Of course, the driver is expected to keep an eye on the road, without letting the vehicle drive on its own since Tesla Autopilot is not meant to function as a self-driving car. But it does indicate that current self-driving technologies have serious limitations because they don't have a good "understanding" about what driving a vehicle means.

It can be argued that these kinds of problems can be fixed by improving the sensory systems of the vehicles so that they don't miss any kind of signal that could possibly indicate a danger. But the set of circumstances that fall into this category appear to be quite large and it would be very difficult to develop systems that can account for every one of them. For example, there has been a fatal crash involving a self-driving Uber car (with a test driver) that had failed to respond in time to a jaywalking pedestrian with a bike where poor lighting conditions may have also played a role. Again, a human driver would know to be more careful when it looks like someone in the curb might start to jaywalk. And what about adverse weather conditions that can suddenly make the roads unnavigable? What about a bridge collapsing ahead of you? What about instances where the map is incorrect (no system will be 100% accurate even in the future)

In short, the argument here is that without a robust knowledge representation and reasoning system to make sense of the environment around it, a self-driving vehicle would not be able to achieve a level of performance comparable with a human driver. The bar is especially high for self-driving vehicles when compared with systems like machine translators or storefront bots, where the effects of errors are far less serious and can be easily corrected.

Conversational chatbots, including voice-activated assistants such as Amazon Alexa and Google Home, are useful devices when used for tasks that they can handle. For example, it's useful to have a text-based chatbot in a travel website, where the customers can get answers to some simple questions. People know that these systems have limitations but are ready to use them with some understanding that they can occasionally get things wrong. The same goes to voice-activated assistant that control the TV or thermostat.

But there are significant limitations in these devices and they are by no means intelligent agents that they are sometimes presented to be. The fact of the matter is that it is fairly easy to fool these devices. Even swapping how words are used, which would make no difference to a human, has been shown to fool them. The example of creating a never ending loop between Alexa and Google Home was previously mentioned. As long as these system do not rely on sophisticated knowledge representation and reasoning systems these problems will persist. This is not to say that they will be not be useful. Rather the point is that they can only be used for a narrow set of tasks and humans have to be careful in using them, based on the understanding that they operate largely by pattern matching. In addition, it means that we don't have to worry about granting legal rights to these conversational chatbots just yet. We don't have the technology to produce intelligent agents such as HAL 9000. Not even close.

The same goes for machine translation. Machine translation is very useful as is especially because humans are capable of making sense of utterances of languages with some errors. But if you are thinking about using a machine translation system to translate the parliamentary proceeding to another language which a level of accuracy close to human translators, we are not there yet. And we won't be there until and unless the problems of knowledge representation and reasoning are solved.

Search engines provide an interesting use case for knowledge representation and reasoning systems in that they have the potential to generate the next generation of these systems. It should be noted that search engines are one of most remarkable products made using AI and ML technologies. When the Internet began to explode in 1990s, the search engines of the time were largely based on library search systems and relied on keywords and queries based on logic and could not handle large datasets. From there, we have arrived at today's Google which can predict your search query as soon as you type a few letters and instantly provide links to documents that contain what you are looking for with the relevant sections already highlighted. A set of inventions - vector-space model, PageRank algorithm, etc. - have powered this. Crucially, knowledge representation and reasoning was not one of them.

In NLP, search engines used to be categorized as "Information Retrieval" or IR systems. The idea here is that given your search query, the system will give you links to documents that contain the information. Then you have to read those documents, which are ranked by relevance in modern search engines (this didn't use to be the case in library search engines), to find the information you need.

But anyone who has used Google recently know that in many cases this is not the way it operates now. In addition to giving you links to the documents that contain what you are looking for, it also "extracts" the information from the most reliable sources and presents them directly to you. Search for "<Your City> weather" or "Albert Einstein" and there will be a box above or to the right of the search results that indicates the information you are looking for that Google has extracted from the source it deems most reliable. This is mostly achieved using Google Knowledge Graph as described earlier. In NLP, there is a name for this - Information Extraction. Information Extraction or IE fundamentally differs from information retrieval in that the system would present the information to the user instead of just a link from which the user has to obtain the information. It requires a more advanced form of NLP than IR.

Does this mean that Google has made a great leap from IR to IE? Well, the answer has to be nuanced here. The general idea behind IE is that there will be a separate system that stores the information extracted from sources and that system would be answering the questions in an authoritative manner. What's the point of extracting the information if the user still has to go back to the original sources? This brings KR&R into play. It's still acceptable to provide links to the documents that were used in obtaining the answer on request or as a secondary resource but the answer itself should come from the system itself. For instance, under "Ontology-based information extraction", a subfield of information extraction, there would be ontologies that store the information extracted from the websites and queries executed over these ontologies would provide the answers.

If the approach described above is used in Google, there would be a Google Sage or something, that answers your question. Mind you, this would be much more sophisticated than the current Google Knowledge Graph and would able to handle the hard problems of knowledge representation and reasoning described above. Also, it goes beyond the objectives of information extraction since information extraction systems are supposed to work on a narrow domain - universities or movies for example. This system, on the other hand, would be capable of answering questions on any domain which requires stepping into the area of natural language understanding falling under general AI. Behind the scenes, there might be a large number of ontologies seamlessly integrated but from the user's point of view, once a question is asked, the system would just provide the answer. It would be different from Google Knowledge Panels in that answers would be provided directly for all questions and sources will be provided only upon request, which would presumably happen only on a small percentage of queries. Granted, this would have serious business implications: Google Knowledge Panels have already been criticized for leading to a drop in traffic for source websites such as Wikipedia. But this is what a true next generation "search engine" supported by information extraction and knowledge representation and reasoning would look like. The fact that there is no such system is another indication that we still haven't solved knowledge representation and reasoning.

In summary, we have to acknowledge that there are serious limitations in developing intelligent agents without tackling the hard problem of knowledge representation and reasoning. It doesn't mean that these system are useless. Just that they would fall short of producing many transformative technologies that the markets are yearning for.

Curb Your Enthusiasm

Therefore, we should listen to Larry David here and curb our enthusiasm. Again, to reiterate, this does not mean that we have to completely give up on these applications or on AI and ML. Rather we have to be realistic about what can be developed with the available technology in the near term. When will be able to solve the problems of KR&R to a satisfactory level and be able to have actual intelligent agents? I would say an optimistic estimate would be 20 years.

What does this mean with respect to leading AI products that the markets are crazy about? Let's take one by one. Self-driving cars, as described earlier, are most probably not going to be operational anytime soon. Instead, the markets should focus on making the best use of assisted driving, with all its complications. Assisted driving involves hard questions such as when to request a human to take over and how to enforce this but tackling these problems would be a much more productive exercise than waiting for self-driving cars to appear.

What about chatbots and voice-activated assistants? They are good for the tasks they are carrying out right now - answering simple questions based on information organized in a structured manner and controlling devices based on simple voice commands. They are not going to evolve into sophisticated conversational agents anytime soon. If you think about a travel agent service, a chatbot is not going to be sophisticated enough to answer all your questions to a satisfactory level and make the reservations for you anytime soon. If you are thinking about replacing the humans taking orders from customers with chatbots, again not going to happen anytime soon. But with regards to this, chains appeared to have come up with a solution that doesn't involve any AI in the form of self-service kiosks that significantly reduces the need for humans to take orders. The same goes for clerks at the checkout counters of supermarkets. More and more stores are nudging customers to use self-checkout kiosk instead of relying on AI to develop intelligent agents that can do the work. Getting the customers to do "shadow work" that are ordinarily being carried out by employees is an old business trick and it works fine without AI.

What about machine translators? Again, they can't be expected to significantly improve from their current state to a level comparable with human translators anytime soon. This means that anything that is produced by a machine translator would have to be checked by a human if it is going to be used for a critical purpose. However, it would be perfectly fine to use them for informal communication purposes, speaking with another person who can work with the problems in the translation for example.

The implications for search engines were discussed above. It is probably time to experiment with the next generation of systems that would just answer the questions you have based on everything it has learnt from the Internet, but again it would take a lot of time to become fully functional. In the meantime, Google can make improvements to its current technique of extracting useful pieces of information from search results and presenting them with the details of the sources.

The bottom line is that there will only be incremental improvements in AI technologies for the time being. There will most probably not be any startling AI products such as self-driving cars or intelligent conversation agents for at least the next couple of decades. There have been several predictions that there would be such products in 2020 or so and none have come in to being. It's time to accept that these things are not going to be made for some more time.

Things We Can Do

Does this mean that companies should invest less on AI and just ignore it. Definitely not. That would be like saying that companies should ignore the Internet after the dot com crash in 2000 because it revealed serious weaknesses of the technology at the time. On the contrary, it means that companies should continue to invest in AI but be judicious about how they are spending their money. Significant improvements continue to be made in several fields on AI and ML and companies can invest in research work - for example in research work KR&R or computer vision if they foresee that such research work can develop something that would add value to their businesses. More importantly many companies can make use of the available AI and ML technologies to improve their businesses.

It is time to make a distinction between AI and ML. Machine Learning is the field of study aimed at developing systems that can improve their performance with more and more data or experience. Think about email spam filters. They get better and better with time and as they analyze more recent data and update their models. In contrast, normal computer programs would do the same thing no matter how many years you use it. Machine Learning has a significant overlap with AI, which can be described as developing intelligent entities or agents, but is narrower in focus. It is simply focused in developing systems whose performance can be improved without the involvement of humans, often by providing more data.

Many products in the AI space in the past couple of decades have been invented due to the developments in the field of Machine Learning. We now have ML applications for tasks ranging from figuring out how to help patients improve their health, fraud detection and weather prediction to calculating credit scores. My employer has cool set of products that make use Machine Learning to help with student retention in academic institutions. Most of these applications avoid the need to have KR&R play a role in the products developed. As such, these applications target tasks that humans find very difficult to do on their own (e.g., real-time fraudulent transaction detection) to tasks that humans think they can do but machines can outperform them (e.g., helping patients improve their health). Recently, there has been a report that Google has produced a system that performs better than radiologists in identifying breast cancers by training a system on a large collection of records. In a way, this is not even surprising. Of course, a machine can do a better job at developing a classification system for radiology images by learning patterns from a large number of images. We have a whole range of technologies at our disposal for these tasks and they keep on getting better and better as researchers are making improvements to the techniques.

What does this mean for businesses? They should seriously think about making use of ML and AI to improve their processes and products. The technologies in the field of "Applied Machine Learning", which is the field of study that concerns itself with using ML techniques for practical applications is mature and it is not difficult to an experienced practitioner to recommend how a business can make use of ML techniques. Contrary to popular belief, the hard questions are often not on figuring out which ML technique to use as many of them achieve comparable performance. Rather the problems are on understanding exactly what needs to be done, cleaning the dataset and developing good features to be used by the model (a process known as feature engineering). Deep learning, of course, does not rely on good features as it can develop features on its own but it has its own drawbacks in the context of many business applications - primarily low interpretability (the inability to get a sense of how the model makes a prediction). It works great for applications such as computer vision and speech recognition where interpretability is not important.

The bottom line is that this an exciting period for the fields of Artificial Intelligence and Machine Learning and businesses should seriously consider making use of them. However, they should have realistic expectations and understand their limitations - primarily ones arising from the current limitations in the field of knowledge representation and reasoning.

How to Achieve Intelligence

I didn't try to answer the question of how to resolve the problems of knowledge representation and reasoning and develop truly intelligent agents. I think the short answer for this is that nobody knows. There have been some improvements in the field but the field is still at an early stage. That's why I think that 20 years is actually an optimistic estimate on how soon these problems can be solved and intelligent applications can be developed. Since this is the current state of affairs, any discussion on how to achieve knowledge representation and reasoning or intelligence in general risks entering the territory of speculation. I'll take a stab at it taking this risk since this is not a scientific paper. The reader should keep this in mind in reading the remainder of this section.

From one point of view, the root cause here is that we don't know how the only system that can do knowledge representation and reasoning (human brain) does it. There have been significant developments in the area of neuroscience but based on my limited understanding of the field, I don't think we are anywhere close to understanding how the brain stores various types of knowledge and instantly retrieves them. Had we known that we would have made an attempt to replicate them using artificial neural networks or just neural networks as they are often called. But neural networks come nowhere close to the human brain in terms of complexity. Even more than the number of neurons, the number of connections between neurons in the brain create a far complicated structure than any neural network: each neuron in the brain is connected to thousands of other neurons whereas neurons in neural networks are connected to just a few other neurons. These rich connections in the human brain creates a complex form of processing that is very difficult to understand and replicate. It can be speculated that it's these complicated connections that store our rich understanding and knowledge and make it possible to retrieve them whenever we want. For example, think of an instance where someone has been rude to you. You would be immediately able to remember one instance after the other as if they were connected by some kind of a chain. (This is a curse when someone is depressed as it would be very easy to remember all the other times that one is depressed). Maybe these events are stored in a set of neurons connected by synapses? I don't think anybody knows the answer. The point is that humans can do this for any concept. This is a very sophisticated from of knowledge representation that we have no way of replicating. And we haven't even touched on the topic of reasoning.

The current research work on knowledge representation and reasoning is primarily based on symbolic logic. This relies on abstract techniques that are not related at all with neuroscience or neural networks. These two approaches for developing AI systems have often been compared to the invention of artificial flight: some people tried to achieve it by dissecting birds and some people tried to achieve it by studying the principles of aerodynamics. In the end, it was studying the principles of aerodynamics that achieved artificial flight. But this is probably an imperfect analogy when it comes to achieving artificial intelligence. Given the complexity of the task, it can be speculated that we would have rely on some form of neural networks that achieve parallelized processing and robustness that are difficult to achieve using symbol processing over computers. Again, it is just speculation at this point. The bottom line is that these are very difficult problems and we are nowhere close to solving them.

In summary, we have a long way to go before we develop truly intelligent systems. There would be host of serious questions when we finally have the technology to produce intelligent agents (will they take over the worlds as sci-fi predicts? should we give them human rights? should we even produce them?) but it's way too early to grapple with these questions.

It may even be possible that we cannot solve the problems of KR&R and produce intelligent agents. After all, evolution took millions of years to produce human intelligence and some species - Neanderthals for example - seem to have tentatively close but fallen short. And a human child takes many years of sensory input collected day and night to develop the sophisticated KR&R system that we associate with humans. It's possible that we simply cannot achieve this result using machines. In other words, we can't be certain that we can develop the technology to achieve intelligence KR&R and more broadly general intelligence, although it exists in humans.

Conclusion

My goal was to present a realistic assessment of what is possible to achieve using AI and ML technologies available right now and available in the near future. The bottom line is that we currently don't have technologies that solve the problem of Knowledge Representation and Reasoning and would not have them for a very long time. As such, expectations from intelligent agents should be scaled down with this realization. However, it is important to understand that even with this limitation we can do a lot of useful things with the available AI and ML technologies. This means that business should continue to make investments in making use of the available AI and ML technologies.

It an exciting time to make use of the fields of AI and ML. We just need to be realistic about what is possible and what is not possible.