The Tale of Two AIs: GPT vs. Expert Systems

ChatGPT is transforming the way we communicate and work. Organizations in the public and private sectors are using this tool to increase workflow efficiency and performance, enhance customer experiences, and eliminate daily routine work tasks. I am pleased to invite Dr. Martin R. Stytz, Ph.D., CISSP, CCSP, in this installment of CTO Corner to discuss GPT, what it does, and how it relates to expert systems to see what models may be right for your team.

Generative pre-trained transformers (GPT) are large language models that use massive amounts of text data (i.e., terabytes) to understand natural language and generate human-like responses. OpenAI created ChatGPT using a GPT model to develop this artificial intelligence system that is revolutionizing how organizations operate.

GPT models are designed to learn patterns, contexts, and relationships in language by processing and analyzing diverse textual information. The models are large neural nets, but the term large in the language model comes from the number of words the model holds and the number of parameters (relationships) between the words. Currently, GPT is said to have over 100 trillion relationships as part of its software and statistical models.

To train the GPT model, a large dataset is fed into the model, consisting of sentences, paragraphs, or entire documents. The GPT model learns to predict the next word or phrase in a given context based on the patterns it observes in the training data. By iteratively adjusting its internal parameters, the model gradually improves its ability to generate coherent and contextually appropriate responses. (Note that the response is not guaranteed to be accurate, just coherent and contextually appropriate). GPT does not understand the input; it simply provides the most likely output (based on its training, discussed below).

GPT is not intelligent; indeed, humans still are not sure what natural intelligence is. A computer scientist bet a philosopher that by 2023, humanity would know how consciousness arose from the combination of human chemicals in the brain. The philosopher won and collected the bet. Intelligence is more complex than consciousness.

GPT is trained using unsupervised learning, which does not require explicit labels or annotations for the training data. Instead, unsupervised learning learns (determines parameters) directly from the input text, capturing the statistical relationships and structures inherent in the language. The statistical relationships and structures allow the model to parse input and generate text naturally and nuancedly. The statistical relationships are captured in matrix hierarchies (which are large and deep), making GPT very computationally expensive.

One of the key strengths of learned language models is their ability to generate creative and contextually appropriate responses. No other AI (Artificial Intelligence) technique can do this. Learned language models can parse input and generate text across a wide range of topics, imitating the style and tone of different writing styles or adapting to specific instructions the user gives. GPT models have the potential to provide informative, engaging, and human-like interactions in various applications. Still, GPT is not necessarily correct and does not understand the exchange between the user and the GPT system.

The key components of a GPT (Generative Pre-trained Transformer) system include:

 

 Exhibit 1. Key Components of GPT.

The current version, GPT-4, is more creative and collaborative than the older version, GPT-3.5. GPT-4 can generate, edit, and iterate with users on creative and technical writing tasks, such as composing songs, writing screenplays, or learning a user’s writing style. Similar to a human who may tell a lie, GPT outputs a few different errors, known as hallucinations. These include incorrect outputs or the GPT fabricating data. OpenAI provides an overview of GPT-4 at: https://openai.com/gpt-4.

GPT-4 is not an expert system but a common AI system that emulates a human expert’s problem-solving and decision-making capabilities in a specific domain. An expert system is designed to provide expert-level knowledge and advice to users, helping them solve complex problems or make informed decisions. Expert systems do not hallucinate; they stop processing.

An expert system’s key components include:

 

Exhibit 2. Expert Systems Key Components

1. Knowledge Base: A knowledge base is a repository of domain-specific knowledge, facts, rules, and heuristics. It contains the expertise of human specialists and experts in the given field, organized in a structured manner.
2. Inference Engine: The inference engine applies the knowledge from the knowledge base to solve problems or answer queries. It uses reasoning techniques, such as forward chaining (works from facts to conclusions) or backward chaining (works from goals to facts), to make logical inferences and draw conclusions.
3. User Interface: The user interface allows users to interact with the expert system. It can be text-based, graphical, or even voice-based, depending on the design and intended use of the system. The interface capabilities are much like those available in GPT.
4. Explanation Module: Expert systems often include an explanation module to provide transparent reasoning and justification for the system’s outputs or recommendations. This helps users understand the underlying logic and builds trust in the system’s suggestions.

Expert systems are widely deployed in various domains, including medicine, engineering, finance, troubleshooting, and quality control. They can assist in diagnosing diseases, offering financial planning advice, solving technical problems, and decision-making support. One of their advantages is the ability to capture and encapsulate domain expertise, making it accessible to a wider audience. Additionally, these systems provide the correct answer and consistent, reliable, and quick responses, reducing the dependence on human experts. Yet, when using them, you must remember these systems are limited in the knowledge to explicitly encode in their rules, and they may fail with novel or complex situations that fall outside their predefined knowledge base. Furthermore, IT experts have found these systems can be notoriously difficult to extend or expand and are difficult to use for cyberattack, cyberdefense, or programming. There are several key differences between GPT (such as ChatGPT) and an expert system:

Exhibit 2. GPT vs. Expert Systems. Which One Do You Need?

GPT and expert systems have their strengths and weaknesses. GPT is better at handling several topics and tasks with a data-driven approach. Expert systems provide in-depth expertise within specific domains, allowing for greater explanation ability and control. Compared to expert systems, GPT is easier to expand and extend into new areas, such as programming, language translation, and imaging. GPT systems have been proven efficient and accurate for use during cyber defense, cyberattacks, and programming. They demonstrate how to provide useful, real-world capabilities.

At ITC Federal, we generate customized solutions to help customers assess how GPT and expert systems are used to support enterprise architectures. We have teams readily available to assist in tailoring these systems to meet requirement objectives. At ITC, we bring certified developers and technical experts who have been early adopters of the expert systems and GPT models who offer expertise in Data Analytics and can build interactive applications to enhance workflows and communication between customers. Our teams are standing by to help you choose the AI approach that will help meet your desired business and IT outcomes.

About the Author

Dr. Martin Stytz is a Sr. Program Manager specializing in cybersecurity, data analytics, distributed computing, and artificial intelligence for large enterprise systems. He brings over 30 years of experience in government IT to various agencies. When Marty is not working, he spends his time maintaining his USAFA Class website, exercising, and learning and traveling.