The most important AI & DDD terms explained

AI Assistant is a generic name of an IT system that uses machine learning models to help people perform complex workflows faster.

Under the hood the such a system would probably be composed from one or more Machine Learning models (usually LLMs), data storage and automated workflows for processing the data and handling user requests.

Mechanics that allow LLMs to selectively focus their attention on specific parts of data for better prediction. Attention mechanism is implemented using a specialised layer of neural networks in the transformer architecture.

Vector-based RAG split documents into smaller parts that can be embedded, indexed and stored for the information retrieval. These document fragments are called “Chunks”. Each chunk is usually 128-1024 characters long.

This limitation helps to strike a balance between level of detail and processing speed. Chunks that are too small could take important information out of context, while chunks that are too large could dilute the meaning or simply not fit into the model context.

In DDD (Domain-Driven Design), a domain expert is a person with in-depth knowledge and understanding of the business area or a specific business problem.

In the context of Domain-Driven Design (DDD), the domain model is an abstract representation of a business area. This model is not simply a direct copy of the real business world, but rather a simplified representation that aims to capture and structure the essential aspects and processes of the business.

The aim of the domain model is to develop a deeper understanding of the business area that is to be automated. It includes important entities (such as products, services, customers, contracts, etc.), their relationships with each other and the rules that govern their interactions.

By mapping the complex aspects of the real business in a structured form, the domain model serves as a bridge between the technical and business aspects of a project.

The domain model helps developers and business people alike to find a common language. It facilitates communication and ensures that the software solution accurately reflects the business requirements.

An embedding is the representation of some text as a numerical vector intended to capture the semantics of the text: when two embeddings have small distance, then their meaning is related. Embeddings are computed by ML models - they are learned.

Post-training process that allows to further specialise LLMs: gain new skills and insights. This uses special targeted datasets.

Most popular form of fine-tuning is LoRA, which freezes most of the LLM and adapts only a small part of it.

Hallucination is when LLM returns an answer that is incorrect or completely disconnected from the reality. Hallucinations normally happen when:

- the model is weak or poorly trained,

- the necessary information is missing in the input context,

- the input context is also filled with irrelevant information,

- the LLM is asked to do something it is not designed for: e.g. math or rhyming.

Inference means using a trained Large Language Model to generate new text based on the input data.

When we talk about creating text with a Large Language Model , you can think of it as a writer writing based on their previous reading and experience. The model has read many texts during the previous learning phase and now tries to write its own text based on a given starting piece or keywords. In this process, it adds one word (or small piece of text, also called a token) at a time to form a coherent and meaningful text.

This process of writing text is called "inference" and differs from the "training phase" in which the model learns.

Other terms that are used synonymously for inference are: Prediction, output generation, model response

Within the context of AI Assistants, knowledge maps are informational aids that help LLMs (Large Language Models) to locate relevant information and provide helpful answers

Knowledge maps can be designed to reflect the workflows and mental models of human experts performing similar tasks. DDD (Domain-Driven Design) offers processes for this design, see "Domain Models."

An LLM (Large Language Model) is a very large machine learning model and belongs to the field of deep learning. More precisely, it is usually a so-called "transformer" that contains several neural networks in its architecture.

The special thing about LLMs is that they are able to recognize input data and generate completely new output data. This is why they are classified as generative artificial intelligence.

GPT-4 is the most well-known LLM - it can understand text and images, generating new texts in response.

You can run some LLMs (Large Language Models) on your own computer or even telephone, completely detached from the Internet.

Such LLMs are usually referred to as local LLMs. Mistral and Llama are the best-known LLMs that can be executed locally.

Running LLMs (Large Language Models) locally offers companies various advantages. One of the main reasons for this is data protection: if an LLM is run locally on company hardware, all processed data remains in the internal network. This reduces the risk of data leaks or unwanted access by third parties, which is particularly important for companies that work with sensitive information.

Another advantage is control. Companies can fully control the environment in which the LLM runs, which means they can adjust the system settings, data protection measures and usage policy to suit their needs. This allows for finer tuning of the model's performance and security.

In addition, the local execution of LLMs often improves the response speed, as there are no delays caused by network requests to external servers. This can be particularly important for time-critical applications where every second counts.

Running LLMs locally can be more cost effective in the long term, especially with high levels of usage. Although the initial set-up and maintenance costs can be high, the ongoing costs of cloud services are eliminated, which can represent a significant saving with constant use.

A parameter is a numerical value that the Large Language Model has learned during the training process. This value influences its behavior. Most modern language models have a large number of parameters.
Each of these parameters helps the model to better capture certain features of the language. The more parameters a model has, the more detailed it can learn and the more finely it can recognize and react to differences in the language. A model like Mistral 7B, for example, has 7 billion such parameters, while a larger model like ChatGPT-4 has up to a trillion. This large number of parameters enables the model to store and utilize very complex and diverse linguistic knowledge, which helps it to communicate more human-like and accurately.

In the context of LLMs (Large Language Models), a prompt is a carefully formulated instruction to the Large Language Model.

The quality of these instructions strongly influences the quality of the LLM's responses. This has created a field of knowledge called prompt engineering (e.g. you can find a prompting guide here).

Over time, the importance of prompt quality is expected to decrease thanks to RLHF (Reinforcement Learning from Human Feedback) and general model training.

RAG (Retrieval-Augmented Generation) describes a system in which an LLM can consult additional documents before providing an answer.

This makes it possible to work with data on which the model has not been trained.

Example: An AI chatbot that can read PDFs and answer questions about them.

RAG systems are therefore particularly interesting and valuable for companies, as they can be used in knowledge management, for example.

Reinforced Learning from Human Feedback - an iterative process that allows a trained LLM to adapt better to human preferences, based on the feedback.

Temperature is a parameter in LLM inference process that controls randomness and creativity of the outputs. If temperature is non-zero, LLM can randomly pick not the next most likely token, but something good enough.

LLMs work with numbers. Tokenisation is a process of translating words into numbers that LLMs can operate with. Most frequently this is done using BPE tokeniser that uses a dictionary of words and subword parts.

The tokenization process differs from the embedding process because it preserves the exact information about the words and letters.

Process of creating a new LLM by teaching it to understand and generate new texts. It learns new language patterns from massive datasets in form of websites, books and forums.

Training can be very computationally expensive. Instead of training an LLM from scratch, it may make sense to fine-tune a pre-trained model instead.

New neural network architecture that most of the popular LLMs are based on. It stacks multiple models in parallel, allowing the resulting system to learn context, pay attention and track relationships between words.

This parallel structure enables the model to process large amounts of data quickly and efficiently. Each layer of the model helps to improve the comprehensibility of the entire system by analyzing and interpreting specific aspects of the data.

Vectors are a sequence of numbers. In the context of language models (LLMs), vectors are numerical representations of words and text fragments that capture the semantic relationship between different texts.

A vector database is a database that can efficiently store and process vectors - large lists of numbers.

Pinecone, Milvus and Weaviate are known as specialized vector databases, but nowadays even ordinary SQL databases can work efficiently with vectors.

By using a vector database, RAG systems can therefore quickly access a large amount of information and retrieve data to answer queries.