Beyond ChatGPT: When do companies need custom AI solutions?
Custom AI use cases and common ways to implement them
There are many reasons a company may need custom AI, specifically custom GenAI and LLM solutions. Here are a few common reasons we’ve seen where a company needs a custom solution, and at how some of these can be implemented at a very high level.
Future posts will deep dive on some of these techniques.
Common use cases for enterprise AI
Chatbot for internal documents
Let’s say I want to use an LLM like ChatGPT or Claude, and ask it a question about some data internal to my company. For example, let’s say my company has a document sitting in a company shared folder, called “Phone Interview Questions for Data Engineers” with a bunch of questions to ask candidates in phone interviews.
If I go and ask Claude about this, I’ll get a pretty predictable response:
It obviously has no idea, because it’s a general purpose chatbot built on a “general purpose” LLM (large language model); it has no knowledge of my company, and since the “Phone Interview Questions…” document is private to our company, the LLM has never seen it.
Many companies want a chatbot that is connected to their data, and answers questions in the way they want to answer them, either for internal use (think employee helpdesk) or for external use (think a website help widget).
Educational chat app with individualized user data
If I’m building a normal app, it’s pretty obvious that I want to use the user’s data in generating whatever they see. When I open Youtube, I’m seeing a feed of videos that’s built up of the users I’m connected to or that Youtube recommends me. If I log into my bank, I’m seeing my own acconut details.
The same can be, and often is, true of a chatbot or AI application.
Let’s say my company builds an educational app, and wants to now add a chatbot that lets students ask questions about work they’ve handed in or tests that they’ve taken. Obviously, I need to individualize the answers the chatbot provides for the specific student, often using private data about the student that no one else should have access to.
Knowledge or workflows tied to a custom business domain
A company will often need solutions that work for their specific domain. A great example is something like a company working on apps for lawyers - maybe it wants to build a GenAI solution that helps create contracts.
You can use ChatGPT for this, and many people do; however, there are often better solutions to be found when customizing the GenAI to a specific domain; LLMs that are custom-built for legal language can be much better and more accurate at generating legal language, with fewer inaccuracies and more protection against hallucinations.
All of the above
Many of the above scenarios can happen at the same time.
A great example is a chatbot for lawyers for a specific legal firm, which needs to generate contracts in the same way as the firm has done in the past (internal documents), which needs to access private documents of a specific client (individualized data and internal documents), and which obviously should be very good at generating correct legal language (custom business domain).
Solutions to custom AI
There are a handful of ways to customize GenAI depending on the specific use case. Many of these are used in combination, especially as AI projects get bigger and better performance is needed, as well as more advanced use cases.
First, it’s important to understand what exactly an LLM is, and how a chatbot like ChatGPT or Claude works under-the hood. After all, they are in some sense a custom AI built on top of an LLM as well.
An LLM (Large Language Model) is essentially a very sophisticated pattern recognition system trained on vast amounts of text, which knows how to “continue” text. Think of it as an incredibly well-read expert that has absorbed billions of documents, conversations, and websites, and can continue talking from any text it’s given. These are all documents that are publically available, of course.
In the context of something like ChatGPT, an LLM is the base model, often called the foundation model. ChatGPT is a custom AI application built on top of this foundation model, which has been “taught” to answer questions in a chatbot style that is comfortable for humans to chat with, not just to complete text. It’s then wrapped in a nice UI, and given lots of other functionality (for example, it can now “search the web” for you to get up-to-date information).
When your company builds a custom solution, it’s also taking a foundation model and building something on top of it. There are many ways to do this, but the main ones are:
Prompt Engineering.
This is a pretty well-known technique by now, and is very easy and cost-effective.
Basically, you use very specifically crafted prompts to get the model to behave in certain ways.
For example, you can build a chatbot and instruct it to “answer in short sentences, always use the Chicago Manual of Style guidelines for writing English, and don’t mention Amazon because they are a competitor”.
You can “silently” add these guidelines to every conversation your users have with your chatbot
RAG (Retrival Augmented Generation).
This is one of the main ways of enhancing a base model with internal company data.
It’s basically a way to inject into the prompt of a chatbot, internal company data. You can think of it like an enhanced Prompt Engineering.
The way it normally works is, when your user asks something of your chatbot, you find data relevant to what they are asking, and add it into the prompt so the LLM has the data available.
This can be general data like things about the user that is chatting with the chatbot.
Or it can be prompt-specific things. For example, if the user mentioned a “hiring document” you can search the company files for a hiring document and add it to the user prompt.
Fine-tuning:
This is a way of taking the base model, and training it on data that is more specific to what you want.
This is usually done to make an LLM that works better on a specific domain.
For example, you can make an LLM that is better at programming, or better at generating legal language.
You can in theory use this to train an LLM on internal company data, so it has it available, in much the same way as ChatGPT “knows about” classic books.
But this is risky - often company data should not be available to everyone in the company, and that is very hard to enforce if the knowledge is baked into a generally-available chatbot.
In addition, this is usually too expensive and time-consuming. Fine-tuning isn’t always a simple process, and internal company documents can often change all the time, so constantly fine-tuning becomes a major barrier.
In our example use cases, the “Chatbot for internal documents” would most likely be implemented via RAG and prompt engineering, the “Educational chat app with individualized user data” would most likely be built via RAG and prompt engineering as well, and the “Knowledge or workflows tied to a custom business domain” would most likely start with Fine-tuning, though would almost certainly use the other techniques as well.
Conclusion
There are many cases in which a company will need a custom Gen AI solution, and many different ways of achieving that goal. In future posts we will deep dive into some of these solutions.
Feel free to contact us for help in deciding what your company needs to fulfill its use cases.