Let’s use the idea of librarians to help you understand different types of AI and how they work. This comparison isn’t perfect, but it should give you a good general idea of the various kinds of AI and their functions.

The Super Librarians: ChatGPT, Claude, Gemini

Imagine AI as a super-smart librarian who has read every book in their library. You can ask them any question in your own language, and they’ll give you a good answer back. They have a great memory, but it’s not perfect. They’re very likely to remember something mentioned in 100 books, but probably not something written in just one book. Also, with so much to remember, they can sometimes get a little confused and tell you something that’s not totally correct or even occasionally make something up.

ChatGPT, Claude, and Gemini are like librarians who have read every publicly available book, website, newspaper, and magazine on Earth. They’ve also watched every public movie and video, and read every bit of public code. In short, they’ve taken in all of publicly available human knowledge.

You can ask them almost anything and get a solid answer. But they do have some limits:

1. They’re expensive to run. Training them cost billions of dollars, so companies need to make that money back. They let the public ask some questions for free, but heavy users have to pay.
2. Their knowledge isn’t perfect. They don’t remember every detail of everything they’ve read. Instead, they summarize the knowledge, and sometimes in doing that, things get lost or mixed up.
3. They don’t remember much about you. They deal with so many people that they don’t easily remember more than your last question or two.
4. If it’s not in their “books,” they don’t know it. They can tell you what’s in the information they’ve learned, but nothing outside of that. They can sometimes connect ideas from different subjects, but they can’t invent completely new information (at least, the current public versions can’t).
5. Privacy: Unless you tell them not to, they might learn from your questions and could repeat that information, including private stuff, to others.

Other Types of AI Librarians

Those super librarians are the most well-known and capable, but there are many different types of AI “librarians” that exist for various reasons.

Machine Learning and Deep Learning AIs

These were the kings of Artificial Intelligence until the new librarians using Generative AI (like ChatGPT) came along. These AIs do everything from predicting the weather to deciding what shows up in our social media feeds. They’re everywhere and run a lot of the world.

They’re not well known for one main reason: they don’t speak human languages. They speak math. To talk with them, experts need years of training and hard-won experience. In their own areas, these AIs are far more knowledgeable than ChatGPT, but they’re much harder for regular people to use because we can’t easily communicate with them.

Open-Source Models

These are like librarians trained on a much smaller set of books focusing on a particular topic. That topic could be finding features in images, turning audio into text, or summarizing long documents. These AIs are great at their one specific job but not very good at anything else. Often, they do their specific tasks about as well as ChatGPT would, but at a much lower cost. These single-focus AIs are great choices for doing one task quickly and cheaply.

Custom Models

This is like bringing in a trainee librarian and having them read only your books. It’s very expensive because, in addition to the effort of reading the books, you must provide a framework for them to understand and use the information. Once done, you have an expert librarian to answer questions just about your books.

Fine-Tuned Models

This is where you find an already trained librarian and have them read all of your books too. This creates a specialist in your information without all the trouble and expense of setting up a framework from scratch. It allows you to tap into the intelligence gained from the base AI (foundation model) while specializing in your topic. It’s expensive, but a fraction of the cost of a custom model.

RAG AI (Retrieval Augmented Generation)

RAG AI is like having a librarian, but when you ask a question, you also give them a few carefully selected books on the specific topic. If you’re good at finding these books quickly and correctly, this method is fast, allows for specific answers, and is much less expensive than fine-tuning. The challenge with this method is finding the right “books” to give the AI. But if you give a super-librarian a book on exactly the subject your question is about the answers are usually exemplary.

Mini Models

These are like junior librarians who have the wide general knowledge of the main librarians but not as good recall. They are much less expensive to employ. For tasks they can handle effectively, these are great to use.

How to Choose an AI

Which AI to choose depends on what you want to do with it.

For most people and most uses, the free version of ChatGPT, Claude, or Gemini should be enough. These AIs are highly versatile, usually follow instructions well, and provide high-quality output. If you need answers from information that isn’t public, like company documents, then these AIs aren’t a good option (unless paired with another method) since they have no knowledge of your data.

If you use AI a lot, especially through an API (a way for computer programs to talk to each other), then which model you use is less important than how you set up your system for using models. It’s really important to automate your model evaluations. The AI world is changing fast, with new capable models being released almost daily. Automating your evaluation of models for your task allows you to change models quickly by letting you know fast if the new model provides the quality and speed of results you need for each specific job.

It’s also important to be able to use many models at the same time. For instance, I use a mini model to handle basic information extraction from data, but I use the full-sized model to handle more complex tasks like answering difficult questions and providing standardized output. The mini version costs 1/33rd the price per unit (tokens, which is approximately one word) that the full version does. So using it when possible is a big savings.

Some Basic Thoughts on When to Use Various Models:

1. Machine Learning and Deep Learning AIs: Because of the immense expertise required to use these, I’d say that if you don’t have this expertise, you shouldn’t consider them. If you do, you’ll know when and where to use them. In short, don’t try to use the librarian who only speaks ancient Greek unless you speak ancient Greek or have an excellent translator.

• Open-Source Models: If you have specific tasks that these models are good at, then they can be a good option because they’re less expensive. This is especially true for things like changing speech to text or video to text. I think of these more like code libraries than models: code to do very specific things.

While I hear of companies using open-source models for all of their processing, I’ve found that these models are extremely weak at general tasks compared to the state-of-the-art models (despite claims to the contrary) and falling further behind. As such, I don’t recommend them for general use. They are, however, unbeatable at very specific tasks.

• Custom Model: If you have a large amount of data that doesn’t change often, then training your own model can work well, if you can afford to do so. This takes considerable expertise and expensive processing power (although it’s getting cheaper by the day). Unless you’re an AI business and deeply understand both how AI works in-depth and its rapid evolution, I wouldn’t recommend this approach. Custom models struggle to keep up with the exponentially accelerating pace of AI models.

• Fine-tuning: With fine-tuning your own model, you get the benefit of all of the framework of the underlying model you’re fine-tuning at a fraction of the cost of training a model. Updating the fine-tuning as your data changes is also easier than training your own model. If the data is changing slowly, and you can fine-tune monthly or less frequently, then this is a viable option. It still takes a fair amount of expertise. If you anticipate a very high volume of use, then the high setup costs for fine-tuning might be worthwhile.

• RAG AI: I’m biased towards RAG AI because it allows using a state-of-the-art AI but with a carefully curated set of data to guide its answer. RAG AI is only as good as the ability to provide relevant guide data to the AI quickly, and that requires expertise. RAG AI is flexible enough to answer based on data just received. The downside of RAG AI is that because you are passing a “book” to the AI with each query, and because the cost is driven by how much data goes in and out, this extra data adds to costs.

Conclusion

Understanding the world of AI can seem overwhelming, but thinking of AI systems as different types of librarians can help make sense of it all. Remember:

1. General AI (like ChatGPT, Claude, and Gemini) are like super-librarians who have read almost everything but have some limitations.

2. Specialized AIs are like librarians who are experts in specific topics or tasks.

3. Different types of AI “librarians” exist for various needs:

   – Machine Learning AIs for complex mathematical tasks

   – Open-Source Models for specific, cost-effective solutions

   – Custom and Fine-Tuned Models for working with private or specialized information

   – RAG AI for flexible, up-to-date responses using specific data. Low upfront cost. Higher operating costs

4. Choosing the right AI depends on your specific needs, budget, and technical expertise.

5. The AI field is evolving rapidly, so what’s best today might change tomorrow.

Hopefully this analogy will help you understand the current world of AI and it’s current permutations.

Remember, the AI world is changing incredibly fast. What’s best today might not be the best choice tomorrow!