AI Model CARD shall be All you need
AI Model CARD shall be All you need
The best way to learn how to use a Model, and for what to use it.
I got you. You just found out a promising LLM. Or maybe you simply heard about it. You cannot but jump on downloading it, ready to make it your own.
But the reality is different. You plug it on, and it only says gibberish and nonsense.
It happens to me all the time!
I recently curated my first LLM on Hugging Face! Well, it was nothing compared to the heavy stuff many of the community active players are doing.
I did not train any model, but converted one existing LLM into OpenVINO format. Surprisingly it was an easy feat, but as a result a new model was pushed into my Hugging Face Repository.
Finally an opportunity!
An opportunity to make a good Model Card: one so easy to understand that you can plug and play it.
Because a Good Model Card is all you need to make a model run smoothly.
Here is what I did, and hopefully, a blueprint that will come in hand to you.
What is a Model Card for?
First thing first. A Model Card is the CV of a Generative AI model. It should tell you everything about the training, the experiences, the success.
I am a greedy HF user. I browse the model library at least twice a week to spot new gems, models so good but treated as underdogs.
And here is the real pain. If you are curious explorer, like me, whenever a new model is released you want to test it straight away.
So you go on Hugging Face, download the model, paste your code snippet and… FAILURE!
It is normal though!
Every model has its own tweaks. You always have to consider:
the correct prompt template
the context window
the languages you can ask question and receive answers in
the training dataset…
Thanks god we have the Model Cards! Or, at least this should be the way…
Here I will give you a quick but comprehensive overview on how to proceed by yourself.
I am a PoorGPU-guy, so I will use only quantized models that can run on CPU: this also means that anyone can do the same 😉
A bad Model Card
Let’s start with a bad example, or better to say a poor example (the worst ones later on). I was checking out PowerLM and PowerMoE by IBM: they looked promising models, small sized. Claimed on the social to be amazing.
Wow, IBM. I was ready to enjoy it. But… there is always a BUT.
It may look good to you. But you cannot really do anything with this model, as it is.
If you try to run the code, you will get a long sequence of nonsense.
Why?
There is nowhere any kind of basic information on ho to use the model: we don’t know:
max tokens (context window)
we don’t know the prompt template
Let’s look at one more example, not a too bad one. Raspberry-3B is indeed a very good model. Considering that 3 Billion parameters model have an amazing accuracy and display proficiency in many tasks. It is like 3B is a soft spot, a golden ratio.
But this model card has nothing… except for the prompt format (ChatML) non even properly formalized. It should have been something like this:
There are dome really bad model cards… they should be banned. If something is not documented it is like it doesn’t exists!
Look at this one: a template with nothing.
But when you think you already hit rock bottom…
You may find something like the picture below… there can be no template at all (at least you are not wasting time reading…)
Anatomy of a Good Model CARD
Here below a good example of what should be done. It is still missing the prompt format, though. That is one of the most crucial information to make a model work properly
Here you have everything:
specialized format output and understanding
Context length
Languages supported
Number of Layers
knowledge base
Thank you Alibaba Cloud!
In the next newsletter I will share with you How I created my first OpenVINO model, and curated its model CARD.
For now, as gift for this week-end, the most appropriate Medium article for the topic. Here you have a practical guide to use all the information from the Model Card and put them into action.
This is only the start!
Hope you will find all of this useful. Feel free to contact me on Medium.
I am using Substack only for the newsletter. Here every week I am giving free links to my paid articles on Medium.
Follow me and Read my latest articles https://medium.com/@fabio.matricardi
Check out my Substack page, if you missed some posts. And, since it is free, feel free to share it!