Technology News – Following Nvidia's conversation with RTX comes AMD's GPT-based compiled chatbot powered by a large language model (LLM) that requires relatively modern hardware.
In February, we also reported on Chat With RTX, which can be run locally on Nvidia GeForce RTX 4000 and 3000 cards with the help of TensorRT-LLM, producing a local data packet. Now we get something similar from AMD, but it can run not only on their video cards. If you have an AI-capable CPU from AMD, you won't have to buy one of the latest Radeon 7000 series video cards to run the technology.
AMD Ryzen 7000 and 8000 APUs can use the XDNA NPU (AI accelerator), and Radeon RX 7000 cards can use AI acceleration cores. The company announced in a blog post how to run the chatbot on your computer. Download the appropriate version of LM Studio (for AMD Ryzen processors, for AMD Radeon RX 7000 series graphics cards) and run it. On the Search tab, copy the following search term depending on what you want to run. For Mistral 7b, you need “TheBloke/OpenHermes-2.5-Mistral-7B-GGUF” and select it from the results on the left, it will likely be first on the list. To run LLAMA v2 7b, search for “TheBloke/Llama-2-7B-Chat-GGUF”. You can also try other models: On the right panel, scroll to the Q4 KM model file and download it. AMD recommends this for most Ryzen MI models. Switch to the chat tab and wait for it to start.
The situation is a little different if our device does not have such a processor and we instead use an AMD Radeon 7000 video card. In this case, the GPU Offload option must be selected on the right panel, which must then be converted to the maximum value. Then we check the selected GPU type to see if it indicates AMD ROCm. Then we can start using the chatbot.
AI has become a part of our lives in every way, which is why we're seeing expansion from Nvidia and AMD.
source: WCCFTech, AMD, LM Studio, LM Studio Rocom
