NVIDIA’s ChatRTX: Run Your Own AI on Your PC
NVIDIA’s ChatRTX is an innovative tool that allows users to run their own GPT Large Language Model (LLM) on their PCs using their own content. This powerful software enables users to extract information from various types of files, including images, PDFs, meeting notes, and more. By simply having a conversation with the AI, users can easily retrieve information about their business and documents, such as customer feedback or upcoming meetings.
To get started with ChatRTX, users need to visit the website provided in the video and download the software. It is important to note that the download size is quite large due to the inclusion of the entire model. Before proceeding with the download, users are recommended to check the system requirements. ChatRTX requires an NVIDIA RTX 30 series or 40 series GPU to operate.
Once installed, ChatRTX opens in two separate windows. The command prompt window displays logs and any errors that may occur, while the main interface is used for configuring and interacting with the AI. The top left corner of the interface allows users to choose the model they want to use. The Mistral model is included by default, which is highly performing and open-source. Other models, such as CLIP and Chat GLM, can be added for different functionalities and language preferences.
Users can also select the dataset the AI model will leverage. The default option relies on the AI model’s original training data, while the folder path option allows users to point the AI to a specific folder on their computer containing relevant files. The AI supports various file types, such as text documents, PDFs, and images. Refreshing the dataset is as simple as clicking a button, ensuring that any changes or additions to the folder are recognized by the AI.
Interacting with the AI is easy. Users can simply type or speak their questions using the microphone icon, which leverages Whisper AI to convert speech to text accurately. The AI responds with relevant information extracted from the content, even providing reference files for further details. Users can ask about salary information, upcoming meetings, customer feedback, or any other information within their files.
One of the newest features introduced in ChatRTX is its ability to analyze images using the CLIP model. By selecting this model, users can ask specific queries about the content of their photos and receive accurate results. For example, users can ask to see photos of chocolate chip cookies or even specific images, such as a blimp or the company’s storefront. The AI’s capability to interpret the content of images without relying on metadata is truly impressive.
In conclusion, NVIDIA’s ChatRTX allows users to harness the power of AI directly on their PCs and leverage their own content. With the ability to chat and extract information from various files and images, this tool provides a convenient and efficient way to access valuable data. The software’s simplicity, offline functionality, and respect for privacy make it an attractive choice for businesses and individuals alike.
Questions:
1. What are the system requirements for using NVIDIA ChatRTX?
– NVIDIA RTX 30 series or 40 series GPU is required.
2. How can users select the model they want to use in ChatRTX?
– Users can choose the desired model from the top left corner of the interface.
3. How can users add additional models to ChatRTX?
– By clicking on the “add additional models” option in the model selection menu.
4. What types of files does the AI in ChatRTX support?
– The AI can process text documents, PDFs, and images.
5. How does ChatRTX handle image analysis?
– ChatRTX uses the CLIP model to analyze the content of images and provide accurate results.
Get the latest high-quality tutorial and tips and tricks videos emailed to your inbox each week: https://kevinstratvert.com/newsletter/
Does it support subfolders in dataset and how the size of the dataset relates to CPU and Rem usage.
And next time when I ask Mistral on the Net it shows me the same numbers and files?
Sucks that they blocked out older card users in order to try and sell they're new hardware 🕵️
learning listening reply
Just tried the software, but it doesn't work as expected
Is it worth downloading other language models if I need to analyze data and afterwards write code in python or will mistral handle it?
Nvidia could create the best system with the best interface. But they don´t. Right now several Github Projects are better, support more file types and have more features.
Excellent presentation thanks 😊
I have no idea why I would use this over other opensource items that do a lot better job and I can use any model I wish…
This AI is basically useless..
Follow up request: how to install different LLMs. Had a look on Huggingface but haven't worked out how to install those.
Also ChatRTX doesn't seem to reference all items in a folder when answering, just the first one that seems to have an answer. So if you need a summary of multiple files, it won't do it.
Heated up my 4090 more then me playing Crysis 4 all maxed out.
ChatRTX please support 2060…
Seems ChatRTX didn't get the memo that my G14 packs an RTX 3060. . Oh well, the disappointment is real!
KEVIN‼️ i subscribed after watching your awesome SQL tutorial. Can you please do a TABLEAU for beginners in the same style? THANK YOU❣️ 🙏🏻
I am so confused to why it works so smothely for you. When I ask the same questions it, simpy answers that it deosn't know the information. Even when telling it which file it is, it just proceedes to inform me how I can open the file.
play at 2x speed, he's milking it
First, GREAT info in your vids. Really useful stuff so thank you. I'm a little off topic but I wanted to find a recent vid. I found one of your instructional vids from about a Year ago re. WhisperAI. fyi Whisper now works with Python 3.11 but not 3.12. In your video you prompt a "-U" command to update but that doesn't seem to do anything so I'm guessing that I'm up-to-date. I've seen some referance to Whisper alternatives that are "faster", "more accurate" etc. Since Whisper seems to be working, more-or-less fine I am of the 'if it a'int broke, don't fix it' ilk. That said, do you have any more recent data re Whisper than the >1Yr. old information? Thanks in advance.
Creatives are usually hating on AI. Now imagine how useful this is going to be for writers referencing across a huge amount of world building materials! I will use it that way and I am already salivating!
I uninstalled and download it again to make a clean install. I find it annoying that there are almost no infos about the update that lets you add your own models or where you see clip. It doesnt show up and nVidia doesnt even maintain a change log list.
Cool but data types is pretty limiting … no excel , no images.
I'm going to call it now…
AI Model X needs 6GB of Memory.
AMD/Nvidia says that 6GB Model needs 16GB's!
Want another Model that does more, oh that needs 32GB's! (Really needs 10).
What better way to force people into buying their new hardware! Just over inflate the requirements for LLM's in order to sell more hardware.
What's the maximum amount of data present in a folder it can handle?
Can you have more than one folder path in the same session?
Does AMD have an equivalent 😅
Guess it works anyway regardless the gpu! Did you try? Wonder if there will be a linux version…
So this is quite similar to Microsoft copilot. But cheaper upfront?
@kevinstratvert This is great. Thanks for the detailed breakdown. I downloaded, installed, and played w/ this a few months back and it struggled a good bit w/ some basic requests. I think i'll need to hear some others shout its praises before i spend the time on this again.
that being said, thanks for the great video
I installed this today. Early impressions(I may update this if I don't decide to delete it): I am using CLIP to search images. RTX 4090.
*It does scan subfolders.
*It sucks at truthfulness/identification. I asked it to show an image of a lion. I had an image in a sub-sub folder. It show the image, but also 2 images of people. 😵💫
*'Show me an image of a car'. Showed 1 image of a person in front of cars, and a folder full of images that had nothing to do with cars.
*It never seems to return a null result.
So, it's very hit or miss. Which for me means I can't use it for anything useful yet.