Uncover the Reality Behind Google’s Gemini AI – Exclusive Insights

By
The AI Advantage
-
39
331

All right so Google just came out with an announcement that is their answer to GPT 4 and GPT 3.5 in its current state and yay they’re coming out with a model that is better than gp4 and this is both very exciting and interesting and very disappointing so in this video we’re going to talk about both plus everything you need to know as a user of these products because a part of this announcement is available today for Google’s B but more on that later because first things first we need to talk about the ginormous announcement here Gemini Ultra, their most capable model moving forward.

HTML tags:

All right so Google just came out with an announcement that is their answer to GPT 4 and GPT 3.5 in its current state and yay they’re coming out with a model that is better than gp4 and this is both very exciting and interesting and very disappointing so in this video we’re going to talk about both plus everything you need to know as a user of these products because a part of this announcement is available today for Google’s B but more on that later because first things first we need to talk about the ginormous announcement here Gemini Ultra their most capable Model moving forward

Summary

It’s multimodal from the ground Up, not as an afterthought as is the case with gp4 based on their design. Some of these use cases are mind-blowing. It actually beats gp4 on most benchmarks, which is amazing. This is a truly Next Level model that will be coming sometimes early next year. So, yeah the numbers check out here from what they’re showing what they’re telling us this is really impressive. Google Executives will be able to maintain pH in front of their board members and shareholders at the end of 2023 and users will be getting Next Level Tech in a few months. I like it, that’s a win-win. Here’s the biggest deal. We just crossed a specific score on this MMLU Benchmark and the score that they talked about is this 89.8% on the MMLU Benchmark. This is what human experts achieve when taking that test. Okay, so this is the very first model we will get access to that is actually better than human experts at specific tasks. Yikes! So as you can see, the Gemini Ultra scores 90% on this and gbd4 scores 87.2. But they actually tried to fool us here. No, not happening on this channel though because look in the blog post they’re comparing the performance of Gemini Ultra and GPT 4 but they’re not using the same prompting technique. Okay, so whether that’s a fair technique to evaluate it or not, they used prompt engineering to get the best result possible. But what they’re comparing it to is to a five-shot approach on gp4 which is a different and generally speaking less capable way of prompting the model. It’s important to be aware of that. But other than that, it just beats gbd4 across the boards, so it’s fair to assume that this is a better model than gp4. Another important fact here is that this model will have a 32k context window which does not match GPT 4 128k. Some other important points that need to be taken into consideration are the multimodality of this model and the developer ecosystem. In addition to this, Google has released other models, including Gemini Pro and Gemini Nano. The Gemini Pro is essentially the 3.5, with better performance in certain tasks and worse performance in others. Gemini Nano is also a significant release with two versions, one with 1.8 billion parameters and the other with 3.25 billion. The Pro model inside of BART today is good, and it includes prompt presets which are beneficial for developers. It’s important to bear in mind that all the prompts learned from this channel and prompt engineering resources can be applied in the BART model. Lastly, it was emphasized that an AI arms race is happening, with multiple companies competing to get ahead in the game.

RELATED ARTICLESMORE FROM AUTHOR

39 COMMENTS

  4. Hallucinations. I asked Bard about spies during WW2 & he gave me 4 names, Oh,ok. So I asked other spies in that era. & he gave more names. But I crossed checked one name at Google but couldn't find the name. I asked him about that & he appologized saying he was still a technology under development, but then he gave a book title with the author's where the man/spy he said was mentioned as a spy, so I crossed checked the book this time, but again I couldn't find the title & the author so I asked him again about that & he appologized again giving me the same excuse. So I crossed checked in ChatGPT, the names of the first spy ring that Bard gave me. ChatGPT game me 5 names, two of the names were the same with the ones Bard gave, but the other 3 were new. So I asked ChatGPT about the two characters that Bard gave me but was not in the group that Chat gave me, and Chat said if there were any new developments in the information about the Spy Ring he wasn't aware, & added maybe there were confusion in the data. Now, why did Bard give me names that' can't be found thru Google Search and a book title that doesn't exist and can't be found thru Google Search? The 1st Spy Ring Bard gave me was the "Sorge Ring". The name I coudn't find as a spy in the 2nd Ring that Bard me was Hermann Ritter he said was a journalist/spy & the book title he gave me was "Spies In Paradise: Five Americans in Japanese Occupied Korea" by Michael J. Allen. Well Michael J. Allen is really an author but the book title doesn't exists. Well, i'm sure improvements with Bard will come eventually & be even more informative. Like he said, he's a technology in developement.

  8. 🎯 Key Takeaways for quick navigation:

    00:00 🚀 Google announced Gemini Ultra, their next-gen AI model, aiming to surpass GPT-4.
    00:27 🌐 Gemini Ultra is a multimodal model designed from the ground up, offering impressive capabilities and outperforming GPT-4 on most benchmarks.
    01:20 📊 Gemini Ultra achieved a benchmark score of 90%, surpassing human expert performance in specific tasks.
    02:29 🔍 Google compared Gemini Ultra with GPT-4 using different prompting techniques, potentially inflating the performance difference.
    03:11 🪙 Gemini Ultra has a 32k context window, smaller than GPT-4's 128k, but sufficient for most use cases.
    04:18 📷 Gemini Ultra's multimodal design allows it to handle video inputs, potentially revolutionizing software interaction.
    05:28 🎩 Google's demo of Gemini Ultra used altered prompts, raising questions about the accuracy of their claims.
    06:41 🌟 Gemini Ultra's ability to handle various use cases makes it a powerful assistant for future software applications.
    08:35 🌐 Google's developer ecosystem still lags behind OpenAI's, despite their recent advancements in AI technology.
    09:05 🆕 Google introduced Gemini Ultra, Pro, and Nano models, with Nano offering a compact option suitable for smartphones.
    10:13 📝 Google's upgraded Bart model provides a different flavor of GPT-3.5 but may not necessarily outperform it.
    11:51 🏁 The AI arms race is intensifying with multiple companies competing to lead in AI technology.
    13:01 🤖 The AI landscape is evolving rapidly, with GPT-5 on the horizon, promising an even more competitive future.

    Made with HARPA AI

  15. 🎯 Key Takeaways for quick navigation:

    00:00 🚀 Introduction to Google's Gemini AI Announcement
    – Google's announcement of a new AI model to rival GPT-4 and GPT-3.5.
    – Highlights the excitement and disappointment surrounding the announcement.
    – Mention of availability for Google's B and an upcoming model named Gemini Ultra.
    00:27 🌟 Summary of Gemini Ultra Features
    – Gemini Ultra is a multimodal model designed from the ground up.
    – Outperforms GPT-4 on most benchmarks.
    – Expected to be available early next year with a 32k context window.
    01:20 🧠 Gemini Ultra's Exceptional Performance
    – Gemini Ultra achieves an impressive 90% score on the MMLU Benchmark.
    – Comparison with GPT-4's 87.2% score using different prompting techniques.
    – Gemini Ultra surpasses human expert performance on specific tasks.
    03:11 🎥 Multimodality in Gemini Ultra
    – Gemini Ultra's unique multimodal design from the ground up.
    – Comparison with traditional multimodal models that combine text, vision, and audio separately.
    – Mention of Gemini Ultra's ability to process video inputs, enhancing its capabilities.
    05:13 🎮 Real-World Use Cases of Gemini Ultra
    – Demonstrates various use cases, including games, identifying countries, and analyzing videos.
    – Emphasizes the model's snappy response time and interactive capabilities.
    – Highlights the potential for Gemini Ultra to become the future of AI-driven software assistants.
    08:49 🌐 Developer Ecosystem and OpenAI Comparison
    – Discusses the importance of a robust developer ecosystem.
    – Compares Google's AI landscape, including Gemini models, with OpenAI's developer-friendly approach.
    – Suggests that OpenAI's developer ecosystem remains unmatched.
    09:45 🆕 Other Gemini Models and Bart Upgrade
    – Overview of Gemini Ultra, Pro, and Nano models.
    – Mention of the potential impact of Gemini Nano on smartphones in 2024.
    – Evaluation of the new and upgraded Bart model, with a subjective assessment of its performance.
    11:23 🌍 The AI Arms Race and Future Competitions
    – Discusses the AI arms race among various companies, including Google, OpenAI, and others.
    – Speculates on the potential announcement of GPT-5 in 2024.
    – Highlights the dynamic and exciting nature of the AI field in the coming years.

    Made with HARPA AI

  27. OpenAI has become CloseAI and on its way to becoming LoseAI (next MySpace). CloseAI team is not better, they just had their first move. Customers and developers will soon leave and find alternatives. GOOG and TLSA will be the ultimate winners along with new comers. Most people will not remember CloseAI within a year or two, just like what happened to MySpace.

  33. SPOT ON . . . thank you, Igor, for pointing out that the 90% COT (Chain of Thought)@32 metric versus their comparison of GPT-4 (pre-turbo) rating of 86.4% (using the 5-shot metric) was unfair along with the 'doctored prompts' which is worse than cherry-picking, imho (but I'm impartial, as you know – LOL). So, no shade on Google, of course. That said, when Gemini "Ultra" releases in 2024, there will be others with newer models (thinking GPT-4.5/T + Anthropic MMM + Apple); that's when REAL comparisons count. Right now "Gemini Pro" is a great improvement to BARD, about equivalent to our 2022 GPT 3.5 model. Either way, the technology acceleration along with healthy competition benefits progress, keeps prices in check, ensures bandwidth, and offers choice. Consumers win, however you look at it. And, looking forward to your next batch of videos, brother. [Shaun]

  35. Well done 👏. Nicely nuanced analysis Igor. Best video on Gemini so far.

    How the heck do you accomplish and learn all of this… and now you have a new channel? You are exceptional.

    I’m so grateful for you and your team but I’m not sure that I can handle another year of this head spinning pace!!😅.

    Oi.