Have you ever thought what it would be like having an assistant, all to yourself – who can communicate with you, get you answers to questions in real-time, and literally be on its toes just to cater to your needs 24/7. 🕰️
Well, a human assistant might not be possible but looks like Google has got something for you just as close but for free!
No, it’s not the typical Siri and Alexa kind of assistant, this one is unique. As the Co-founder and CEO of DeepMind, Demis Hasabbis would say – The future of AI Assistants.
“For a long time, we have been aiming to build a universal AI agent that can be truly helpful in everyday life.“
Demis Hasabbis, Co-Founder & CEO – DeepMind
For Google bringing AI to life wasn’t a vision they thought of yesterday, it has been in the talks and building for over a decade now. Gemini, which has been a multimodal AI assistant since the beginning itself, is a result of that.
Gemini was a transformative experience in itself, but with Project Astra, we are moving a step further and hopefully entering the new era of AI assistants, from little that is there about Project Astra.
What is Project Astra?
Astra (अस्त्र) is a Sanskrit word picked up from hindu mythology with the meaning, ‘supernatural weapon.’ I see the effect that Google was going for here, and by looking at the tech, the name seems quite promising as per the features of this AI assistant as well.
“An agent like this has to respond to our complex, dynamic, and ever changing world, just like we do,” says Hasabbis in Google I/O ‘24. He further adds that it would need to take in and remember what it has seen so it can understand the context and take action.
To do so, it would have to be proactive, teachable, and personal, so you can talk to it naturally without any delay or lag.
That’s exactly what Project Astra aims to do and has already done to some degree.
In simple words, Project Astra is a universal AI agent built by Google which can answer your queries in real-time through images and videos that it sees and can also remember things that it has seen within the surroundings or talked about.
Gemini Project Astra is ‘almost’ human 🫀
Built on Google’s Gemini models and AI trends, Project Astra is more or less an extension of Gemini or it could be called ‘more human’ Gemini.
Whilst it is still an AI technology talking, its advanced features can also give you a false sense of companionship, which might be fortunate or unfortunate – we are yet to find out.
Till then, here is all you need to know about AI assistance.
Just like a weapon, it makes you feel equipped
Project Astra lives by its name, and is different from the already existing AIs and voice assistants as stated by the journalist Anuj Bhatia, who got to give it a try.
“Astra’s capabilities go beyond what we have seen in existing AI assistants. The Google researchers told me that Astra uses built-in “memory,” meaning after it scans the objects, it could still “remember” where specific items were placed.”
Anuj Bhatia, journalist
Along with providing solutions to your queries it can also remember things for you. While the memory window of Astra is right now very small, it is yet helpful, and if it gets expanded, then the possibilities are insane. You probably won’t have to worry about where your keys are.
It can stay in your phone and also glasses
When Astra would be accessible, it would see the world with you, just as you see, and also learn from it. This is an AI assistant that you can access not just from your phone but also through smart glasses.
Sounds hard to believe? Have a look while Google shows off Astra through Meta’s smart Ray bans.
What can Project Astra do for you?
The better question is, what can AI not do for you.
Astra is based on the most advanced and modern AI tech which includes generative AI, multimodal AI, and computer vision.
Using computer vision, your phone’s camera or through your glasses, it captures the image, or what’s there in front of it, then amalgamates and applies multimodal and generative AI to answer your questions.
As simple as it sounds right now, the tech in the back end is quite the opposite. But this is what you need to know about its functioning at the surface level.
Keeping that in mind, here are all the things Astra can do for you, some of them surprised me as well.
Explain Physics Drawings
As shocking as it might sound, Astra is capable of understanding even poorly drawn images which are related to Physics. Be it a roughly drawn Albert Einstein or an image describing the application of gravity.
It will be able to recognize those, and answer accordingly as well.
I wish I had access to Astra in my high school!
Can solve mathematical problems
Astra sounds like a real geek because not just physics but it can solve mathematics problems as well that too within seconds. From questions involving factorization to understanding graphs, and being able to comprehend them – Astra can really do it all.
Recognize things/landmarks/monuments/literatures from drawings
Your poorly drawn sketches will not be scrutinized or made fun of for being unrecognizable.
You know why?
Because even if your Taj Mahal does not have swiftly drawing edges or even if your zebra looks more like a stick figure without structure, Astra would be able to figure out what it is. You have got a cheerleader for yourself.
That is not it, but through the sketches it can also make out which literature or story you are talking about. Even your stick figures of Romeo and Juliet with a poison bottle and a dagger would be recognized by Astra as the famous play of Shakespeare.
Tell your location through the surroundings
Among all the things that Astra can do, the most surprising one was that it can share your location just by a look at your surroundings and where you are standing. Now that is brilliant not just from the perspective of how advanced it is but also from the eye of safety.
You will know which neighborhood you are in even if it is the most random street with buildings. Now that might be a hyperbole, but from what we see right now, it feels real, and very possible.
It can converse naturally
Many journalists, who got a demo of Astra, including Anuj Bhatia and Kerry Wan, say that it is far from human, but they also recognize that this ‘most human AI’, we have got so far, which I believe is a huge leap from generative AI tools like ChatGPT.
I agree with the fact that it is not a human, and it can never be, but I also acknowledge everything that it brings to the table as an AI. Top of them being its tone and way of talking.
“Perhaps more notable was how natural the AI sounded as I panned the Pixel 8 Pro camera around and asked random questions about various objects in the room. The natural-sounding voice goes hand in hand with the Storyteller and Pictionary capabilities, both of which keep children, students, and people who have time to spare entertained.”
Kerry Wan, Senior Reviews Editor at ZDNET
While Astra’s conversational skills are earning it some brownie points, let’s nor forget that its response timing is praiseworthy as well.
DeepMind’s team shared that engineering something to be conversational is a huge challenge, keeping that in mind, it is commendable how talking to Project Astra feels like talking to a fellow peer. The response time is significantly better than other AI tools and you can feel that the conversation is flowing.
Project Astra AI agent is just the beginning 🌌🚀
“I think you will agree it’s amazing to see how far AI has come especially when it comes to spatial understanding, video processing, and memory. It is easy to envision a future where you can have an expert by your side, though your phone.”
Demis Hasabbis, Co-Founder & CEO – DeepMind
While Project Astra is an exciting venture that we are all looking forward to, it is also just the beginning. It definitely has insane potential, but there are some curvy edges that still need to be sharpened for it to make it through the real world.
Either way, I am looking forward to getting my hands on it and making AI work for me.
