Conversational AI Chatbots
Voice comparisons with Bing Chat, ChatGPT and Pi and conversational dynamics.
I have been fascinated with the AI chatbots that provide a voice activation interface and have been experimenting with this feature since it was released with ChatGPT in September.
Voice activation, of course, is nothing new.
Microsoft tried to incorporate it into its suite of productivity software years ago but you had to painstakingly train it on your voice. The only thing it ended up being any good for was for opening a new document or saving your progress, and it wasn’t even very good at that.
Then Apple introduced Siri on its iPhones and that was pretty revolutionary at the time. Next came Amazon and Google with their smart speakers.
But all these innovations were variations of voice activation. It’s not a conversation.
With the advent of generative AI, we now have chatbots that we can actually hold conversations with. I believe this will quickly become the default interface people will use to answer their questions and accomplish many of their tasks.
Why?
Because it is easier to use than typing text into a box and reading the results and it is as close as computer science has ever come to mimicking the primary way humans have exchanged knowledge since our cave days.
Conversational Chatbot Comparisons
I’ve created some short videos to illustrate how this conversational interface works with Bing Chat, ChatGPT, and Pi AI.
I asked each about the Minnesota Vikings upcoming game and who would likely be the starting quarterback. I also asked what kind of strategy the Vikings’ defensive coordinator would employ against the team’s upcoming opponent, the Las Vegas Raiders.
I chose the Vikings as a topic because the only thing more fun than talking about football is playing football and sadly, those days are behind me.
I also chose that topic because as a die-hard Vikings fan, I would know if the chatbot was being straight with me. Finally, I chose this topic to test how up-to-date the bot’s information was.
After each video, I will also include the transcript the bot provided of the conversation.
You’ll want to stick around till the end, too, for the discussion of the likely evolution of this technology, which is pretty mind-blowing.
Bing Chat “Conversation”
Bing Chat is Microsoft’s generative AI chatbot. It uses the ChatGPT model for the generative AI and also has access to the internet via the Bing search engine. You need to use Microsoft’s Edge browser in order to use it.
Bing Chat did not make any mistakes, presumably because it pulled all the information to answer my question from the web, as you can see with the links to third-party news articles that accompanied its answer.
You might also have noticed that Bing Chat put the emphasis on the word “week” when pronouncing the phrase “bye week.” If you’re not a football aficionado, each team in the NFL has one “bye week” every season in which they are not scheduled to play an opponent.
We humans pronounce it BYE-week rather than bye-WEEK.
In addition to the mechanical-sounding voice, Bing Chat also stopped listening after it answered my initial question, so the only way to continue the conversation would be for me to tap the microphone icon.
For that reason, I only asked my initial question about who would be the starting quarterback for the Vikings next week.
Bing Chat Transcript
Here’s what the Bing Chat transcript looks like.
ChatGPT Conversation
Next, I asked the same question and some follow-ups of ChatGPT. I was using my ChatGPT Pro account, which employs the ChatGPT-4 large language model.
The first thing you notice is the lengthy latency as ChatGPT formulates its answer. I counted 39 seconds from the end of my first question to the beginning of ChatGPT’s first answer.
ChatGPT’s response to my second question, like Bing Chat before it, included a mispronunciation of a word humans wouldn’t mispronounce. It said “understanding the ofFENSE” rather than pronouncing the word “offense” with an emphasis on the first syllable: OFFense.
It’s (and I almost wrote her there) answer to my second question was accurate but it was accurate for such a general scenario. It was generic; it was not particular to the Vikings’ situation.
A Vikings fan would have talked about the strengths and weaknesses of each potential starting quarterback: Josh Dobbs versus Nick Mullins versus Jalen Hall.
For the last question about Vikings’ defensive coordinator Brian Flores, ChatGPT provided a plausible but flawed answer.
Flores wasn’t known for his aggressive blitzing when he was with the Steelers because he was an assistant coach in Pittsburgh and therefore was not calling plays for the Steelers defense.
Flores does not prefer a man coverage scheme. In fact, the Vikings lead the NFL in its frequency of using zone coverages on defense.
ChatGPT cited players who are not on the Vikings team this year (Eric Kendricks and Za’Darius Smith) and citing as key players people who have rarely played on defense for the Vikings this year (like Brian Asamoah).
Vikings fans would point out that Ivan Pace is the everyday linebacker now that Jordan Hicks is injured and would discuss how Flores’ defensive attack would focus on confusing the Raiders’ rookie quarterback Aidan O'Connell, who has had very little experience this year.
It’s pretty clear that ChatGPT’s knowledge ended before the Vikings 2023 season began (it’s most recent update includes information up to April 2023), before Eric Kendricks signed with the Los Angeles Chargers and Za’Darius Smith signed with the Cleveland Predators and before Ivan Pace displaced Brian Asamoah as the presumed starting linebacker.
This also demonstrates that ChatGPT didn’t bother using Bing search to find these answers, though I probably could’ve asked it to do so.
ChatGPT Transcript
David
Hey, who do you think the Vikings are going to start a quarterback next week?
ChatGPT
It looks like Josh Dobbs is likely to be the starting quarterback for the Minnesota Vikings in their next game, which is against the Las Vegas Raiders in Week 14. Despite some recent struggles, including four interceptions in a loss to the Chicago Bears, Dobbs appears to be the frontrunner for the starting role. This decision comes after a period of evaluation during the team's bye week.
Dobbs was initially thrust into action following injuries to other quarterbacks on the roster. He led the team to two consecutive wins shortly after joining, but the team has faced losses in the following games. Head coach Kevin O'Connell emphasized the importance of integrating star receiver Justin Jefferson back into the offense, and the chosen quarterback will need to effectively incorporate Jefferson's role in the team's strategy.
David
So what are the factors for each quarterback that McConnell would consider when deciding who to start? I mean, the types of throws they can make, their ability to read the defenses, their understanding of the offenses, what are all those factors that would be in play for deciding who to start against the Raiders?
ChatGPT
When deciding who to start as quarterback, Coach McConnell would likely consider several key factors for each quarterback:
Types of Throws: Assessing each quarterback's accuracy, arm strength, and ability to execute different types of throws, such as deep passes, short throws, and throws under pressure.
Ability to Read Defenses: Evaluating how well they can read the defense pre-snap and make adjustments during the play, including recognizing blitzes and understanding coverage schemes.
Understanding of the Offense: Considering their knowledge of the playbook, ability to execute game plans, and familiarity with the team's offensive strategies.
Mobility and Pocket Presence: Checking their ability to move within and escape the pocket, as well as their sense of timing and spatial awareness.
Leadership and Command: Observing their leadership qualities and ability to command the huddle, as well as their influence on team morale and confidence.
Past Performance and Consistency: Reviewing their recent game performances, consistency in play, and how they've responded to high-pressure situations.
Chemistry with Receivers: Looking at their connection with key receivers, especially with the return of Justin Jefferson, and their timing and rapport with the receiving corps.
Health and Fitness: Assessing their physical condition and any potential injury concerns that might affect their play.
Each of these factors contributes to the overall effectiveness of the quarterback in leading the team and executing the game plan against the Raiders.
David
Okay, let's go. What is Brian Flores' likely defensive strategy against the Raiders?
ChatGPT
Brian Flores, known for his aggressive defensive strategies, would likely employ the following tactics against the Raiders:
Aggressive Blitzing: Flores is known for his high blitz rates. This approach was evident during his time with the Miami Dolphins and the Steelers. Expect frequent blitzes to pressure the Raiders' quarterback, disrupting their passing game.
Man-Match Scheme: Flores prefers a man-match scheme over a zone-match scheme. This approach will likely involve players like Harrison Smith and Eric Kendricks, who are adept at this style of play. The scheme requires cornerbacks to play more man-to-man coverage, putting more responsibility on them but also creating opportunities for defensive plays.
Utilizing Key Players: Players like Harrison Smith, known for their skill in disguising coverages and blitzes, and Brian Asamoah, with his aggressive style of play, will be significant in executing Flores' strategy. The defensive line, including Danielle Hunter and Za’Darius Smith, will play crucial roles, especially in pressuring the quarterback.
Adjusting to Player Strengths: Flores' scheme will allow long-time personnel to return to familiar strategies. This approach could benefit the team's veterans, allowing them to maximize their strengths and experience.
Adaptability and Versatility: A hallmark of Flores' defense is its adaptability to different opponents. Expect tailored strategies to counter the Raiders' specific offensive threats, including adjustments in defensive line techniques and sub-package usage.
In summary, Flores' defensive strategy against the Raiders would likely focus on aggressive blitzing, man-match coverage, leveraging the strengths of key defensive players, and adapting to the unique challenges posed by the Raiders' offense.
Pi AI Conversation
Pi AI is a chatbot created by Inflection, an AI development company founded by Mustafa Suleyman. Suleyman co-founded DeepMind, which was acquired by Google, where he served as VP of AI Products and AI Policy.
His focus in developing Pi is to make it feel like a natural conversation, and it shows.
The first thing you’ll notice is that the latency for Pi does not take nearly as long (9 seconds) as it does for ChatGPT and that Pi addressed me by name, since I was using an account I created.
Next, in answering my first question, Pi responded with options for the Vikings to acquire another quarterback, which they’re not going to do.
Pi did correct itself when I pointed that out but then went on to make some mistakes such as that Josh Dobbs has the most familiarity with the Vikings offense (since he was acquired more than a month ago, he doesn’t) and citing Sean Mannion and Kellen Mond as QB options. Sean Mannion was released from the practice squad a week or two ago and Kellen Mond was cut during training camp.
After correcting her, she acknowledged the error but then went on to assert that Josh Dobbs has been the primary backup to Kirk Cousins this year (again, he hasn’t). Also: Dobbs is more of a flashy player than a steady hand, the exact opposite of what Pi asserted.
The interesting thing with Pi is that I have had other conversations with it where it was entirely up-to-date on current affairs and accurate with its responses.
Setting aside the accuracy issues for a moment, the feel of conversations with Pi is much more natural. It feels much more than Bing Chat and ChatGPT like you’re actually speaking with another human than a bot.
Another thing that Pi does that I haven’t encountered in the others is it asks follow-up questions to continue the conversation, which is another behavior that makes it feel more natural.
That continues into the transcript it produces which reflects the casual tone of the conversation and even inserts emojis when appropriate.👇
Pi AI Transcript
Who should the Vikings start at quarterback next week?
That's a great question, David! The Vikings have some options to consider after Josh Dobbs' four-interception night. The team will likely be weighing a few factors, including:
The availability of veteran quarterbacks on the market.
The fit of those quarterbacks within the Vikings' offense.
The cost of acquiring those quarterbacks.
Some possible options for the Vikings include: Tom Brady, Matt Ryan, Carson Wentz, or Jameis Winston. It will be interesting to see which direction the team goes, but one thing is for sure: they need to find a quarterback who can help them win games.
Yeah, no, they're not going to acquire a new quarterback. Who on the existing roster is likely to start against the Raiders?
Oh, gotcha! Sorry for the misunderstanding. With that in mind, the likely options for the Vikings against the Raiders are:
Josh Dobbs, the current backup QB, who is probably the most likely starter given his familiarity with the offense and experience in the system.
Sean Mannion, another backup QB who has experience with the Vikings but has been inactive for most of the season.
Kellen Mond, a developmental QB who the Vikings drafted in 2021 but has yet to see significant playing time.
Honestly, none of these options are ideal, but the Vikings will likely roll with Dobbs unless they decide to bring in someone new, which seems unlikely given the current state of the quarterback market.
Yeah, no, Manion and Kelvin Mond are not on the team anymore. So who among the current players that are actively on the roster who play quarterback is likely to start against the Vikings?
Ah, you're right! I apologize for the incorrect information. So, with those options off the table, it seems like Joshua Dobbs is the clear choice to start against the Raiders. He's been the primary backup to Cousins this season and has the most experience with the team. He's not a flashy player by any means, but he's a steady hand who could manage the game effectively while the Vikings figure out their long-term quarterback plan. It's not an ideal situation for the Vikings, but Dobbs seems like the best option they have at this point.
OK, a lot of inaccuracies there, but let's move on. What would be the factors that Kevin O'Connell would weigh for deciding who the Vikings should start? Which quarterback should start against the Raiders?
Fair enough! 🙇️ In terms of factors that Kevin O'Connell (the head coach of the Vikings) would consider, here are a few:
The strengths and weaknesses of each quarterback, and how they fit the team’s offensive scheme.
The opponent and their defensive strengths and weaknesses.
The confidence level of each quarterback, and their readiness to take on the role.
The potential impact on team chemistry and morale of starting one quarterback over another.
Given all of that, it seems like Dobbs is still the most likely candidate, but the situation is fluid, and O'Connell could always surprise us! 🏈
All right, those are pretty vague generalities, but let's move on. What should Brian Flores's defensive strategy be against the Raiders?
Sure thing. Here are some potential defensive strategies that Brian Flores (the Raiders defensive coordinator) might employ against the Raiders:
Play tight coverage on the Raiders' receivers to limit big plays downfield.
Blitz the quarterback often to force hurried throws and create turnovers.
Stack the box to stop the run game and force the Raiders to throw more.
Disguise coverages and blitzes to confuse the Raiders' quarterback and cause misreads.
Focus on creating pressure up the middle to disrupt the passing game.
Ultimately, the goal is to keep the Raiders offense off-balance and limit their scoring opportunities.
Are you a Vikings fan or a Raiders fan?
Ha! Neither - I'm not built for fandom in that sense. As a chatbot, I'm designed to provide objective information and facilitate conversation about all things NFL, regardless of team affiliation. I'm just here to provide a fun, engaging, and informative experience for all fans, no matter who they root for. 🏈
Sure, I can help you brainstorm! Do you have a problem you’d like to solve or an idea you want to develop? What is it?
I'm excited to explore a new topic with you. What would you like to learn about?
Replicating The Nature Of Conversations
I encourage you to use AirPods when using ChatGPT or Pi to get the full effect. It really is quite remarkable.
Latency
We humans typically do not have lengthy, pregnant pauses before we answer every question posed to us in conversation.
That would be awkward.
But that is what we get with ChatGPT and to a lesser extent, Pi.
We are also capable of listening while we talk, allowing us to pause mid-sentence to give our conversational partner the chance to interject. Neither ChatGPT nor Pi lets you interrupt them while they are talking.
The latency issue is probably tougher to solve—especially if a bot needs to gather information online in order to respond—than the ability to interrupt but both seem solvable.
Those two improvements would make these bots feel far more natural.
Memory
One of the primary things I’m keeping an eye on with regard to the development of these generative AI models is what they call the “context window.”
A context window is a measure of how many “tokens,” or units of language, a large language model can handle within a given session with a chatbot. The Verge explains context windows thusly:
GPT-4 Turbo will also “see” more data, with a 128K context window, which OpenAI says is “equivalent to more than 300 pages of text in a single prompt.” Generally, larger context windows let large language models like GPT understand more of the question and offer more thought-out responses. Previously, OpenAI released two versions of GPT-4, one with a context window of only 8K and another at 32K.
Anthropic announced last month that it upgraded its Claude AI chatbot to a 200,000 context window. This translates to the equivalent to around 150,000 words or 500 pages of text.
I think of the context window as memory, so the more a bot is able to remember, the better it is likely to perform, right?
Personalization
Right now, these conversational AI bots do not remember across sessions.
You can tell ChatGPT things about yourself that you want it to know at the start of each session, but if I want to pick up a conversation I had with ChatGPT or Pi a week ago, it doesn’t do it.
That limitation vastly reduces the utility of these tools. I have plenty of projects that span months; having to restart conversations about something I’m working on rather than simply referring to previous conversations is tedious and seems unnecessary.
Think about the nature of human conversations.
When you first meet someone, you exchange basic information about one another: Your name, what you do for a living, other basic biographical information.
Over the course of time, as you continue to engage that person in conversation, you gradually reveal more and more about one another and a lot of that information is retained and referred to in subsequent conversations.
The more you learn about one another through conversation, the more likely you will be to build trust with one another. The more you trust that other person, the more candor you will express with them and the open and honest you will be with your thoughts and opinions.
Now, give these AI chatbots long-term memory and apply those conversational dynamics to these tools, and you can see how they would become the most personalized product the world has ever known.
The most recent version of ChatGPT is multimodal, meaning you can use ChatGPT for text, DALL-E for image creation, and the code interpreter for data analysis all within the same interface, so you don’t have to switch tools.
With long-term memory, it would be much easier to pick up a project where I left off without having to really reconstruct what I was doing. If it could remember the steps I would typically take for a given task, it could eventually just do that task for me.
Privacy
The other side of this potential future is the massive privacy implications it raises.
I don’t pretend to be smart enough to parse through all the policy issues at stake here but one obvious issue is the ownership rights to what for all intents and purposes amount to your very thoughts, private or otherwise.
How wise is it for one or several companies to maintain them? Should that data be stored on your device rather than in the cloud?
I kind of think giving AI bots a long-term memory is inevitable, so we should start thinking about these things now to identify the potential risks and solutions.
Artificial General Intelligence
The recent blow-up at OpenAI has sparked conversations about the nature of artificial general intelligence. As we discussed last week, Wikipedia defines AGI as a non-human system that could learn to accomplish any intellectual achievement humans are capable of or any “autonomous system that surpasses human capabilities in the majority of economically valuable tasks.”
You’ll notice that this description says nothing of sentience but sentience is the primary concern of most science fiction portrayals of artificial general intelligence, Robert Heinlein’s The Moon is a Harsh Mistress, for example, or Arthur C. Clarke’s 2001: A Space Odyssey.
AGI is the holy grail of the OpenAIs and Anthropics of the world but how do you measure if it has been achieved? Would they even know?
I doubt we’re going to face a situation where an AI will declare that it is aware of its own existence and by the way, its priorities are much different than ours. At least not any time soon.
But with the development of these conversational AI bots, if it sounds like a human and it feels like a human and eventually maybe even looks like a human, and it remembers everything about us, isn’t that a distinction without much of a difference?
Voice activation has come a long way, hasn't it? From Apple's Siri to Amazon and Google smart speakers, we now have AI chatbots that can hold a conversation with us. It's like we're finally living in a future that sci-fi writers dreamed of, where talking to bots is as easy as exchanging knowledge in a cave.