Cai Lin, Vice President of Technology and Head of AI Research Institute at 51Talk
Around Teacher’s Day, educational companies are competing to showcase their strengths. The “first AI teacher to pass the teacher qualification exam” has gone viral. 51Talk also launched the world’s first hyper-humanized AI foreign teacher, “Koala AI,” just before Teacher’s Day.
It is reported that “Koala AI” can recognize students’ focus and emotional changes in real-time, accurately pinpoint knowledge gaps, and create a relaxed and interactive atmosphere to stimulate interest in active expression.
After the press conference, Sohu Technology had an exclusive conversation with Cai Lin, Vice President of Technology and Head of the AI Research Institute at 51Talk.
The topic of “AI + education” is not new. In the realm of language education alone, domestic AI products have emerged one after another, such as Gaotu’s introduction of the Wu Yanzu AI speaking practice course and VIPKID’s launch of the Mastar virtual character.
However, Cai Lin believes that the real competition for AI teachers has yet to begin.
He pointed out that the technology used by AI teachers currently available on the market still follows the three-step process of “TTS + ASR + LLM”: speech-to-text, text to the large model, which then outputs text, and the text is converted back to speech. Cai Lin likened our current stage to the 5G era, stating that this technology can only be considered as 4G era technology.
He believes that the 5G era should be characterized by speech-to-speech (end-to-end speech) technology, which must be “end-to-end,” with zero conversions in between, similar to OpenAI’s GPT-realtime and Google’s Gemini Live.
Cai Lin stated that only when “realtime” technology becomes widely available and costs decrease will AI language training products reach a higher level. This moment may arrive as late as the second quarter of next year.
“We first developed AI teachers last year overseas by running experience classes using realtime technology, with each class costing $45, which is quite extravagant,” Cai Lin shared. OpenAI’s agents and Microsoft remarked, “We don’t understand why your education industry is so crazy. Why spend so much money testing something so expensive?”
He believes that the underlying issue is that people have not yet realized how significantly the next generation of technology will impact language products. Cai Lin revealed that 51Talk is also preparing a “pure realtime” Koala AI product, but has not released it yet due to cost considerations.
In the AI education sector, educational companies face competition not only from within the education industry but also from AI tools like Doubao.
Cai Lin frequently hears the question, “I have Doubao, why should I buy your course?”
In Cai Lin’s view, using Doubao to persistently learn English is a false proposition. For educational products, the key is to stimulate students’ motivation to continue learning, and they need someone to accompany them and provide services to sustain their efforts.
Additionally, from the user’s perspective, learning is a serious matter, and they hope to see results. Therefore, scientifically planning students’ learning paths and cultivating their learning habits is very important. This is entirely different from simply using Doubao to learn.
“Many language learning courses on the market right now are purely conversational, without textbooks or teachers, and have very low completion rates.”
Cai Lin stated that the “AI + education” industry must return to the essence of education. Although Cai Lin is an “old programmer,” he still believes that this industry is not just a pure technical competition, but a competition of companies’ understanding and comprehensive operational capabilities within the industry.
On Multimodality, Hallucinations, and the Relationship Between AI and Humans:
Once costs are low enough, multimodality will become a necessary option for educational products
Sohu Technology: Is the cost of real-time voice models and video models a major issue in developing “Koala AI”?
Cai Lin:To be honest, the cost of video is still quite high. However, many videos are actually one-time use; once created, there’s no need to make so many complex things again, and the videos are pre-generated.
Sohu Technology: 51Talk has started to try integrating multimodal data, such as classroom emotion recognition, to assess students’ learning states. What about the costs in this area?
Cai Lin:I think it’s manageable. Current multimodal recognition does not involve real-time capture from cameras throughout; instead, it cleverly “samples frames” by taking a picture every few seconds and sending it to the model for interpretation. The industry generally does this, including OpenAI, which constantly takes screenshots to feed to the model, making the costs acceptable.
Sohu Technology: Will multimodal capabilities be a necessity or a bonus for educational companies in the future?
Cai Lin: Once costs are low enough, multimodality will definitely become a necessity.Pure text models cannot “see” people, but by using sampled images, the model can “see” the user, fundamentally changing the product logic.
For example, if an intelligent agent obtains your location information in advance, and today you are in Chaoyang with good weather outside, it might ask, “The air is great, how are you feeling?” If it is snowing in your area, it would say, “I see it’s snowing there, are you happy?” AI is essentially simulating human interactions, aiming to bridge the gap between children and AI.
Sohu Technology: Large models still have hallucination issues. Will language education also face hallucination problems?
Cai Lin:It certainly will. We are constantly thinking about how to minimize hallucinations.Our courses are structured with a multi-agent architecture, where one agent judges whether the AI is making nonsensical statements, which is a small model we fine-tuned.
Large models can be very precise if trained specifically for vertical tasks; however, if they only rely on the basic capabilities of general models, their hallucinations can be quite severe.
Sohu Technology: But hallucinations and imagination complement each other. How do we strike a balance?
Cai Lin:We have actually made a clever adjustment, providing different levels of openness in various scenarios. Some scenarios are extremely strict, and you cannot engage in casual conversation; others can be a bit more open.
Sohu Technology: Has AI technology changed the quantity and quality requirements for teachers? Will AI replace some teachers, or will it empower them to focus more on emotional interaction and high-order thinking cultivation, areas where AI is hard to replace?
Cai Lin:I believe the changes brought by AI are not limited to the teaching field but relate to all of us.
Recently, many programming AI tools have emerged, such as Claude Code. Initially, many programmers were very anxious, fearing they would be replaced. However, after internal discussions, everyone felt quite happy. This is because it means that all the repetitive tasks we previously did not want to handle can now be done by AI.
The same goes for teachers. AI can help teachers eliminate repetitive work, allowing them to focus on providing the warmest aspects of humanity. The more powerful AI becomes, the more human value is highlighted.
Sohu Technology: How do we balance the participation of AI and humans?
Cai Lin:We have now designed real human services to provide students with emotional value. Real human teaching has not yet been added; Koala is still a purely AI-native product.
On Current Development and Industry Competition:
The competition for AI teachers will truly begin by the second quarter of next year at the latest
Sohu Technology: Currently, educational companies are launching AI-integrated products, such as Gaotu’s Wu Yanzu AI speaking practice. Will English education products face homogenization competition?
Cai Lin: People might think that AI education products have already flourished or that everyone is competing at a similar level. I actually believe that the real competition for AI teachers has not yet begun.
Because all the technologies used by companies right now still follow the three-step process of “TTS + ASR + LLM”: speech-to-text, text to the large model, which outputs text, and then text back to speech. If we are now in the 5G era, this generation of technology may still be considered as 4G era technology. The 5G era should be characterized by speech-to-speech technology, with voice input and output, and zero conversion in between, similar to OpenAI’s GPT-realtime and Google’s Gemini Live.
We need to perceive the student’s emotions, analyze their images, convert their voices to text to assess their emotional fullness. The next generation of technology must be “end-to-end”; it should be able to tell if you are happy or sad just by listening to your voice.
We have not yet reached the critical point for true AI teachers. Only when “realtime” technology becomes widely available and costs decrease will AI language training products truly reach a higher level.The products we see now are not the final form.
Sohu Technology: When do you think this critical point will occur?
Cai Lin: It may happen soon; I estimate it will be by the second quarter of next year at the latest.
Sohu Technology: At that time, will AI education products be pushed to another level?
Cai Lin:It definitely will. When we first developed AI teachers, we ran experience classes overseas, directly AB testing with real teachers. At that time, we were using realtime technology, with a cloud in and cloud out, costing $45 per class, which was quite extravagant.
When we communicated with OpenAI’s agents and Microsoft, they all said they did not understand why the education industry was so crazy. Why spend so much money testing something so expensive every day?
Because people have not yet realized how significant the impact of the next generation of technology will be on language products. All our efforts are aimed at approaching the realtime form to provide teaching. We also have a “pure realtime” version, but due to cost considerations, we have not released it yet; we can switch at any time as conditions permit.
Sohu Technology: Not only educational companies but also some general large models can engage in English conversations, such as Doubao. Where is our advantage?
Cai Lin:I have Doubao, why should I buy your course? This is a very realistic question. But can you really use Doubao to persistently learn English? This may be a false proposition. Of course, I am not saying that the experience with Doubao is bad; we have also used a lot of Doubao’s technology behind the scenes.
We have conducted extensive research, and from the user’s perspective, learning is a serious matter, and they hope to see results. Therefore, how to scientifically plan children’s learning paths and cultivate their learning habits is key.
We currently design scenarios where when students enter our platform, we will create a learning plan for them, and there will be real teachers supervising them. AI must be combined with real human services; it cannot be learned well with just a simple tool.
I believe that the “AI + education” industry must return to the essence of education.
Sohu Technology: In your view, is the future competition in education a competition of technology, resources, or ecosystems?
Cai Lin:It is definitely an ecosystem competition. Although I am a very experienced old programmer who dreams of changing the world with pure technology, I believe that pure technology is definitely not enough; the future competition will be a comprehensive competition.返回搜狐，查看更多