Caixin
Mar 30, 2019 05:01 PM
SOCIETY & CULTURE

Pilot Programs Are Spying on Kids in the Classroom: Part One

Photo:Sixth Tone
Photo:Sixth Tone

Editor's note: This is part one of a two-part series on face-scanning technology in Chinese classrooms. The second part will run next Sunday, April 7.

(Sixth Tone) — Jason Todd first discovered his school’s secret on the internet.

It was late September, less than a month after high school had started. Jason was idly scrolling through his news feed on the Chinese microblogging site Weibo when he saw a trending hashtag — #ThankGodIGraduatedAlready — and clicked it.

Under the hashtag, someone had posted a photo depicting a bird’s-eye view of a classroom. Around 30 students sat at their desks, facing the blackboard. Their backpacks lay discarded at their feet. It looked like a typical Chinese classroom.

Except for the colored rectangles superimposed on each student’s face. “ID: 000010, State 1: Focused,” read a line of text in a green rectangle around the face of a student looking directly at the blackboard. “ID: 000015, State 5: Distracted,” read the text in a red rectangle — this student had buried his head in his desk drawer. A blue rectangle hovered around a girl standing behind her desk. The text read: “ID: 00001, State 3: Answering Questions.”

Jason thought the photo was a scene from a sci-fi movie — until he noticed the blue school badges embroidered on the chest of the familiar white polos worn by the students. It was exactly the same as the one he was wearing.

“F**k, no,” he thought.

2

Photo: Sixth Tone

Jason is a 16-year-old student at Niulanshan First Secondary School in Beijing. He wears a pair of black-framed glasses and likes to read DC comics so much that he chose to use the name of one of his favorite characters for the sake of anonymity. If he hadn’t seen that image online, he wouldn’t have questioned the presence of the tiny white surveillance camera installed above his classroom’s blackboard. After all, Niulanshan never informed him — or any of its 3,300 other students — that facial recognition cameras were capturing their every move in class. In fact, it’s unlikely that the combined 28,000 students in the six other schools testing the same system know they are part of China’s grand artificial intelligence (AI) experiment.

In July 2017, China’s highest governmental body, the State Council, released an ambitious policy initiative called the Next Generation Artificial Intelligence Development Plan (NGAIDP).

The 20,000-word blueprint outlines China’s strategy to become the leading AI power in both research and deployment by 2030 by building a domestic AI industry worth nearly $150 billion. It advocates incorporating AI in virtually all aspects of life, including medicine, law, transportation, environmental protection, and what it calls “intelligent education.”

Following the NGAIDP’s release, Chinese tech companies have rushed to secure government support and investor funding for various AI projects, several of which are being tested in Chinese schools. Over the past two months, I interviewed a dozen students in schools that have installed different “intelligent education” systems and spoke to tech entrepreneurs whose companies are developing the systems used in those schools.

While advocates claim that using facial recognition to monitor students’ in-class behavior can accurately assess attention levels and help them learn more efficiently, most students I spoke with had a different opinion. They showed concern that the school never asked for their consent before harvesting their facial data. One student expressed anxiety at the idea that the times he was caught slacking off in class would eventually be used to determine his chances of attending his dream university. Other students disconnected the cameras at their school before final exams in protest.

1

In addition, teachers and experts question the extent to which facial recognition systems can improve student performance. Experts say there are many technological, legal, and moral barriers to overcome before facial recognition can be widely deployed in Chinese education. But the government’s AI push is already introducing this technology before settling such debates, leading experts to agree that regulations on the technology urgently need to catch up.

The ‘Class Care System’

In the upper left corner of the photo Jason found on Weibo, you can just make out the words “CCS Class Care System.” CCS is one of the flagship products of Hanwang Education, a subsidiary of Hanwang Technology Co. Ltd. Hanwang is a household name for China’s younger generation. Growing up, they watched catchy commercials for the company’s text-to-speech reading pen on television. Today, Hanwang builds hardware and software products that provide facial and biometric recognition services and optical character recognition, as well as air quality monitors and purifiers. Hanwang Education was founded in 2014 as part of an initiative to expand the company’s market into China’s education sector.

At the research center, which occupies a nondescript, three-story house in a residential estate in northwestern Beijing, Hanwang Education General Manager Zhang Haopeng rushes downstairs to greet me. He’s a busy man, currently helming the deployment of CCS in partner schools across the country. Fortunately, he’s managed to find time for an interview.

If transitioning from text-to-speech pens to facial recognition technology seems like something of a quantum leap, it’s really not, Zhang explains. Recognizing a student’s facial expression and reading out handwritten notes are variants of the same thing: pattern recognition. To the machines, faces and handwriting are just data, and identifying them is all about spotting patterns. Since 2015, Hanwang Technology has also been using deep learning, a type of AI that mimics the brain’s analytical models, to buttress its existing technology. “Hanwang is at the forefront of pattern recognition,” Zhang says proudly.

3

Zhang Haopeng, Hanwang Education’s general manager, checks the company’s “Class Care System”servers in Beijing, Jan. 2, 2019. Photo: Xue Yujie/Sixth Tone

At first, Hanwang struggled to monetize its cutting-edge facial recognition products, which it only used for commercial purposes, such as face scanners that log employee work hours or smart billboards that tailor their advertisements based on the gender of the viewer. Zhang tried unsuccessfully to push Hanwang into the education sector, marketing interactive whiteboards and an “e-schoolbag” tablet that acted as a multipurpose textbook. But once China announced the NGAIDP in 2017, Hanwang finally found its niche: education analysis.

The plan’s “intelligent education” section describes in detail how China’s government hopes to use AI to boost the country’s education system. Zhang reads me one paragraph from the guidelines without stammering. “So detailed. It’s like they wrote it with a [facial recognition] product right in front of them,” he says.

Zhang, who keeps a framed photo of his smiling children on his desk, says that parents in China are yearning for more information about their children’s school performance. It’s a country where test scores can make or break an individual’s future. Visit any Chinese primary school at the end of the day, he says, and you’ll see parents bombarding teachers with questions. “Did my son fall asleep during English class again?” he says, mimicking the questions parents might ask. “Did my daughter and her desk-mate talk too much during class? Should we separate them?”

Zhang says that for most Chinese parents, school is the only time they let their kids out of their sight. “Parents worry when their children aren’t around. They want to take care of every aspect of their lives,” he tells me. “But the teacher only has one pair of eyes.”

“Do you know the two types of students teachers pay the most attention to?” Zhang asks. “The smartest and the naughtiest.” Hanwang’s CCS technology was born from the desire to care for every kid in the classroom, even the “often-ignored, average students,” he adds.

Zhang shows me one of Hanwang’s CCS cameras. It’s the size of a mandarin orange, but it can recognize all 50 students in a classroom. “Just five years ago, this kind of technology was unimaginable,” he says, explaining how the camera homes in on students’ unique facial features to identify each individual — and in a much less obtrusive way than a fingerprint or iris scanner. “We can identify a person’s face from just one picture, even when the frame size is as low as 640 x 480 pixels,” Zhang says.

Even though CCS hasn’t received official approval from China’s Ministry of Education yet, it’s already been implemented as a pilot project in seven schools around the country since its launch in December 2017. In these schools, a white, dome-shaped camera is installed above the blackboard at the front of each classroom. Once per second, it takes a photo of the entire class and sends the footage to a server where Hanwang’s deep-learning algorithms identify each student’s face and classify their behavior into five categories: listening, answering questions, writing, interacting with other students, or sleeping. The algorithms then analyze each student’s behavioral data and give them a weekly score, which is accessible through a mobile app.

Zhang takes out his phone and logs into a user account on CCS’s mobile app. The account belongs to a teacher at Chifeng No. 4 Middle School in the city of Chifeng in northern China’s Inner Mongolia Autonomous Region. The interface allows teachers to view scores for every student in class. A green down arrow appears next to the student’s score when it decreases, and a red up arrow when it increases. A bar graph shows how many minutes the student spent concentrating, sleeping, or talking in class.

“The parents can see it, too,” Zhang says, tapping on a student’s name. “For example, this student’s report shows that he rarely volunteers to answer the teacher’s questions in class. So his participation in English class is marked as low. Number of questions answered: one,” Zhang reads from the AI-generated report. “This week, the student spent 94.08 percent of class time focusing. His grade average is 84.64 percent. He spent 4.65 percent of the time writing, which was 10.57 percent lower than the grade average.”

“What’s your takeaway from the report?” Zhang asks, before once again answering his own question. “This student doesn’t like to answer questions in English class, so maybe his parents and teacher should do something.”

On the app, teachers and parents are also able to see up-to-date photos of the classroom and check students’ behavior, just like in that photo with the colorful rectangles. Zhang plans to expand CCS to 100 schools around the country by the end of 2019 and, eventually, he wants to create a nationwide platform for all schools.

“Now do you understand the ‘care’ part of our product name? Not a single student is missed,” Zhang says, smiling.

He leads me to a tiny room where two dusty CCS servers sit under a desk. One, which costs 60,000 yuan ($8,900), can analyze facial recognition data from five to six classrooms simultaneously. The other can monitor 20 classrooms and costs 150,000 yuan. Wiring a single classroom costs around 30,000 yuan. Zhang won’t tell me how much it costs per school, but if the high school I went to installed CCS in its 36 classrooms, it would cost at least 1.38 million yuan.

But so far, none of the schools using CCS have paid that staggering price. The government has offered financial incentives to local education bureaus to encourage them to use big data and AI — incentives that cover the installation costs. “[Local education bureaus] couldn’t be happier to implement the [AI development] policy,” Zhang tells me.

But if the social media reaction to CCS is anything to go by, the technology isn’t convincing everyone. I ask Zhang about the fearful messages posted under last year's leaked photo of the children at Jason Todd’s school. “People are overreacting,” Zhang says with a slight smile. “CCS doesn’t violate the students’ privacy. We don’t share the reports with third parties, and you see that on the in-class pictures we send to the parents, all the faces other than their child’s are blurred out.”

Before I leave, I ask Zhang one final question: “Do the students know?”

“Of course. You can’t use it without their consent,” he replies.

“What do they think of it?”

“They hate it.”

Zhang told me that at Chifeng No. 4, students unplugged the cameras before the day of their final exams. The cameras captured everything until the last moment.

Illustrations: Wang Zhenhao; visual editor: Ding Yining; editors: Julia Hollingsworth, Chris Bolin, Clayton D’Arnault, and Matthew Walsh.

This story was published in collaboration with “The Disconnect.” It was originally published by Sixth Tone. To read the original, click here.

You've accessed an article available only to subscribers
VIEW OPTIONS
Share this article
Open WeChat and scan the QR code
GALLERY
Copyright © 2019 Caixin Global Limited. All Rights Reserved.