A nonprofit newsroom. Reader supported since 2022.

The Overseen

Investigating surveillance, data and automated systems, and the people on the other end of them.

Work

A nonprofit data labeler now scores its home workers through their webcams

Rungwell, a Toronto social enterprise whose labelers in Nairobi and Manila prepare training data for machine-learning systems, says Focus Assurance helps people keep their attention where they want it. Some of them say they do not know what it records, or what a low score costs them.

By Lydia Chepkoech

2 min read

One Focus Assurance session A line showing a focus score over a two-hour session. It drifts downward, crosses a threshold near 60, a break is suggested, and the score rises again. Focus score, one two-hour session 050100 Threshold: not disclosed “Take a short break?”
One Focus Assurance session A line showing a focus score over a two-hour session. It drifts downward, crosses a threshold near 60, a break is suggested, and the score rises again. Focus score, one two-hour session 050100 Threshold: not disclosed “Take a short break?”
A Focus Assurance session as labelers described it to The Overseen: the score falls when the camera judges that attention has drifted, and a break is suggested below a threshold the labelers were not told. Graphic: Noor Shalabi

Since March 20, labelers who work from home for Rungwell have done so with their laptop cameras switched on. Focus Assurance, a new feature of the Rungwell Tasks app, uses the camera to estimate whether a labeler is attending to the task on screen. When it decides that attention has drifted, it suggests a break. Each working session ends with a focus score.

Rungwell describes itself as a social enterprise run on not-for-profit lines. Its labelers, most of them in Nairobi and Manila, draw boxes around objects, transcribe speech and sort images for technology companies building machine-learning systems, and are paid by the task. The company says Focus Assurance does two things: it helps labelers see their own patterns, and it gives clients evidence that work done at home meets the standard of work done in its centers. Several clients that allowed only on-site work have now agreed to remote work, it says.

The company says the camera image is processed on the laptop and that Rungwell receives “a score, not a video stream.”

The Overseen spoke to six labelers who work through Rungwell at Home, four in Kenya and two in the Philippines. All six asked not to be named because they feared losing access to remote work. Most said the program had let them work from home for the first time; two care for young children, and one lives three hours from the nearest center.

None of them knew what happens to a low score. “When it says I am not focused, I don’t know if a person sees that,” said one labeler in Nairobi. Another said that after a week of low scores her queue had seemed shorter, and with it her pay, though she could not say whether the two were connected.

In its announcement, Rungwell said that in a three-month pilot with 212 labelers, quality returns from clients fell by 18 percent, and that 81 percent of participants said they would keep Focus Assurance switched on if it were optional. It did not answer when asked whether labelers can work from home without it, or whether scores are shared with clients.

“Focus Assurance is how we keep that door open,” Oren Feldman, Rungwell’s chief operating officer, said in the announcement, referring to remote work.

Correction, : An earlier version of this story said Rungwell’s pilot of Focus Assurance involved 221 labelers. It involved 212.

More from The Overseen