10 modern feats of automation that were just humans in disguise

Amazon grocery stores phased out Just Walk Out on Tuesday. Amazon Fresh’s technology allegedly used an automated system of scanners and cameras to ring you up without any checkout at all. You simply scanned a QR code when you walked in, got your groceries, and walked out. Incredible, right?

Well, it turns out that Amazon heavily relied on a team of over 1,000 associates in India to power this automation feat. In 2022, roughly 70% of Just Walk Out customers needed to be reviewed by human associates.

A company named Presto Voice sold its “drive-thru automation” services to Carl’s Jr, Chili’s, and Del Taco. The company claimed to let AI take your order at the drive-thru, and offered to replace drive-thru workers with its technology.

SEC filings recently revealed that humans are required to help with over 70% of Presto’s orders. Presto used offsite workers in the Philippines to listen to customer orders and manually insert them into a restaurant’s computer systems more than half the time. This was another bait and switch, where remote workers posed as true automation.

Facebook released a virtual assistant named M in 2015. It spoke with users over Facebook Messenger, and the AI could allegedly book you movie tickets, tell you the weather, or even order you food from a local restaurant. If you’re curious how Facebook developed a generative AI chatbot in 2015, the answer is that it didn’t.

There was a very early AI system in the works around M, but mostly the assistant was just human operators stepping in and performing tasks for people. The idea was that eventually, M would get smart enough to operate independently, but it never did. M was only released to a small group of beta testers because Facebook couldn’t “afford to hire operators for the entire world to be their personal assistant.”

You might remember that impressive Gemini demo in Dec. 2023. It showed how Gemini’s AI could allegedly decipher between video, image, and audio inputs in real-time. It was an impressive showing that four months later is still nowhere close to reality. So what happened?

That video turned out to be doctored up. For one, the video was sped up and edited. Not to mention, humans actually fed Gemini long, text and image prompts to produce any of its answers. Gemini is fully released today, but it can barely even respond to controversial questions, let alone do the backflips it performed in that demo.

This is a marketplace from Amazon literally named after the phenomenon of Mechanical Turks. A little too on the nose? It’s a marketplace where developers can hire humans to complete tasks computers have a hard time completing. The tasks could be almost anything, from labeling images to training artificial intelligence to moderate online content.

Real people could be paid as little as one penny for a task, largely for tasks where humans posed as automation. There were horror stories that surfaced around this where gig workers were exposed to graphic images or transcribing Social Security numbers.

One notorious use of Amazon Mechanical Turk was Expensify in 2017. The expense reporting company had a feature where you could take a picture of a receipt and expense it for you. The app would automatically verify that it was an expense compliant with your employer’s rules, and file it in the appropriate location. Can you guess what happens?

When the AI failed, Expensify said it used a team of secure technicians to file the expense on your behalf. That turned out to be Amazon Mechanical Turk workers in some circumstances, according to receipts spotted by Twitter users. That means real humans were viewing private information such as credit card numbers, full names, addresses, food orders, and more. It’s unclear how many of these expense reports were filed by humans.

Before X.ai was Elon Musk’s AI company, the AI startup had a rocky history. In 2016, X.ai offered virtual assistants that had access to your calendar and could correspond with you over email. However behind every email was a human, including one 24-year-old who spoke with Bloomberg.

Humans, posing as AI, responded to emails, scheduled meetings on calendars, and even ordered food for people. Once again, it was a case where the company claimed that the AI was in its early stages. In reality, it was barely doing anything, while humans did all the impressive parts.

Google used to let its computers scan your Gmail inbox for information to personalize ads, but then it vowed to stop. However, that didn’t stop the company from letting outsourced human developers take over that job.

Employees of Edison Software told the Wall Street Journal in 2018 how they had personally reviewed hundreds of user emails to build new features. Other companies allowed employees to read emails to train software and develop other marketing tools.

One source called it the industry’s “dirty secret” that humans were reading peoples’ emails. “It’s kind of a reality.”

Elizabeth Holmes was recently sent to prison for massive fraud around her company Theranos. The company claimed to have created an automated blood testing device that could make accurate assessments of a patient’s health with a few drops of blood drawn from a finger.

However, it turned out to be nothing more than smoke and mirrors. There was no breakthrough in automated blood testing. During demonstrations for pharmacy executives, Theranos employees were secretly running tests on outside lab equipment. It was one of the biggest automation frauds in history, that sent Holmes to jail for an 11-year sentence in 2023.

Even modern AI companies are not excused from this behavior. Behind Scale AI, the $7 billion startup based in San Francisco, is 10,000 remote workers in the Philippines training its models, according to The Washington Post. These workers ensure that artificial intelligence models are operating accurately. They label thousands of images as various politicians and celebrities, edit chunks of text, and train video models to identify various objects.

This is the not-so-automated part of modern AI systems. It requires thousands of remote workers to train these systems, and these employees often get underpaid and working ridiculous conditions. Scale AI does work for firms such as Meta, Microsoft, and OpenAI.