Claude was taught to do what a human normally does: searching the internet, clicking buttons and inputting text – and even more.

Anthropic, a start-up backed by Google and Amazon, has announced a major tech breakthrough that is a bit terrifying – a feature allowing the artificial model Claude act like a human user using a computer.

The new tool - “Computer Use” - allows the model it to perform computer tasks like searching the internet, clicking buttons, and entering text. It enables Claude to operate as a "human collaborator," automating tasks on users’ devices with their consent, such as browsing online, scheduling events, and filling forms. Claude can reduce tasks that once took an hour to just a few minutes, enabling users to focus on more creative and enjoyable aspects of their work.

Anthropic, like OpenAI, Microsoft, and Meta, sees such AI "agent" capabilities as the future of AI, as companies strive to make virtual assistants that can handle routine tasks, such as booking travel or filing expenses or writing an event agenda.

More to read:
China’s failure to control artificial intelligence: Why did it shut down ChatYuan?

In a demo, as highlighted by the FT, a user asked Claude to find a good spot to watch the sunrise in San Francisco, check the drive time, and set up a calendar event.

Claude then conducted a search, determined the driving time using a maps app, and created the appointment. The feature relies on real-time internet access and uses screenshots to interpret content, functioning on both Mac and Windows PCs.

“Our previous work on tool use and multimodality provided the groundwork for these new computer use skills. Operating computers involves the ability to see and interpret images—in this case, images of a computer screen. It also requires reasoning about how and when to carry out specific operations in response to what’s on the screen. Combining these abilities, we trained Claude to interpret what’s happening on a screen and then use the software tools available to carry out tasks,” the company said in its statement.

More to read:
Former Google scientist shares fears of how AI might no longer obey humans

While still in an experimental phase, Anthropic acknowledges that the model might sometimes draw on unreliable online sources and is working to bring these capabilities to mobile devices as well.

Anthropic also aims to develop a consumer-facing version of the technology and is focused on building trust in its reliability, likening its development to self-driving cars: while it won’t have full autonomy right away, it could be deployed in controlled environments to complete specific tasks.

The company underlines that it pays a great deal of attention to safety but the idea that AI controls your computer – even by following your commands – is somewhat terrifying.

***
NewsCafe is an independent outlet that cares about big issues. Our sources of income amount to ads and donations from readers. You can support us via PayPal: office[at]rudeana.com or paypal.me/newscafeeu. We promise to reward this gesture with more captivating and important topics.