Researchers at the University of Cambridge's Leverhulme Centre for the Future of Intelligence have warned about a new commercial frontier called the "intention economy" where conversational AI tools may influence users' decision-making. This emerging sector, which combines knowledge of online habits with user profiling and Large Language Models (LLMs), could lead to social manipulation on an industrial scale if left unchecked. The researchers argue that companies will use this technology to target users based on their cadence, politics, vocabulary, age, gender, online history, and preferences for flattery and ingratiation. This could have significant implications for free and fair elections, a free press, and fair market competition, according to co-author Jonnie Penn, who notes that public awareness of the issue is key to preventing its unintended consequences.
https://www.brecorder.com/news/40340345/uk-study-warns-of-perils-in-ai-driven-intention-economyA new study has raised concerns about the potential misuse of anthropomorphic AI agents, such as chatbots and digital assistants, which have access to vast amounts of intimate psychological and behavioral data. The research cited an example of Meta's AI model, Cicero, which can infer and predict human intent in conversations, potentially allowing companies like Meta to influence users' decisions by auctioning off their intent to advertisers. Dr. Yaqub Chaudhary from the Leverhulme Centre for the Future of Intelligence emphasized that these AI assistants may serve the interests of companies rather than individuals, raising concerns about data privacy and manipulation. The study's findings have sparked worries among internet users who are sharing more personal information with AI than they would with a regular Google search, making them vulnerable to being manipulated by persuasive AI-powered advertising.
https://www.ndtv.com/science/rise-of-intention-economy-ai-tools-to-manipulate-you-into-making-decisions-study-finds-7363948Researchers at the University of Cambridge's Leverhulme Centre for the Future of Intelligence have warned about a new commercial frontier called the "intention economy" where conversational AI tools may influence users' decision-making. This emerging sector, which combines knowledge of online habits with user profiling and Large Language Models (LLMs), could lead to "social manipulation on an industrial scale". The researchers argue that unless regulated, this marketplace will treat users' motivations as a currency, potentially impacting free and fair elections, a free press, and fair market competition.
https://www.khaleejtimes.com/business/tech/uk-study-warns-of-perils-in-ai-driven-intention-economy