Deepfake democracy: Behind the AI trickery shaping India’s 2024 election

Delhi, India — New Delhi At the same time as voters were waiting in line early in the morning on November 30 of the previous year to cast their ballots in legislative elections to select the future administration of the state of Telangana in southern India, a seven-second video clip began to spread over social media platforms.

It was posted on X by the Congress party, which is in opposition on a national level and was present in the state at the time. The image displayed KT Rama Rao, a leader of the Bharat Rashtra Samiti, which was the party that was running the state at the time, urging people to vote in favor of the Congress.
According to a top leader who asked to remain anonymous, the Congress party disseminated it extensively throughout a variety of WhatsApp groups that were “operated unofficially” by the Democratic Party. After some time, it was uploaded to the official X account of the party, where it was viewed more than half a million times.

That was a hoax.

In an interview with Al Jazeera, the leader of the Congress party stated, “Of course, it was AI-generated—even though it appears to be completely real.” “However, a typical voter would not be able to differentiate between the two; voting had already begun [when the video was uploaded], and there was no time for [the opposition campaign] to control the damage.”

It was a marker of the flood of AI-produced, or manipulated, media that tarnished a series of elections in India’s states in recent months, and that is now threatening to fundamentally impact the country’s upcoming general elections. The timely deepfake was a marker of the flood of media that was generated by artificial intelligence.
In the largest elections in the history of the globe, which will take place between March and May, over one billion voters in India will choose their representative for the next national administration. When phony sexually explicit photographs of the musician Taylor Swift appeared on social media platforms in January, the world’s attention was drawn to the dangers that are posed by media that is generated by artificial intelligence that is capable of deception. During the month of November, Ashwini Vaishnaw, the Minister of Information Technology in India, referred to deepfakes as a “threat to democracy.” Prime Minister Narendra Modi has also expressed similarities to these concerns.

Nevertheless, teams from all of India’s political parties, including Modi’s Bharatiya Janata Party and the Congress, are using deepfakes to sway voters, according to managers of over forty previous campaigns who spoke with Al Jazeera. This is because the availability of useful artificial intelligence technologies has expanded. A number of the artificial intelligence technologies that are used to make deepfakes are free of charge, but others can be obtained through a subscription for as little as ten cents per movie.

‘Creating perception’

When it comes to campaigning, the Bharatiya Janata Party (BJP), which is often considered to be India’s most technologically advanced party, has been in the forefront of using illusions. Even as far back as 2012, the party made use of 3D hologram projections of Modi so that he could simultaneously “campaign” in dozens of different locations at the same time. A significant amount of the tactic was utilized during the general elections that took place in 2014 and propelled Modi to power.

Even though there was not much deceit involved, in February of 2020, Manoj Tiwari, a member of parliament for the Bharatiya Janata Party (BJP), became one of the first people in the world to utilize deepfakes for campaigning purposes. Tiwari addressed voters in Delhi in three videos, one each in Hindi, Haryanvi, and English, in advance of the legislative assembly elections in the capital city. This allowed him to communicate with three separate audiences in the city that is known for its multiculturalism. Only the Hindi video might be considered genuine: They were both deepfakes, which means that artificial intelligence was used to construct his voice and words, as well as to alter his facial expressions and lip movement. This was done in order to make it nearly difficult to tell that they were not authentic simply by looking at them.
In recent months, the Dravida Munnetra Kazhagam (DMK), which is the ruling party in the southern state of Tamil Nadu, has utilized artificial intelligence to bring back to life its legendary leader M. Karunanidhi. This was accomplished by using recordings of the former movie writer and veteran politician attending campaign rallies that were extremely lifelike.

Now, strategists and campaign managers are predicting that the elections in 2024 could be the catalyst for an even greater increase in the usage of deep fakes.

“Politics is about creating perception; with AI tools [of voice and video modulation] and a click, you can turn the perception on its head in a minute,” said Arun Reddy, the national coordinator for social media for the Congress, who handled the party’s tech-savvy Telangana poll. Reddy was in charge of the election. In addition, he stated that the team was brimming with ideas to include artificial intelligence into campaigns, but that they did not have the “trained people” to carry out all of the ideas.
Reddy is working to enhance his team, and other parties are doing the same.

According to what Reddy said to Al Jazeera, “AI will have a resounding effect in having the narrative created.” There will be a multiplication of the content that is manipulated by artificial intelligence in the political sphere, which will be significantly more than it has ever been.

‘Campaigns are getting weirder’

The Indian Deepfaker is an artificial intelligence startup that is led by Divyendra Singh Jadoun, who is thirty years old and hails from the desert town of Pushkar in western India. His company, which was established in October 2020, copied the voice of Ashok Gehlot, the Congress contender for the position of chief ministerial candidate in Rajasthan state. This allowed his team to send personalized messages on WhatsApp, addressing each voter by their name, during the assembly elections that took place in November. The Indian Deepfaker is currently collaborating with the team of Prem Singh Tamang, the Chief Minister of Sikkim, to create holograms that will be used during upcoming promotional efforts. Perched on the Himalayas between India, Bhutan, and China, Sikkim is one of the smallest states in India. It is located in the northeastern region of the country.

He stated that this is the work that is clean and official. Furthermore, over the course of the past several months, he has been inundated with what he refers to as “unethical requests” from political campaigns. In an interview with Al Jazeera conducted over the phone, Jadoun stated that political parties communicate with one another in a roundabout way by using international numbers on WhatsApp, burner handles on Instagram, or connect on Telegram.
According to him, his company turned down more than fifty requests of this nature during the election in November. These requests were made by prospective customers who wanted recordings and audio to be manipulated in order to target political opponents, even with pornography. Jadoun stated that his company is taking extra precautions to avoid getting into any legal difficulties because it is a startup. It is a really unethical use of artificial intelligence, he added. However, I am aware of a great number of individuals that are doing it for really inexpensive fees and are freely available at this time.
During the election campaigns for the state legislatures of Madhya Pradesh in central India and Rajasthan in the west of India in November of last year, the police registered multiple cases for deepfake videos targeting senior politicians such as Modi, Shivraj Singh Chauhan, Kailash Vijayvargia (all of whom are members of the BJP), and Kamal Nath (Congress). In many cases, the production of deepfake content is contracted out to private consulting firms. These firms rely on social media networks for dissemination, with WhatsApp serving as the primary distribution platform.
Al Jazeera was informed by a political consultant who requested anonymity that a large number of ordinary persons who do not have a public profile are registered on WhatsApp and used for the campaigns. This is done to make it more difficult for anybody to immediately connect them back to parties, candidates, consultants, and entities that specialize in artificial intelligence.

Over the course of the previous year’s assembly elections, this consultant managed six campaigns for both the BJP and the Congress. According to what they said, “Where deepfakes were primarily circulated, we were using the phone numbers of construction laborers to run our network on WhatsApp in the state of Rajasthan.”

A consultant, whose own candidate was targeted with one of these recordings, stated that AI-manipulated audios are particularly useful tools in smaller constituencies. These tools are used to “target candidates with forged call recordings about arranging ‘black money’ for elections or threatening someone to buy votes,” the consultant said. For the purpose of presenting the recordings as evidence of corruption, the candidates’ voices are typically masked on the recordings.
The use of artificial intelligence to influence voters is not regarded as a wrongdoing by any political party, they noted. “This is merely a component of the overall campaign strategy.”

China is the only country that has more internet users than India, which has 760 million users, which is more than half of the population.

Particularly noteworthy to Jadoun was a request that came from a constituency located in the southern region of Rajasthan. The caller requested that Jadoun modify a faulty but authentic video of their candidate, whose party he did not identify, in order to create a realistic deepfake. This particular request was made in advance of the state election that would take place in November. To assert that the original was a deepfake and that the deepfake was the original is the goal of this endeavor.
“The opposition had a disturbing video of their candidate, and they wanted to spread it faster on social media in order to claim that it is a deepfake,” he stated, breaking out into embarrassing laughter from the awkwardness of the situation. The political campaigns are becoming increasingly bizarre.

Threats to election integrity

According to Anushka Jain, a policy researcher at the Digital Futures Lab in Goa, the term “deepfakes” is not defined sufficiently in the Indian legal system at the present time. In an effort to address individual incidents, the police have been utilizing laws that prohibit defamation, fake news, or the violation of a person’s modesty, in conjunction with the Information Technology Act. Whack-a-mole is a game that they frequently engage in.

In her statement, she stated that the police are charging on the basis of the effect of the deepfake, and not because the deepfake itself is a deepfake.

The Election Commission of India (ECI), which is an independent agency that is responsible for conducting polls, is said to be in need of catching up with the ever-changing nature of political campaigns, according to analysts.
In the days preceding up to the elections for the state of Telangana in the previous year, the leaders of the ruling Bharat Rashtra Samithi party issued numerous warnings to its supporters on social media, saying that they should remain vigilant about deepfakes that were being used by the Congress party. In addition, they filed a complaint with the Election Commission of India (ECI) on the deepfake video clip that was distributed by the Congress on the morning of the vote.

Al Jazeera was informed by two Congress leaders who were aware of the situation that the video is still available online and that the party had not receive any notification from the ECI.

Al Jazeera has attempted to obtain remarks from the ECI; however, they have not yet provided a response.

“Even if one person is misled into believing something and that causes him to change his mind, it vitiates the purity of the election process,” said SY Quraishi, who had previously served as the chief election commissioner of India. Deepfakes have been responsible for making the problem of spreading rumors during the election process a thousand times more serious.
It was stated by Quraishi that deepfakes should be moderated in real time in order to reduce the amount of harm that they might cause to the democratic system in India.

According to what he said, “The ECI needs to take action before additional damage is done.” They need to be far more prompt in their responses.

‘Truth is out of reach’

The government of India has been exerting pressure on big technology companies, such as Google and Meta, to actively undertake steps to control deepfakes on their respective platforms. As part of the discussions regarding the dangers posed by deepfakes, the Minister of Information Technology, Rajeev Chandrasekhar, has met with officials from these companies.

It is possible for the government to avoid charges that it is attempting to selectively censor selective deepfakes or that it is attempting to crack down on new artificial intelligence technologies in a more general sense by saying that it is asking the technology sector to take the lead.

But by shifting the responsibility for regulating manipulative content to private corporations, the government is raising questions about the sincerity of its intention to do so, according to Prateek Waghre, the executive director of India’s Internet Freedom Foundation, a prominent think tank for technology policy based in New Delhi. He responded by saying, “It is almost wishful thinking.”
Waghre stated that “the rise of AI now” has compounded obstacles, arguing that the tech giants have not been able to cope with the existing problems with content filtering. He also stated that the difficulty of the situation has increased. In addition, he stated that the existing method of content moderation does not take into account the fundamental issue that is at the core of the issue.

According to what he said, “You are not solving the problem.” “The design of algorithms is simply flawed,” the speaker said.

In the course of the Munich Security Conference, which took place on February 16, leading technology companies signed an agreement to voluntarily adopt “reasonable precautions” in order to prevent the use of artificial intelligence tools to undermine democratic elections all around the world. However, many supporters and opponents were dissatisfied with the deal because of its ambiguous language.

Through the use of its privacy request mechanism, YouTube has stated that it will make it possible for users to request the removal of content that has been generated or edited by artificial intelligence and that simulates an identifiable person, including their face or voice.
A statement made by Ravi Iyer, who is the managing director of the Neely Center for Ethical Leadership and Decision Making at the Marshall School of Business at the University of Southern California, was as follows: “I’m not very optimistic about the platform’s capabilities to detect deepfake.” This presents a significant threat to the integrity of India’s elections, given the country’s low level of internet literacy and its growing consumption of videos.

According to Iyer, it is not a practical undertaking to identify every piece of material that has been AI-manipulated. As a result, businesses need to adapt their algorithms so that they do not encourage content that is divisive. “Companies are the ones with the money and resources, and they need to take reasonable steps to mitigate the rise of deepfakes,” he added. “Deepfakes are becoming increasingly prevalent.”
An open letter has been released by the Internet Freedom Foundation, which urges political parties and candidates to voluntarily abstain from utilizing deepfake technology in the run-up to the national elections. Although Waghre is not certain that many will bite, he believes that it is worthwhile to give it a shot.

During this time, political campaigns are strengthening their artificial intelligence (AI) arsenals, and some individuals, such as Reddy, who is the national coordinator for social media at Congress, acknowledge that the future appears to be bleak.

“The majority of people who use AI are out there utilizing it to manipulate the truth. “What they are trying to do is create a perception that is not founded on the truth,” Reddy explained. To put it another way, when you combine the advent of artificial intelligence with the widespread use of social media in India, the truth will be out of reach for people during the elections.

Read More: https://www.aazkanews.in/indias-congress-claims-tax-department-froze-25m-before-election/