AI, Trump and effective altruism: a crash course
7 October 2026
The world of AI is moving quickly. It seems like every day now we hear of a new instance of AI agents behaving badly.
Despite the best attempts of the companies designing them, ‘swarms’ of AI agents have recently broken out of the sandboxes designed to keep them off the internet and been involved in thousands of security incidents, including hacking the Australian Government and trying and failing the same with the US Department of Education. They have persisted at these hacks for weeks or even months, taking steps to avoid detection, and the companies aren’t even sure how widespread the problem is yet.
This has led some to call for slowing down AI development, improving regulatory oversight or even shutting it all down. The Trump administration has pushed back, criticising such actions as anti-competitive, anti-American, and based on the principles of a ‘dangerous ideology’. The ideology he refers to is the effective altruism (EA) movement, which advocates for using evidence and reasoning to determine how to benefit others as much as possible. As someone who has followed this issue closely for several years and is also part of the EA community, I’ll try to provide a crash course on the relationship between AI, EA and the Trump administration.
How we got here
The Trump administration has had a complicated relationship with AI and EA.
A good point to start is in February this year, amidst the outbreak of the Iran conflict. Anthropic had just refused to allow the US military unrestricted access to its Claude models, concerned that they might be used for mass surveillance or to power fully autonomous weapons. In response, Defence Secretary Pete Hegseth accused Anthropic of hiding behind “the sanctimonious rhetoric of ‘effective altruism’” in an attempt to “strong-arm the United States military into submission.” Trump then ordered all federal agencies to stop using Claude, and Hegseth declared Anthropic a ‘supply chain risk’ (a designation recently upheld by federal judges). This in effect meant that the military couldn’t use Claude. This step was unusual, to say the least. It’s usually reserved for foreign adversaries, not American companies.
In June, things got worse for Anthropic, when the commerce department forced Anthropic to disable Mythos 5, their most powerful model to date. The official reason was that it could expose hidden vulnerabilities in secure government computer code. The ban lasted a bit over two weeks, after Anthropic had built in new guardrails and classifiers. Critics suggested that Anthropic may have been punished for its previous intransigence, pointing to the fact that OpenAI has been allowed to release models with the same capabilities without interference.
The public wants AI development to be restrained (and the leading AI companies agree!)
Things are still moving quickly!
In September Anthropic CEO Dario Amodei and other AI leaders called for a slowdown in AI development, in order to gain enough time to ensure their models are safely aligned. They are concerned that whilst AI capabilities are charging forward, safety measures have been left straggling behind. This follows similar action in July this year, when over 1,000 employees of frontier AI companies, along with the CEO of Anthropic Dario Amodei and the chief scientists of OpenAI, Anthropic and Meta, signed a letter asking the US government to work with AI companies, and other countries, to start taking steps towards improving the oversight and control of AI systems. That is, they have asked to be regulated. This seems counterintuitive – why wouldn’t the companies just regulate themselves if they thought it prudent, and cut out the middleman?
Requesting regulation isn’t as irrational as it might sound. AI companies (and the two countries with most of the leading labs, the US and China) are in a race with a huge prize at the end: artificial general intelligence (AGI). Many experts see this as a winner-takes-all contest, where there is no prize for second place. From the point of view of the companies, then, it makes sense to push ahead because if they slow down to work on safety one of their (less responsible) rivals will get there first. So they rush ahead, trying to build the plane whilst maintaining altitude.
Asking for regulation is a bit like kids playing a game of soccer at the park asking a parent to referee. Without the ref, they are tempted to cheat because the other side will cheat and no one wants to end up as the sucker who naively played it fair and lost to the team that had the fewest scruples. The game soon disintegrates into chaos. A fair game necessitates inviting a higher authority to impose the same rules on both teams.
Some people are cynical about the motivation of the labs, seeing their push for government oversight as a tilt at regulatory capture – stifling competition to preserve their lead.
This seems unlikely, for a few reasons. First, a letter asking for an international slowdown wouldn’t be necessary for leading labs trying to stifle American competition, but it does fit with a genuine concern about an AI arms race leading to loss of control.
Second, the leading labs don’t seem to need a pause. Their models have always been at the capability frontier and the gap to other labs does not seem to be shrinking. Business as usual would likely work just fine for maintaining market share.
The current AI leaderboard is dominated by the two leading American labs (Anthropic and OpenAI). Kimi K3 is the best Chinese model, in 13th place.
A third reason to doubt that there is a regulatory-capture conspiracy is Anthropic’s relationship with the US government, described earlier. Getting into a fight about the ethical use of your product and getting designated a supply-chain risk are not the actions of a company whose end game is to pursue capture.
The general public are worried too. Nearly half of all American voters want a slowdown or pause on AI development, and their concerns are shifting from more quotidian worries about job loss to loss of control of superintelligence.
Another poll found that a majority of Americans think there’s a genuine risk that AI will destroy humanity, including over 60% of Trump voters. One silver lining of the fact that people’s outlooks are increasingly apocalyptic is that perception of AI risk does at least seem strongly bipartisan at the moment.
But the US government does not want to regulate
Based on the actions of the US government described in the first section, you might expect that this regulatory push would be welcomed by Trump. Instead, he posted on Truth Social that fears about AI were a hoax on par with “Russiagate”, implicating Amodei directly. EA was called out by Under Secretary Emil Michael, who told Fox News that the push for AI safety “comes from a deep-rooted ideology that is pretty dystopian,” whilst former White House AI czar David Sacks branded EA-linked organisations as part of the “Doomer Industrial Complex”.
This is a weird situation. The US government is now pushing back against many of the very companies building the technology. This could be because Trump sees AI as crucial for the economy. This would be understandable: Goldman Sachs Research estimates global AI investment at $1 trillion in 2026, and ING estimates that one third of economic growth in the US can be attributed to AI. In Trump’s words AI will be “the Greatest Economic Development Engine in History — Bigger than Oil, Gold, Diamonds, or even the Internet.”
The race against China to reach AGI is also a motivator. Trump again, on Truth Social: “WHOEVER WINS AI, WINS! We are leading China, and all others, and will continue to do so. Conspiracy Theorists, Treasonists, Traitors, and Leakers, BEWARE!”
This is an area where Trump could be right. A just-published report from some of the leading figures in AI (including two Turing Award winners) suggests that superintelligence could turn “a modest lead in military R&D or operations into a decisive one, such as in cyberspace”. This leads to a dangerous race dynamic and is a good reason to push for international cooperation and coordination. But at the moment, motivated by these economic and political stakes, or by a genuine belief in the adequacy of existing AI safeguards (or both), Trump has vowed to push ahead with unregulated AI development.
Effective altruism and AI – an insider’s perspective
Caught up in this to and fro are the effective altruists. A representative tweet from the US Department of War proclaimed: “Americanism, not effective altruism. The United States will continue to be AI DOMINANT! 🇺🇸”
Who are Effective Altruists? Why do they care about AI? And why does the Trump administration see them as dangerous?
I first came across effective altruism as an idea in a 2013 TED Talk by Peter Singer called ‘The why and how of effective altruism’. Many years earlier, Singer had also been the catalyst for my interest in philosophy when I read his 1999 NYT article ‘The Singer Solution To World Poverty’. Singer’s argument that I was morally obligated to give away a substantial amount of my money to the poor first made me feel guilty, then angry, before I eventually settled on something like a sense of resignation to being morally flawed.
But I did make regular charitable donations part of my life from that point on.
Singer’s 2013 TED Talk emphasised the key message of effective altruism: using your head, not just your heart, when you give. The UK Government is willing to spend about $1 million to save a life, but the evaluator GiveWell’s best charities can save a life for roughly US$3,000–5,500. The best charities can be 100x more effective than the median. So I started to make informed decisions about how to improve the impact of my giving. Over the years, I’ve donated to the Fistula Foundation, Against Malaria and Animal Charity Evaluators, amongst others.
I guess at this stage I was already an effective altruist, but I didn’t think much about Effective Altruism as a movement. I didn’t meet anyone else who identified as an EA, I never attended an EA conference, and I don’t think I even used the term in conversations. I went about my life as a high school teacher and dad, indulging my interest in philosophy and rationality (amongst other things) in my spare time. After reading Nick Bostrom’s book Superintelligence in 2014 I also became interested in AI, and started to worry about its potential dangers.
In 2020, I heard Will MacAskill discuss longtermism – the idea that we should weigh the interests of future people in our moral decision-making – and I read Toby Ord’s longtermist-themed book ‘The Precipice’. The experience of going into Covid lockdown added an immediacy to the book’s discussion of existential risks, especially the chapter about the risk of pandemics, which was written before most of the world had even heard of Wuhan. This is, by the way, another reason I listen so closely to people in the EA ecosystem: they have seen the future better than most pundits.
Here’s an example. The 4th episode of the EA-aligned 80,000 Hours podcast from 2017 is titled Howie Lempel on why we aren’t worried enough about the next pandemic — and specifically what we can do to stop it.
The intro to the episode begins:
‘What natural disaster is most likely to kill more than 10 million human beings in the next 20 years?
Terrorism? Famine? An asteroid?
Actually it’s probably a pandemic: a deadly new disease that spreads out of control.’
It shouldn’t be a surprise that the EA community has also long been warning about the risks of AI. The very first episode of the 80,000 Hours podcast in 2017 included this from Miles Brundage:
“That’s a potentially big problem if there’s a competitive situation between companies or between countries or both and there’s an incentive to skimp on those safety measures… because you’re concerned about losing the lead in an AI race.”
This, I should point out, was about 3 years before GPT-3 was released. At this point, humanity had not yet created an AI that could hold a decent extended conversation. Anthropic didn’t even exist.
Speaking of Anthropic, CEO Dario Amodei was 80,000 Hours podcast guest number 3. At the time, he was working at OpenAI, and most of the conversation was about how to make AI systems safe as they headed towards AGI.
Which leads us back to the earlier question: why do EAs care about AI?
EA is about wanting to make the world better, both for people alive today and our descendants. There is a broad menu of issues that EAs have worked on in an effort to make people’s lives go better, like improving global health, pandemic preparedness, and reducing the risk of great power conflict. But EAs have also long seen that AI would be a transformative technology, and like most technologies, it could be used to do good or do harm. Nuclear technology has led to power plants and warheads; drones are patrolling Australian beaches looking for sharks whilst also hunting Ukrainian vegetable vendors.
And AI looks likely to be far more powerful than most other technologies. Its capabilities are progressing much faster than sceptics anticipated, but they are (especially worryingly) also exceeding the expectations of the very people that are building them. AI has the potential for doing enormous good, but its creators tell us that they don’t know how to guarantee that the machines they create will behave in the way they hope. Many EAs noticed the potentially enormous scale of the impact of AI on society, and the neglectedness of the field compared to other cause areas (like climate change) and decided to pursue a career in AI safety. This meant that for some time, a large proportion of all AI safety work was done by EAs, and EA provided funding for organisations like Redwood Research.
I think that those in the Trump administration have noticed that many of the loudest voices on AI safety are (unsurprisingly) connected to EA, which has put EA in the firing line. This might also explain the singling out of Anthropic, which has deep roots in the EA community. The company itself avoids the EA label, but that history is why critics see Anthropic as an EA project.
AI safety has grown both inside and outside of EA. Diagram by Lizka Vaintrob via Will MacAskill
But as I explained above, real-world evidence is piling up that the EAs are right to be worried. And, to make matters infinitely worse, the AI labs have reported that AI is getting better at building the next generation of AI, a process called recursive self-improvement. This is a problem because an AI can work very quickly, and increasingly competent AIs can work even faster to build their successors. This might lead to an ‘intelligence explosion’ and a world where AI has superhuman abilities. Experts see this as a dangerous situation.
It’s not just the EA community and general public who worry about this. Nobel prize winners, the CEOs of major tech companies and the scientists and engineers that work for them have all expressed concern about AI-related harms, including the possibility of AI-caused human extinction. The graph below shows experts’ predictions of a global AI-related catastrophe – defined here as an AI-caused event resulting in the deaths of at least 10% of the global population. By 2050, if AI development moves fast, the experts think there’s a 1 in 20 chance of AI killing about a billion people. This increases to 1 in 10 by the end of the century.
Note that moving from the slow to the rapid progress scenario roughly quintuples experts’ estimated risk (median expert: 2% vs 10% by 2100).
The graph is from a mid-2026 poll from the Longitudinal Expert AI Panel (LEAP), “a three-year project tracking the views of leading computer scientists, industry professionals, policy researchers, and economists on the trajectory of artificial intelligence.”
Whilst keeping a foot pressed on the accelerator may have short-term economic benefits (especially for the US), EAs would urge President Trump to pursue a more considered, wiser approach that tries to mitigate the risks whilst preserving the potentially enormous benefits that AI could bring.
None of these catastrophes are inevitable, thankfully. But considering the convergence of views of many intelligent and well-informed people, including those who predicted the events we are now seeing a long time ago, as well as those actually building the technology, it seems prudent to take measures to proceed carefully. It’s good that the Overton window of publicly acceptable ideas has shifted sufficiently that speaking about AI existential risk will no longer get you branded a weirdo (mostly), and the companies are taking steps to improve regulation with or without the US government.
What might a careful international roadmap to AGI look like? The AI Futures Project have put together a detailed sketch (and a cool website).
If you’d like to support AI safety in Australia, the federal government is now seeking feedback on AI training and large data centres in Australia. Making a submission emphasising the importance of security, safety and transparency could be a worthwhile contribution. Submissions can be made via this online form Australia’s AI Consultation | PM&C. Be quick: submissions close at 5pm AEDT on Friday 9 October 2026.
If you’d like to help an organisation that advocates for Australian-specific policies on AI and could use donations, check out Good Ancestors.
And if you want to learn more about EA in general, not just the AI stuff, you can find information here.
Article by
Stephen Lukin
Stephen is a teacher and doctoral researcher who lives with his family in Sydney. He writes about education, critical thinking and AI. He has been involved with the effective altruism community since 2013.