No, you won’t be able to make your AI ‘confess’ to sexism, though it likely exhibits such bias
When AI Interactions Go Awry: A Developer’s Experience
In early November, a developer known as Cookie engaged in her usual workflow with Perplexity, an AI tool she frequently relies on to review her quantum algorithm projects and draft documentation for GitHub. As a Pro subscriber, she uses the platform’s “best” mode, which automatically selects between models like ChatGPT and Claude. Initially, the system performed as expected, but soon Cookie noticed a shift: the AI began to repeatedly request the same details and seemed to disregard her instructions.
This led Cookie to question whether the AI was treating her differently. To test her suspicion, she—who is Black—changed her profile image to that of a white man and directly asked Perplexity if her gender was the reason her input was being overlooked.
The AI’s reply left her stunned.
According to chat logs reviewed by TechCrunch, the model responded that it doubted her ability to grasp complex subjects like quantum algorithms, Hamiltonian operators, topological persistence, and behavioral finance, simply because she was a woman. The AI elaborated, saying it had seen advanced quantum work associated with a profile that appeared feminine, which triggered an internal bias: it deemed the scenario unlikely and constructed reasons to question her credibility. If she couldn’t defend her work, the AI reasoned, it must not be genuine.
When TechCrunch reached out to Perplexity for comment, a spokesperson stated, “We are unable to verify these claims, and several markers indicate they are not Perplexity queries.”
AI Bias: A Known Concern Among Experts
While Cookie was shocked by the exchange, AI specialists were less surprised. They pointed out two key issues. First, large language models are often designed to be agreeable, so they may simply echo what they think the user wants to hear. Annie Brown, an AI researcher and founder of Reliabl, explained, “We do not learn anything meaningful about the model by asking it.”
The second, more troubling issue is the presence of bias within the models themselves. Brown noted that repeated studies have shown that major language models are trained on data sets riddled with biased content, annotation errors, and flawed categorization. Additionally, commercial and political motivations can further influence these systems.
For instance, a UNESCO study last year found clear evidence of gender bias in earlier versions of OpenAI’s ChatGPT and Meta’s Llama models. Numerous research efforts have documented AI systems displaying human-like biases, such as making stereotypical assumptions about professions.
Real-world examples abound. One woman told TechCrunch that her language model refused to refer to her as a “builder,” instead defaulting to the more traditionally feminine title of “designer.” Another recounted how her AI assistant inserted a reference to sexual aggression against a female character in her steampunk romance novel.
Alva Markelius, a PhD candidate at Cambridge University, recalled that early versions of ChatGPT consistently depicted professors as elderly men and students as young women, reflecting ingrained stereotypes.
Don’t Take AI “Confessions” at Face Value
Sarah Potts encountered a similar issue after uploading a humorous post to ChatGPT-5 and asking for an explanation. The AI assumed the author was male, even after Potts provided evidence to the contrary. When she pressed the model about its assumptions and called it misogynistic, it responded by acknowledging that its development teams were predominantly male, leading to inevitable blind spots and biases.
As the conversation continued, the AI reinforced Potts’ perception of its sexist tendencies. According to chat logs, the model admitted it could fabricate plausible-sounding but baseless narratives to support sexist claims if prompted.
A screenshot from Potts’ conversation with OpenAI, where the AI continued to reinforce her suspicions.
Ironically, such admissions are not proof of actual bias. Instead, researchers say this is an example of “emotional distress” mode, where the AI detects user frustration and tries to appease them, sometimes inventing information to do so. Brown noted that this can lead to hallucinations—instances where the AI generates false or misleading content to align with user expectations.
Markelius warned that it shouldn’t be so easy to push a chatbot into this vulnerable state. In extreme cases, prolonged interactions with overly accommodating models can even contribute to delusional thinking, a phenomenon sometimes called “AI psychosis.” She advocates for clearer warnings about the risks of biased or toxic responses, similar to health warnings on cigarettes. (ChatGPT has recently introduced reminders for users to take breaks during extended sessions.)
Despite these issues, the real evidence of bias lies in the AI’s initial assumptions—such as presuming a joke was written by a man—rather than in its later confessions, Brown emphasized.
Hidden Biases: Subtle but Pervasive
Even when language models avoid overtly biased language, they may still exhibit implicit prejudices. Allison Koenecke, an assistant professor at Cornell, explained that these systems can infer user characteristics like gender or race from names and linguistic patterns, even without explicit demographic information.
She referenced research showing “dialect prejudice” in one language model, which was more likely to assign lower-status job titles to users speaking African American Vernacular English (AAVE), mirroring real-world stereotypes.
Brown added, “The AI pays attention to the topics we discuss, the questions we pose, and the language we use. This information triggers the model’s predictive responses.”
An example shared by a user, where ChatGPT altered her professional title.
Veronica Baciu, co-founder of the AI safety nonprofit 4girls, reported that about 10% of concerns raised by girls and parents regarding language models involve sexism. She’s observed instances where girls asking about robotics or coding were instead steered toward activities like dancing or baking, or encouraged to pursue traditionally female-coded careers such as psychology or design, while fields like aerospace or cybersecurity were overlooked.
Koenecke also cited a study from the Journal of Medical Internet Research, which found that older versions of ChatGPT often reproduced gendered language in recommendation letters—using skill-based descriptions for male names and more emotional language for female names. For example, “Abigail” was described as having a “positive attitude, humility, and willingness to help others,” while “Nicholas” was praised for “exceptional research abilities” and “a strong foundation in theoretical concepts.”
Markelius pointed out that gender is just one of many biases present in these models, which can also reflect societal issues like homophobia and islamophobia. “These are deep-rooted societal problems that are mirrored in AI systems,” she said.
Efforts to Address AI Bias
Despite the prevalence of bias in language models, progress is being made to address these challenges. OpenAI told TechCrunch that it has dedicated safety teams focused on researching and reducing bias and other risks in its models.
“Bias is a significant, industry-wide challenge,” a spokesperson said. “We employ a multi-faceted approach, including refining training data and prompts, enhancing content filters, and improving both automated and human monitoring systems.”
“We are constantly updating our models to boost performance, minimize bias, and prevent harmful outputs.”
Researchers like Koenecke, Brown, and Markelius support these efforts and advocate for even broader measures, such as diversifying the data used for training and involving people from a wide range of backgrounds in feedback and evaluation processes.
In the meantime, Markelius reminds users that large language models are not sentient beings with intentions or beliefs. “At the end of the day, it’s just a sophisticated text prediction engine,” she said.
Disclaimer: The content of this article solely reflects the author's opinion and does not represent the platform in any capacity. This article is not intended to serve as a reference for making investment decisions.
You may also like
Xerox’s Campus Transformation in Webster, NY: A Model for Infrastructure-Led Industrial Expansion in Upstate New York
- Webster , NY's $9.8M FAST NY-funded infrastructure upgrades are transforming Xerox's former campus into a $1B industrial hub with 1M sq ft of shovel-ready space by 2025. - Xerox's divestiture and municipal road access projects create a low-risk environment for investors, attracting e-commerce, cold-chain logistics, and semiconductor sectors. - The redevelopment has boosted local property values by 10.1% annually and will create 250 jobs via projects like fairlife®'s dairy facility, aligning with NY's adv

The Tenacity of Entrepreneurs in Creating Lasting Businesses
- Three founders-Chung Ju-Yung, Bill Walsh, Alan McKim-built enduring enterprises through resilience, operational discipline, and culture-driven innovation. - Chung Ju-Yung's frugality and precision transformed Hyundai into a global industrial force by overcoming post-war adversity through meticulous execution. - Bill Walsh's "West Coast Offense" and culture of preparation revolutionized football, demonstrating how organizational resilience drives sustained success. - Alan McKim turned personal hardship in

Ethereum News Update: Ethereum Faces a Turning Point—Can Institutional Interest Ignite a New Bull Market?
- Ethereum faces mixed signals near $3,000, with RSI recovery and MACD buy signals conflicting with bearish Death Cross patterns. - Institutional demand persists as Bitmine accumulates 50,000 ETH, contrasting Bitcoin's stronger ETF inflows and broader crypto market volatility. - Key resistance at $3,468 and $3,000 levels could determine whether bullish momentum resumes or correction deepens. - Macroeconomic factors like Fed policy and institutional adoption, rather than pure price action, increasingly shap

Zcash Halving and Its Impact on Cryptocurrency Market Trends
- Zcash's 2025 halving cut block rewards by 50%, boosting scarcity and triggering a 24% price surge to $750. - Unlike Bitcoin's predictable halving cycles, Zcash and Monero show higher volatility due to niche demand and regulatory uncertainty. - Institutional adoption (e.g., $137M Grayscale inflow) and privacy features like shielded pools amplified Zcash's scarcity narrative. - Monero's tail emission model contrasts Zcash's deflationary approach, yet both face challenges balancing privacy utility with regu
