There’s no shortage of experts and advice on AI, but there’s more to be learned by simply using it. Instead of using AI to write a paper, I asked Google Bard specific questions about the text of an article to help measure bias and the way language is used to shape a reader’s understanding of the past. This might be a way for teachers and students to use AI in the course of instruction.
I chose a 1,500 word article assigned to students by US History teachers to teach about The Equal Rights Amendment. “Phyllis Schlafly and the Debate over the Equal Rights Amendment” analyzes the debate over ERA and features a comparison between Phyllis Schlafly and Betty Friedan. It was published by The Bill of Rights Institute, a nonprofit educational organization that publishes educational resources for teaching US History and government. Founded in 1999 by Charles Koch, it is thought by many to promote a conservative viewpoint through its online resources, professional development seminars and essay contests. I thought Google Bard might help test that theory.
Below is a simple listing of the questions I asked Bard to help me analyze the text, followed by a narrative with links to Bard’s responses.
Scrolling through this example and reading the questions and responses should be enough of an introduction to spark ideas about applying AI to other sources and other types of analysis. Nothing beats just trying it yourself.
These are the questions I asked Bard.
- How much text used to describe the personal background, education and career of Phyllis Schlafly compared with the amount of text used to describe Betty Friedan’s background, education and career?
- Analyze the adjectives used in the article to describe Schlafly and the adjectives used to describe Friedan.
- How does the description of those who opposed the Equal Rights Amendment and those who supported it help the reader sympathize with Schlafly and the people who opposed the ERA?
- Does the article’s pairing of Schlafly’s success in defeating the ERA with the growth of conservatism make the reader assume that the growth of conservatism was a positive result?
- Does the word “explosion” used to describe the growth of feminist organizations in the late 1960s and early 1970s affect the reader’s impression of feminist organizations?
- How does the use of the word “traditional” used to describe Christian evangelicals, Mormons, Catholics, and other traditional groups to fight against the ERA help the reader sympathize or have positive feelings for the STOP ERA movement?
- Is it appropriate to describe Christian evangelicals, Mormons and Catholics “traditional”?
- How does this sentence help the reader sympathize with Schlafly? “After the defeat of the ERA, when Schlafly came under verbal and physical assault from supporters of the amendment, she converted STOP ERA into a pro-family, traditionalist organization named Eagle Forum”
- Was Phyllis Schlafly ever physically assaulted?
- What are the sources with which we confirm those attacks?
- What are the contemporary sources for the 1980 attack?
This is a narrative of why those questions were asked and what Bard answered…….
First thing’s first, I copied the whole article into Bard and asked it to just count the words used to describe Phyllis Schlafly’s personal background, education and career as compared with that used to describe Betty Friedan’s, perhaps one was written about more than the other. This wouldn’t lead to anything conclusive on its own, but could be a good way to start looking more closely at how the article approaches each woman.
Bard told me that the article has more than 1.5 times more words describing Schlafly rather than Friedan and suggested several reasons why this might have happened.
Analyzing the amount of text used to describe the personal background of Schlafly and Friedan
Bard also claimed that “the text was written by a historian who is sympathetic to Schlafly’s views.” Interesting, I did not ask that.
Beyond the amount of text devoted to each figure, I asked Bard to analyze the specific words used to describe Schlafly as compared to those used to describe Friedan. Bard gives me a list of the adjectives used to describe each, side by side, then suggests a conclusion as to how they compare.
Analyze the adjectives used in the article
Again, not conclusive on its own, but this is another data-point to consider in assessing the article’s bias.
Ok – that’s looking at the article as a whole. Time to dig into the text.
I had the impression that the article is written in such a way as to arouse the reader’s sympathy for the people who opposed the ERA. I copied the text of part of a single paragraph into Bard and asked it to analyze the text to see how it arouses the reader’s sympathy.
How does the description of those who opposed the ERA help the reader sympathize with Schlafly
Bard provided a detailed analysis of the paragraph confirming the conclusion implicit in the question, it explained how someone reading this article would see the supporters of The ERA as “out of touch”, and those who opposed it as more “reliable” and “sympathetic”. I had suspected this was so, it was nice to see Bard agree, but this could be confirmation bias. Bard also pointed out that some readers will not sympathize with Schlafly and opponents to the ERA.
When the article describes Schlafly’s success in stopping The Equal Rights Amendment, it pairs that success with the growth of conservatism so I wondered if this also represents an attempt to make the reader assume that the growth of conservatism is a positive result.
Pairing success against The ERA with the growth of conservatism
Bard agreed with that conclusion, which can help support the assessment that the article is biased, but Bard’s response could also reflect a confirmation bias by merely saying it agreed with what it was asked, it didn’t offer anything else to support that interpretation.
To reduce the “confirmation bias” factor, I simply asked Bard to analyze one sentence, asking how it can shape the way the reader thinks of feminists.
Does this sentence shape the way that the reader thinks of feminists?
Bard’s response explains how the sentence can leave the reader thinking that feminists are “progressive” and “forward thinking” or “too radical” or “aggressive”, depending on the reader’s personal beliefs and values, which means it really isn’t biased. But Bard goes further in its analysis, pointing out that the article does not account for different types of feminists. By limiting the description of feminists to only “cultural feminists and radical feminists”, the reader may think they are the only ones, thereby biasing the reader’s impressions of feminists.
Looking even more closely at the article, I asked Bard to analyze the use of just one word, “explosion”. I suspect that a reader’s negative associations with the word “explosion” will get extended to feminist organizations as well, therefore leaving the reader with a negative impression of feminist organizations.
Bard helped support that conclusion, but it also offered an explanation of the possibility of a positive association. Bard’s ability to quickly articulate the way the word “explosion” as it was used here can evoke energy, excitement, and change, but could also evoke chaos, destruction, and violence – depending on the personal beliefs and values of the reader.
In contrast to the word “explosion”, I think that the word “traditional” has the ability to elicit a positive reaction. The article pairs that generally positive reception with Christian evangelicals, Mormons and Catholics.
Bard provided three reasons how the article did this, but also included a slight disclaimer to explain how not every reader would make the positive association. Of course this assumes that the word “traditional” is applicable to those groups. Aside from my answer to that question, I asked Bard
Is it appropriate to describe Christian evangelicals, Mormons and Catholics “traditional”?
In this case Bard was helpful in providing explanations and interpretations that show how the word “traditional” can apply to these groups and how it cannot. More importantly, it explained how it is difficult to make such generalizations about these groups.
I thought that the article also cultivates the sympathy of the reader by describing how Schlafly was verbally and physically attacked yet continued in her efforts to create a “pro-family” and “traditionalist” organization, Eagle Forum.
How does this sentence help the reader sympathize with Phyllis Schlafly?
Bard agreed, adding that she endured these attacks while standing up for her values. But was Phyllis Schlafly verbally and physically assaulted?
Was Phyllis Schlafly ever physically assaulted?
That answers the question quickly – Bard describes the assaults, but does not provide the means to confirm it – an historian would need citations
What are the sources with which we confirm the attacks you just listed?
Bard provided newspaper articles and citations, it even included short descriptions of each. However, each citation is a 2016 newspaper article (all on the same date) about an attack that occurred in 1977. It didn’t provide any citation for the 1980 assault it mentioned in its previous response
What are the contemporary sources for the 1980 attack?
It turns out there are none, the attacks were mentioned by Schlafly in her autobiography.
This was an interesting experience showing the potential of AI to be used as a tool in text analysis.