Joanna Fu's blog

Fooling the ETS e-Rater Grader

Thu, 26 Apr 2012 15:02:21 -0400

In light of the several talks that we've had recently about scoring essays and other free text response sections on tests, I came across this example of a poorly written essay that could get awarded a top grade from the E.T.S. e-Rater system.

The accompanying article notes the following flaws in the technology:

1. The e-Rater cannot decipher truth, so a student will not be penalized for writing that the War of 1812 started in 1945.

2. Longer essays (even if they are filled with nonsensical sentences) are awarded higher scores than shorter essays. The e-Rater also prefers long sentences to short.

3. When in doubt, using a big word in place of a smaller word increases one's score.

It's fascinating to see how far technology has evolved so that we can now use an algorithm to measure the quality and development of an argument -- and it's also interesting to see how much further there is to go before a computer scorer can fully replace a human grader.

EdLab Review: Learn21

Wed, 18 Apr 2012 15:23:03 -0400

In 2010, the West Virginia Department of Education began Learn21, a website that offers interactive online games for students from pre-kindergarten to twelfth grade in the following subject areas: math, science, social studies, language arts, wellness, art, music, dance, theatre, agriculture, finance, and engineering.

Pros:
Teachers can use the games in class, and students can access the website when not in school if they would like to play some more or review a subject. Learn21 offers a wide variety of subject areas for study -- the humanities are prominently featured in addition to math and science -- and the entire site is free. Some of the games are quite innovative and intriguing, even for adults: for example, "The Great Flu" in Grade 9-12 Wellness attempts to educate players about the dangers and difficulties of containing flu pandemics.

Cons:
Such a large database of games and tools requires ongoing maintenance -- I noticed that several of the links to games (e.g., "Multiple Percents" in Grade 9-12 math resources) no longer work. The games linked on Learn21 are on third party sites, and the West Virginia Department of Education claims no affiliation or association with these sites -- which means that some of these games are very well developed and supported while others are not. In addition, some of the instructional tools and videos are slow to load and do not offer assistance when it would be helpful to the user.

Infographic: Where and How People Use Tablets

Wed, 11 Apr 2012 17:33:43 -0400

I've found that whenever I meet someone who owns a tablet, I want to know what usage habits are, and when they see a need for a tablet over a smartphone. The infographics below help to summarize things:

via: Adweek

The Importance of Habit Testing

Mon, 09 Apr 2012 13:11:03 -0400

As we’ve been working to revamp a couple of existing EdLab projects (Survey Sidekick, Pocket Knowledge), I’ve been thinking a lot about how people will be using these products and how we should design these tools so that people will prefer to use them over anything else. For example, with Survey Sidekick, I know that I've always preferred to begin the survey brainstorming process in a plain text editor before moving it to the actual creator. What features should Survey Sidekick therefore have to possess to compel me to use it through all stages of survey development? How do we redesign Pocket Knowledge so people think to continually return to it as a resource for research?

This TechCrunch article discusses “habit testing” -- measuring and collecting data about user habits in order to understand who your users are, what aspects of the product is habit forming, and why those qualities are habit forming.

The "Stupidity" of Games (and an Upcoming Talk)

Thu, 05 Apr 2012 14:27:50 -0400

I’m fascinated by games, though I sometimes feel as though I come to them from an outsider perspective – I barely played any while growing up (I didn't encounter a single console until the ripe old age of 25!). I’ve generally resisted playing any games on my iPhone, and I've tried but failed to see the addictiveness of popular games like Angry Birds, Words with Friends, and Fruit Ninja. But games still hold a special allure, and I love trying to understand why and how people play.

In the New York Times article "Just One More Game..." Sam Anderson discusses the recent proliferation of mobile games, as well as the addictiveness of these "stupid games" and their impact on productivity. In one memorable line he notes, "Plants vs. Zombies, ate up, I’m going to guess, a full 'Anna Karenina' of my leisure time."

I do feel that he fails to address one key point: that not all games are “stupid games.” One could make a point that all forms of entertainment produce enjoyable but unchallenging media. But a movie like “Wrath of the Titans” can’t (and shouldn't) be compared to “Mulholland Drive,” nor should “Twilight” be judged by the same standards as “Madame Bovary.” I would argue that “smart games” develop a strong affinity for characterization and storytelling in players and can hone puzzle-solving skills. I'd like to see how educational games can harness the qualities that these "smart games" possess.

Most desirable life skills for 2020?

Wed, 29 Feb 2012 11:48:25 -0500

According to a new report from the Pew Research Center’s Internet & American Life Project and the Imagining the Internet Center at Elon University, Millennials (those born between 1981-2000) will both “benefit and suffer due to their hyperconnected lives.” While a slight majority of respondents felt that the future for the hyperconnected will generally be positive, there are clear worries that this connection will also lead to a thirst for instant gratification and shallow decision-making abilities.

I can certainly empathize with study participants about the difficulties of living at a time where there is so much new content being created -- but what I found to be most interesting in the study (and also relevant to the EdLab) were the predictions about the most desired life skills for young people in 2020. They included:

- public problem-solving through cooperative work (crowd-sourcing solutions)

New videogame website archive at the British Library

Fri, 17 Feb 2012 14:38:34 -0500

Curators at the British Library are in the process of creating an archive to preserve computer game websites and gaming culture. They are specifically interested in collecting resources that "discuss the cultural and societal impact of computer games, for example research on the impact of games on children’s development."

They are looking for gamers and designers to suggest material they consider worthy of preservation -- what are some seminal games that come to mind?

One note: while the British Library's collection will be U.K.-centric, there is a similar archive, the "How They Got Game" project at Stanford University.

Failing well?

Fri, 10 Feb 2012 09:38:26 -0500

A top high school in London is holding a “failure week” featuring discussions, activities, and workshops discussing coping with failing and the negative effects of never experiencing failure -- in an effort to show that “it is completely acceptable and completely normal not to succeed at times in life.”

This brought to my mind our recent EdLab Seminar on SuperFutures – a career planning platform to help teenagers determine a job path and teach real-world skills from real-world industry experts. I’m thinking two things:

1. Should failure (and the skills of resilience and risk embracing) be some of these real-world skills that are taught to young adults?

2. Is it unrealistic for students to know precisely what their best fit career is while still in high school, and are we trying to provide too much of a safety net (and thus hindering development)? I know that many of my peers changed majors multiple times while in college, and others have changed career paths several times as well… but is the road half taken completely wasted? Or does that exploration help provide us with the space we need to truly grow and learn?

The Academic Journals Edition of Clash of the Titans?

Wed, 01 Feb 2012 23:17:06 -0500

In response to a post written by noted British mathematician Tim Gowers, nearly twenty members of MIT’s faculty have signed a pledge to boycott Reed Elsevier (the world’s largest scientific journal publisher) and are “refusing to publish, referee, or do editorial work unless they [Elsevier] radically change how they operate.” In addition, an online pledge against Elsevier currently has 2,892 signees from universities around the world, from both junior and senior scholars across a wide array of disciplines.

The frustrations from these scientists and academics boil down to three key points: 1) exorbitantly high pricing practices (running in the tens of thousands), 2) the practice of bundling subscriptions of less desirable journals to more valuable ones, forcing libraries to purchase journals that they don’t want and won’t use, and 3) Elsevier’s support of the Research Works Act which would prevent the free flow of information.

Live, from the 2012 Youth Media Awards!

Mon, 23 Jan 2012 11:01:20 -0500

I’ve been in Dallas, TX for the last few days to serve on the Stonewall Book Awards Committee at the ALA Midwinter conference. In addition to nonfiction and fiction book prizes, the committee selects a winner for the Mike Morgan and Larry Roman Children’s & Young Adult Literature Award -- given annually to "English-language children’s and young adult books of exceptional merit relating to the gay, lesbian, bisexual and transgender experience." I'm so happy that after months of confidential reading and discussion, I can finally reveal this year's winner: “Putting Makeup on the Fat Boy,” written by Bil Wright and published by Simon & Schuster BFYR, an imprint of Simon & Schuster Children’s Publishing Division.

The awards were presented at the ALA Youth Media Awards, which includes The Printz, Caldecott, and Newbery Awards (often dubbed "the Oscars" of the youth book world). Also presented was the winner for the Odyssey Award (best audiobook for children and/or young adults) and the Andrew Carnegie Medal (for excellence in children's video).

Gamifying the Library Experience

Sun, 22 Jan 2012 18:18:48 -0500

I attended a session this weekend on gamifying the library experience, based on a game launched by the University of Huddersfield called Lemontree. This game came about after the university’s library impact data project seemed to indicate a strong correlation between a student’s level of honors upon graduation from college and the time spent in the library/using library resources. Lemontree – a game in which users earn point for carrying out activities in the library and using library resources -- was created in response to this to try and increase library engagement amongst students. Some examples of ways one can earn points while playing and level up:

- points for entering the library
- points for logging into an e-resource
- points for borrowing books
- “early bird” or “night owl” badges are earned for coming into the library at certain times of the day
- users that ask good reference questions are given a bonus code by a librarian to enter for additional points

Mandatory Online Education in Idaho

Wed, 04 Jan 2012 15:27:54 -0500

This article in yesterday’s New York Times touches upon a whole host of issues including:

- the requirement for Idaho students to take two online courses to graduate high school

- the emerging (and unwelcome) presence of technology in classrooms

- lack of teacher development and training to effectively use these new tools

- the unproven effectiveness of online teaching

Of particular interest was the way the article juxtaposes high-tech classroom tools with one teacher’s more traditional use of the Socratic method in her high school English classroom. This contrast also seems to exist here at the Gottesman Libraries: we have Socratic Conversations coexisting with the EdLab under the same roof!

However, the consensus amongst educators isn’t that we ought to forbid technology in classrooms for the sake of human-only instruction; the same teacher that uses the Socratic method in her class also used a YouTube video to supplement the lesson on World War II.