Well, IBM’s Watson has returned to compete this time against two of the finest most accurate professional transcriptionists. Now why would a company want to do this you ask? The Company decided to determine which would be more accurate the new age of machine-based voice or the old text technologies and as an added bonus hoped to obtain some amusing mondegreens. Mondegreens are misunderstandings of not only song lyrics but also words in general and tend to be extremely strange if not funny.
The Company had the two transcriptionists “pinky sweared” that they never heard or would look-up the words to four very different song types and lyrics. Watson did not have to “pinky swear” presumably, Watson cannot lie or had any knowledge or ability to obtain the lyrics to the four songs. The test included four songs by four different artists.
This was no pure scientific experimentation as scientific experiments usually performed by professionals but only attempts to match apples to apples. To keep the testing on somewhat of a level field, a Capella group sang all songs so that Watson had no chance of being confused or distracted by music. Secondly, only the four lines of lyrics were sung not the complete songs.
Watson success did not equal the success he experienced on Jeopardy. Whereas, the professionals performed perfectly, meaning they made zero errors and zero missing words with all four-test lyrics.
Taylor Swift’s lyrics to “I Knew You Were Trouble” as challenge one caused Watson some problems. He had four errors and missed six words completely. Van Halen’s lyrics to “Running with the Devil” as the second challenge did not go as good as the first challenge. Watson ended up with nine errors and fifteen missed words. Maybe Watson did not like Van Halen because the third challenge to Elton John’s “Tiny Dancer” Watson performed better not significantly better but better. He had eight errors one less than Van Halen’s Test and only three missed words better than he did either with Taylor Swift or Van Halen. Brad Paisley’s lyrics to “Ticks” as the fourth and final challenge had mixed results. Watson had the most errors with these lyrics with twelve errors but remained relatively level at five missed words.
Remember all two of the transcriptionists transcribed all four lyric challenges with zero errors and zero missing words. At this point, the transcriptionists did win over Watson although they produced no amusing mondegreens. Humans won. Watson well Watson needs a little help understanding the words sung by a Capella group. Guess Watson may need a little tweaking before he transcribes lyrics again.