Documents are located in P:/Othmer Library/Digital Collections - Internal Access/UX Testing/Transcription and Translation

User Testing Results 

Eight users were tested to understand and evaluate the use and efficacy of the transcription and translation features in the Digital Collections. 

Testing conducted by Annabel Pinkney, Digital Collections Librarian  

...

SUBJECT DEMOGRAPHICS OVERVIEW 

Researchers (4) Students (2)  Community Member (1)  Educator (1) 

Age 

Users tested were between the ages of 18 and 65, with three users under 25, one user between 26-35, one user between 36-45, one user between 46-55, and one user between 56-65. 

Education 

Five user’s highest level of education was a graduate degree, two users had an undergraduate degree, and one user’s highest level of education was a high school degree. 

Technological Proficiency 

Four users identified themselves as slightly advanced, one user identified themselves as average, and three users identified themselves as somewhat technologically proficient. 

Profession 

6 of 8 users tested were academic professionals, ranging from curators and historians to German language professors and writers. One user was an undergraduate student, and another was a concierge. 

Translations Experience 

Six users identified previous experience(s) using translation features in a digital collection, and two users reported no previous experience with translation features. 

...

TASK RESULTS ANALYSIS 

Task 1: You’ve heard that the Science History Institute recently digitized the Papers of Georg and Max Bredig and you want to view the digitized records. Please find a record from this collection. 

All users were able to identify a record from the collection. All users were started at the Digital Collections homepage and knew to use the search bar to begin searching. Search terms ran the gamut from “Papers of Georg and Max Bredig” and “Bredig” and “Max Bredig” and “Max and Georg.” Two users had typos in their initial searches which produced no results, but both were able to identify their mistakes and correct their errors. 

Two user searches resulted in the collection being the first result. They each selected the collection and understood that all the results which appeared on the collection page were records belonging to the archival collection.  

Two users found and used the Collection metadata field on a record to confirm the collection the work belonged to. 

Three users, while correct in their assumptions, did not utilize any metadata to confirm the records they located were part of the collection. Two users selected a record from their search results based on the title including some reference to the Bredigs and assumed it was a part of the collection. One user didn’t select a record at all but assumed all of their results were in the collection due to their titles and descriptions. 

From these results, it’s clear that users knew how to search our collections and could locate a record based on some criteria.  

Task 2: You want to do research about Svante Arrhenius and his relationship with Georg Bredig. Can you locate and read a letter written by Svante Arrhenius to Georg Bredig? 

All users were able to locate a work of correspondence between Arrhenius and Bredig. 

Users were given this prompt from the webpage where they ended from the previous prompt. 7/8 users used the small search bar at the top of the page to begin a new search, and one user navigated back to the homepage before beginning a new search. 

Again, search terms varied but most included both “Arrhenius” and “Bredig”. One user inserted the AND Boolean operator term. 

To read the letter, seven of eight users immediately opened the image before looking at the metadata or giving themselves an opportunity to find the transcription or translation tabs. It either took additional probing from the tester or a long time spent on the page for the users to find the tabs. 

All users demonstrated navigating back and forth between the image viewer and metadata.  

Task 3: Can you locate, view, and save the transcription and translation of a letter from Fritz Haber to Georg Bredig written in December 1920? (Note: Only 6 users were tested with this prompt; community member and educator personas were excluded.) 

All users were able to locate the specific record. One user used the date search facet to select the year, and others used 1920 as a search term. One user did neither but remarked that they assumed there was a way to filter by date. This user was still able to find the record from their results. Search terms varied much like the previous prompts. Three users used the small search bar on the results page and three users used the header search bar to complete their searches. 

All users selected the download button at one point while looking for a way to save the transcription and translation text. Three users state they would copy and paste the text into a word document after they realized they couldn’t download the text, one user additionally added they may take a screenshot. Two users selected the download .pdf option assuming it included the transcription and translation. 

...

FOLLOW UP QUESTIONS 

How does our layout compare to your expectations? Do you have any other general comments or feedback that you’d like to offer the Digital Collections team?

Six of eight users remarked very positively on the ease of use and the aesthetics of the website.  

Examples of constructive feedback on the site included: 

• Lack of options to apply multiple facets to a search or to purposefully exclude a facet in their search.  

• It currently doesn’t lend itself well to browsing, a user would need to know what they were searching for prior to entering the site. 

• Don’t always find the subject headings useful. 

How would you use transcriptions and translations in your research/schoolwork? What transcription/translation functionality would you need to do that?

Seven of eight users expressed positive feedback. Users highlighted that the translation and transcription being free and publicly accessible and credible due to its creation by a reputable institution were the most important factors for their usage. Both of which have already been achieved by the website. Several users noted that our features are luxurious in comparison to other digital collections. 

One user said it was cumbersome switching back and forth between the viewer and the transcription and because of their experience using other translation technologies our interface seemed outdated. 

Would you ever click “View”, assuming the transcription or translation would be next to the enlarged version of the image? Would you ever want this option?

Two users explicitly said yes this would be helpful for them. Three users said they wouldn’t necessarily need it for their work, but that it could be interesting and could see a use for it for others. Three users said they wouldn’t need this functionality. 

Would you ever want to download a PDF of the transcription or translation, or because the transcription field is in the metadata, would you be more inclined to cut and paste from it? 

Seven of eight users remarked that having the option to download the transcription and translation would be helpful to their work. They would need to download it for a variety of reasons depending on their work including work with citation managers. Five of eight users additionally or alternatively stated that copying and pasting would be sufficient for their work. One user stated that they would only download the images for their work.