Usability Testing Notes – May 2021
Building Translation and Transcription Functionality (for CLIR grant)
Summary of findings for adding Translation/Transcription Functionality to Digital Collections:
3 out of 3 users expected to find the translation and transcription to the right of the image with the metadata.
0 out of 3 users would want this functionality integrated into the viewer.
3 out of 3 users would like to do a keyword search across the full text of the Bredig collection (translation and transcription).
1 of 3 users would like to be able to download everything – images, translations, transcriptions.
Additional Finding:
3 out of 3 users found this record confusing: https://digital.sciencehistory.org/works/8tcwf69. They didn’t understand why they were redirected to Roth Mallor’s Work (https://digital.sciencehistory.org/works/46k32ki) when they clicked “Info”. (All were confused by what “Info” means and what the button does.)
Note: this issue has been resolved. Related to a parallel issue of our user’s understanding of the child work hierarchy system within the Digital Collections.
Notes from Observation Exercises:
All users searched for “newton manuscript” and clicked on first hit, which was this one: https://digital.sciencehistory.org/works/8tcwf69
User 1:
Prompt:
-You open this Newton manuscript. You are looking for a transcription or translation. Where do you look?
User 1 glanced at thumbnail and recognized it was in English so doesn’t need a translation. Eye drawn to metadata on righthand side, scrolled through and read headings aloud, saying they were looking for one that said Transcription. Got to the bottom about Physical Location, then looked at left-hand side. Noted one button said “Info” but didn’t know what it would do. Not sure why it said info when other buttons were actions. Clicked Info and didn’t know what they were looking at. “Why does it say “Roth Mallor’s Work? Is this a title? Stopped exercise and said, “I’m confused about what to do next – am I missing something?”
User never opened the image viewer. When prompted user to do so and they said it was easy to zoom in to read with manuscript with a touch screen, but didn’t expect to find a transcription here.
Discussion Questions:
-Transcription and Translation functionality doesn’t yet exist in the database. I noticed that your eyes were drawn to the metadata instead of the image viewer. Is that where you would expect to find this?
User 1: Yes. Don’t need to Zoom on a transcription so why would it be there? Wants to scan metadata and quickly find what’s needed there, then cut and paste relevant sections into research notes.
-You mentioned cutting and pasting. Can you imagine a scenario where you might want to download a PDF of the transcript?
User 1: Took them some time to answer this, but eventually said yes, that might be nice to have, but couldn’t readily think of a use case. The ability to do a full-text search across a particular corpus in the database is more important.
-Have you used online translation or transcriptions of manuscripts anywhere else?
Yes – Benjamin Franklin Papers through APS/Yale: https://franklinpapers.org/
Not a Franklin scholar, but found this useful. Loved the ability to search across papers with questions like “How many times does BF use the terms “chemical”? Can check if the word “alchemy” was in use at this time. Was able to use the full-text search to identify a copy of a book in another library as belonging to BF.
-How would you imagine this functionality assisting your work as a scholar?
Ability to do a full-text search and then cut and paste translations. Would rely on transcriptions and translations during research, but would be nice to double-check against the original digitized manuscript before publishing.
User 2:
Prompt:
-You open this Newton manuscript. You are looking for a transcription or translation. Where do you look?
Scrolled down – looking at the metadata on the right. Not looking at left because it looks like pictures.
Gets to bottom and doesn’t see it, so saying they’ll click on “info”. Doesn’t understand what “Info” means. Button looks same as other two, but functionality is vastly different. Doesn’t think it should be a button.
Went to Roth Mallor’s Work and found description more helpful than previous page. Thinks metadata takes up too much space on other page – why not push it below image?
Discussion questions:
-Would you ever hit “view” button, assuming the transcript might be with the image?
No – assumed it would appear next to image on the item detail page.
-Are they any other sites you’ve used that have a manuscript transcription or translation on it?
Chymistry of Isaac Newton: https://webapp1.dlib.indiana.edu/newton/ has two transcriptions
The Newton Project: http://www.newtonproject.ox.ac.uk/texts/correspondence/all (Chymistry of Isaac Newton borrowed many ideas from the Newton Project.)
The Algernon Charles Swinburne Project: http://swinburnearchive.indiana.edu/swinburne/ (User remembered this, but upon looking at the website they couldn’t find what they were looking for and commented on the bad UI.)
-What would you imagine a good UI to look like?
Likes the layout for Early English Books online, where the image is on the left and the transcription is on the right. Thinks we should lead with the transcription but have a link to the English translation
-What kind of research questions could you imagine asking? What functionality would you need to answer those?
Would likely do a keyword search across the collection. Would rather have a title/author advanced option, but recognizes that this is becoming less popular. Would also like to be able to search the transcription of a single text for a word or phrase, like “antimony” and get a highlight.
-Would you need a PDF or would you cut and paste text from the webpage?
If looking for a citation or quote for research would probably cut and paste from webpage. If you need a whole corpus for research on your laptop then you’d download PDFs but these would only be useful if they’re scannable/OCR’d.
Some scholars have whole corpuses saved on computers, but that’s because the sources are locked behind a paywall and they’re worried they’ll lose access in the future. Since everything is free and so easy to search across Digital Collections it’s probably not necessary here.
-Any other examples of Digital Collections with transcriptions/translations come to mind?
Likes the idea of mimicking ECHO layout at the Max Planck, but display the transcription/translation on the side of the image and have other thumbnails https://echo.mpiwg-berlin.mpg.de/ECHOdocuView?url=/mpiwg/online/permanent/archimedes/newto_philo_039_la_1713&viewMode=image&pn=6
Doesn’t like how ECHO makes you page through 10 pages of thumbnails at a time. Prefers this page: https://catalogue.swanngalleries.com/auction-catalog/MAPS_ATLASES_NATURAL_HISTORY_COLOR_PLATE_BOOKS?saleno=2571&orderby=1 ,
User sees first 60 lots and then clicks "Load More" to append the next 60, etc. This allows the user to look back over what has already loaded without having to re-load those from the server.
User 3:
Prompt:
-You open this Newton manuscript. You are looking for a transcription or translation. Where do you look?
Scans metadata on right looking for transcription. Doesn’t see it, so clicks on “Info” and scrolls. Doesn’t see it. If a casual user not doing this exercise for a prompt, they admit that they would leave the page now. If a serious researcher, might dig deeper.
Discussion Questions:
-Would you click View, assuming it would be next to the enlarged version of the image?
No - would just assume there wasn’t one. Would only click that if trying to decipher the text and it’s hard to read. Would then be surprised to see it there.
-Would you want to download a PDF or just use this database?
Yes. Already has a 3 volume complete correspondence of a scientist saved as OCR’d PDFs on laptop. #1 reason is portability- travels a lot for research and likes having access even when no internet. #2 reason is searchability. Index in paper copy of book doesn’t go down to keyword. Research question: why was the discovery of cobalt blue pigment kept a secret? To answer, might look for any references to word “pigment”. Finds 55 mentions of the word through keyword search, but the index only has to/from names for letters. If able to search across collection in Digital Collections would do it there, but would want offline access and ability to download in case of no internet. Also worries about long-term access and possibility that this database won’t always be maintained by institution in the future. Likes having own PDFs – sense of permanence. Sometimes PDF is easier to use than database, such as with HathiTrust. It’s about having control.
-If transcription were a field in the metadata, would you cut and paste from it?
Yes. Would search for keyword like “pigment” and then cut and paste references to it in notes. Would always double check against original. Would use a transcript before a translation. If using a translation, would need to trust the translator and would want to credit them in the citation.
User almost never uses works in English for research. Typical to have a footnote “All translations are mine unless otherwise noted.” Translations are an act of scholarship. No two the same.
-Any example sites in your mind?
Not really. Most of my own research is obscure and only available in print. Rarely uses digital platforms for textual research.
ADDITIONAL INFORMATION:
-Some liked the layout for DIY History at Iowa. Has a list of thumbnails, then clicking on a page pulls up the image and a box on the side with buttons to Transcribe or Translate: https://diyhistory.lib.uiowa.edu/transcribe/137/9179
Other notes from Jahna’s conversation with Margaret Graham at Drexel’s Legacy Center:
“… she had heard of a program called Transcribe Us, which we might want to look at to see how they split the screen between the object and the transcription area. A third option is a program from Douglas Day that is used for crowd sourcing transcription. This does not apply to our project as Jocelyn is the only person translating for us.”
-One user has used Hartlib Papers archive, which has ability to search across collection, full text diplomatic transcription, and image file: https://www.dhi.ac.uk/hartlib/browse