User Testing Phase 1 - Findings

User 1 glanced at thumbnail and recognized it was in English so doesn’t need a translation. Eye drawn to metadata on righthand side, scrolled through and read headings aloud, saying they were looking for one that said Transcription. Got to the bottom about Physical Location, then looked at left-hand side. Noted one button said “Info” but didn’t know what it would do. Not sure why it said info when other buttons were actions. Clicked Info and didn’t know what they were looking at. “Why does it say “Roth Mallor’s Work? Is this a title? Stopped exercise and said, “I’m confused about what to do next – am I missing something?”

User never opened the image viewer. When prompted user to do so and they said it was easy to zoom in to read with manuscript with a touch screen, but didn’t expect to find a transcription here.

User 1: Yes. Don’t need to Zoom on a transcription so why would it be there? Wants to scan metadata and quickly find what’s needed there, then cut and paste relevant sections into research notes.

User 1: Took them some time to answer this, but eventually said yes, that might be nice to have, but couldn’t readily think of a use case. The ability to do a full-text search across a particular corpus in the database is more important.

Yes – Benjamin Franklin Papers through APS/Yale: https://franklinpapers.org/

Not a Franklin scholar, but found this useful. Loved the ability to search across papers with questions like “How many times does BF use the terms “chemical”? Can check if the word “alchemy” was in use at this time. Was able to use the full-text search to identify a copy of a book in another library as belonging to BF.

Ability to do a full-text search and then cut and paste translations. Would rely on transcriptions and translations during research, but would be nice to double-check against the original digitized manuscript before publishing.

Scrolled down – looking at the metadata on the right. Not looking at left because it looks like pictures.

Gets to bottom and doesn’t see it, so saying they’ll click on “info”. Doesn’t understand what “Info” means. Button looks same as other two, but functionality is vastly different. Doesn’t think it should be a button.

Went to Roth Mallor’s Work and found description more helpful than previous page. Thinks metadata takes up too much space on other page – why not push it below image?

No – assumed it would appear next to image on the item detail page. 

Chymistry of Isaac Newton: https://webapp1.dlib.indiana.edu/newton/ has two transcriptions

The Newton Project: Newton’s Correspondence (Chymistry of Isaac Newton borrowed many ideas from the Newton Project.)

The Algernon Charles Swinburne Project: http://swinburnearchive.indiana.edu/swinburne/   (User remembered this, but upon looking at the website they couldn’t find what they were looking for and commented on the bad UI.)

Likes the layout for Early English Books online, where the image is on the left and the transcription is on the right.  Thinks we should lead with the transcription but have a link to the English translation

Would likely do a keyword search across the collection. Would rather have a title/author advanced option, but recognizes that this is becoming less popular. Would also like to be able to search the transcription of a single text for a word or phrase, like “antimony” and get a highlight.

If looking for a citation or quote for research would probably cut and paste from webpage. If you need a whole corpus for research on your laptop then you’d download PDFs but these would only be useful if they’re scannable/OCR’d.

Some scholars have whole corpuses saved on computers, but that’s because the sources are locked behind a paywall and they’re worried they’ll lose access in the future. Since everything is free and so easy to search across Digital Collections it’s probably not necessary here.

Likes the idea of mimicking ECHO layout at the Max Planck, but display the transcription/translation on the side of the image and have other thumbnails https://echo.mpiwg-berlin.mpg.de/ECHOdocuView?url=/mpiwg/online/permanent/archimedes/newto_philo_039_la_1713&viewMode=image&pn=6

Doesn’t like how ECHO makes you page through 10 pages of thumbnails at a time. Prefers this page: https://catalogue.swanngalleries.com/auction-catalog/MAPS_ATLASES_NATURAL_HISTORY_COLOR_PLATE_BOOKS?saleno=2571&orderby=1 ,
User sees first 60 lots and then clicks "Load More" to append the next 60, etc. This allows the user to look back over what has already loaded without having to re-load those from the server.

Scans metadata on right looking for transcription. Doesn’t see it, so clicks on “Info” and scrolls. Doesn’t see it. If a casual user not doing this exercise for a prompt, they admit that they would leave the page now. If a serious researcher, might dig deeper. 

No - would just assume there wasn’t one.  Would only click that if trying to decipher the text and it’s hard to read. Would then be surprised to see it there.

Yes. Already has a 3 volume complete correspondence of a scientist saved as OCR’d PDFs on laptop. #1 reason is portability- travels a lot for research and likes having access even when no internet. #2 reason is searchability. Index in paper copy of book doesn’t go down to keyword. Research question: why was the discovery of cobalt blue pigment kept a secret? To answer, might look for any references to word “pigment”. Finds 55 mentions of the word through keyword search, but the index only has to/from names for letters. If able to search across collection in Digital Collections would do it there, but would want offline access and ability to download in case of no internet. Also worries about long-term access and possibility that this database won’t always be maintained by institution in the future. Likes having own PDFs – sense of permanence. Sometimes PDF is easier to use than database, such as with HathiTrust. It’s about having control.

Yes. Would search for keyword like “pigment” and then cut and paste references to it in notes. Would always double check against original. Would use a transcript before a translation. If using a translation, would need to trust the translator and would want to credit them in the citation.

User almost never uses works in English for research. Typical to have a footnote “All translations are mine unless otherwise noted.” Translations are an act of scholarship. No two the same.

Not really. Most of my own research is obscure and only available in print. Rarely uses digital platforms for textual research.