Navigation

HathiTrust Quality Assurance and Standards Working Group

HathiTrust is committed to providing a high quality digitized corpus for all its stakeholders. With a focus on quality assurance and advancement as an affirmed priority by the membership,  the Program Steering Committee (PSC) is establishing the HathiTrust Quality Assurance and Standards Working Group. This group will build on the initial work of the ad hoc Quality Working Group, convened in August 2015, which offered recommendations on development of a quality metadata schema, called for the provision of quality indicators (or “signals”) visible to end-users, and developed and shared documentation for Google partners to use in replacement and insertion of pages in Google-scanned volumes. The Quality Working Group will report to the PSC, with liaisons to the Collections Committee and the Metadata Policy, Strategy, Use and Sharing Advisory Group (MUSAG).

Charge

The HathiTrust Quality Assurance and Standards Working Group is charged to recommend strategies, processes and techniques detailing how members, end users, and/or HathiTrust Operations can contribute to the quality of the corpus through making scalable improvements of digital surrogate fidelity at the item/object level.  This includes, but is not limited to recommending on development of a quality certification program, identifying and reporting errors and omissions, making corrections, and running pilot tests to monitor the quality of ingested content.  Out of scope are collection development-related quality improvements (e.g., collection completeness, underrepresented collection areas, etc.).

Specifically, the Working Group is charged to conduct the following initial and longer-term work:

Initial Work

  • Inventory existing sources of quality metadata (GRIN, METS, OCR scoring, user feedback, locally gathered information, etc.) and recommend an approach, methodology and/or system for the collection, aggregation and storing of relevant quality metadata from available sources;

  • Liaise with MUSAG and the Zephir team, which hold primary purview for bibliographic metadata quality, on assessing the requirements for and development of a metadata schema for quality status or characterizations of items in the corpus, including the recording of quality issues reported by members and end-users;

  • Work with HathiTrust Operations staff to test and monitor the quality level of content that is still gated (i.e., disallowed from ingest into the repository) due to quality issues; and

  • Review HathiTrust standards and specifications related to ingest and quality assurance of the digital objects in the corpus; and recommend any needed changes on the gating practices based on the findings.

Longer-Term Work

  • Liaise with the Collections Committee to identify categories or specific content that should be given priority for quality improvement efforts, and to recommend actions,  including remediation designs and pilot testing, that would advance this work;

  • Evaluate the need for and potential value of a quality certification program.

  • Liaise with MUSAG to propose and, ultimately, develop and test methods of displaying to users identified quality attributes (or “signals,” positive and negative) in specific items and relative to intended purposes (e.g., quality fitness for print disabled users with respect to OCR accuracy, verified completeness, etc.);

  • Develop, document, and monitor procedures for page insertions and inclusion of non-Google and other locally digitized supplementary or replacement content; and

  • Communicate quality improvement initiatives and liaise with the proposed Digital Object Quality Corrections subgroup of the User Support Working Group, which intends to focus on responding to quality problems flagged by users through the HathiTrust User Support mechanism, according to the standards and policies set.

Timeline

The Working Group will conduct its initial work August 2016 – July 2017, consulting with the PSC at which time the PSC will review possible adjustments involving priorities and workplans.

Communications and Guidance

The PSC will appoint 1-2 of its members to serve as attending liaisons to the Working Group. Issues, requests, and informational reporting between the Working and the PSC will be primarily conveyed by these members. The PSC also requests brief periodic progress reports from the Working Group chair (e.g., quarterly), which may prompt interest in occasional invitations for discussions with the PSC at its biweekly meeting.

Membership

  • Paul Fogel, California Digital Library, Chair

  • Aaron Elkiss, University of Michigan

  • Janet Gertz, Columbia University

  • Peter Gorman, University of Wisconsin-Madison

  • Kat Hagedorn, University of Michigan

  • Martha Hruska, University of California, San Diego (liaison to HT Collections Committee)

  • Erik Mitchell, University of California, Berkeley (liaison to PSC)

  • Michelle Paolillo, Cornell University

  • Sandra McIntyre, HathiTrust, liaison to Zephir team and Operations staff

  • Angelina Zaytsev, HathiTrust, liaison to MUSAG and USWG

Selected Background Documents

Archival Quality and Long-term Preservation: A Research Framework for Validating the Usefulness of Digital Surrogates; by Paul Conway; 2011

https://deepblue.lib.umich.edu/handle/2027.42/86643

HathiTrust Program Steering Committee Planning Brief: HathiTrust Quality and Validation Issues; September 2014
https://drive.google.com/a/umn.edu/file/d/0B-kkSzFLihRaX2h3dTh0aTJPWVE/view

HathiTrust Commitment to Quality Statement; June 2016

https://www.hathitrust.org/quality

HathiTrust Quality Assessment and Validation: Report of the Visibility and Feedback Working Group
https://docs.google.com/document/d/1oo71S022-nTowvnmzTuSxPahErVcKcer2s98IvZkCqM/edit

HathiTrust Quality Validation Group: Foldouts and Missing Pages Working Group

https://docs.google.com/document/d/1oDJXOg72426_4CDahnWlOxWb6YpO6gpVwpGqz9NOTHQ/edit

Quality in HathiTrust; blog posting by Jeremy York (HathiTrust) and Kat Hagedorn (University of Michigan Library); May 13, 2015

https://www.hathitrust.org/quality-in-hathitrust