Day of Digital Archives: Artist's Collections

Well, this year's Day of Digital Archives ha been much more successful for me than last year (I broke my elbow in a cycling accident that day and spent much of it loopy because of the pain pills. I did type out a one-handed blog post but I don't think it ended up being coherent.) This year I want to talk about an artist's collection we've been working on for a bit at UO.

The Tee A. Corinne papers are one of the many hybrid collections we have in Special Collections and University Archives at the University of Oregon. Tee Corinne was a lesbian visual artist, writer, and activist who explored female sexuality in her visual and written works. Upon her death in 2006 she left her entire estate, including the rights to her literary and artistic works, to the University of Oregon Libraries. Owning the rights is nice because, once we've done our initial processing and preservation work on the files, we don't have to worry about any rights issues when providing access to the digital objects.

However, before we can even start worrying about access to the materials we've had to devise a plan for working with the digital records. When UO received Tee's collection in 2006, it included a laptop and a desktop computer as well as removable media containing various works and papers. At that time, the UO did not have well-developed procedures of workflows in place for ingesting or otherwise processing digital objects. The files were pulled off Tee's computers and the various media and moved over to library servers, but nothing else happened to them for a number of years. In the meantime, there was a gap of more than a year between the time my predecessor (the first e-records archivist at UO) left and the time I was hired. The Tee Corinne e-records were left on the servers and until now I haven't been able to work with them at all.

When I started my initial assessment of the digital portion of Tee's papers, my first task was to try to gather all the digital objects from the collection into one place on the server. Because of the lack of workflows when the collection was taken in, the digital objects ended up in a number of different places on the server. Although I think I've managed to round up most of them now, I still run across stray files that have to be added in with the others. When we started the project this summer, we identified 65,328 digital files we knew came from Tee's computers or from the removable media in her collection. Although I would love to be able to declare that all those files in fact belong in Tee's collection, she shared her computers with Beverly Brown, her lover, whose collection the UO also owns. In addition, Bev Brown was the founder of and was heavily involved with the Jefferson Center, an organization whose records the UO holds as well. Once we started looking at the files from Tee's computers, we realized that her files, Bev's files, and files from the Jefferson Center were all mixed together. The organic file structure the women were using did not clearly distinguish among these three separate groups. Often a single directory will contain files from all three collections. This has slowed down our processing: we're trying to develop some content-based filters so we can do some batch sorting of the files. Most of the textual documents were created in version of WordPerfect, so we're also working on batch converting those files. In addition, of course, we're having to do a lot of renaming so that the file names of the preservation copies don't have any of the potential trip-ups you see in organically-named files.

The most interesting challenge in dealing with this collection, however, has been the photographs. Photography was one of the many media in which Tee worked, and she made extensive use of Photoshop. Sometimes she created prints of several digitally-altered versions of a single photograph; we are often able to match physical prints with digital files, but in some cases we have digital photographs for which no physical print exists or vice versa. Tee also tended to revise her photographic series depending on the context in which she was exhibiting or publishing them. This means we sometimes have several different series of a single image or group of images. The series may or may not be consistent; that is, sometimes a series of images was published in one form in on place and in a different form somewhere else. In the digital files, this means that in some cases we have many duplicate copies of a single image (if Tee organized the files based on the various publications) as well as multiple different versions of an image. We would prefer not to transfer multiple copies of a single image onto our preservation servers, but we do want to preserve the different versions of the images because we feel these are an important artistic statement. Sorting out the files themselves has proved to be an enormous challenge, however. Luckily I have a team of graduate students and volunteers who are working hard on this (as well as other) projects.

What have I learned from my work with this collection so far? Obviously, documentation is a hugely important factor when you're talking about a born-digital collection. One of my main problems right now is the lack of documentation from previous work that occurred with this collection (however cursory that work might have been). I'm trying to document every step I take with these records so that my successors have a clear picture of what has and hasn't been done with the materials. It's also important for the digital archivist to be involved in the donation process if at all possible; this helps lessen the amount of triage work you have to do when the born-digital records arrive on your doorstep.

Day of Digital Archives

My Day of Digital Archives is not going at all as planned. I meant to be working on web archiving and training my students in processing the e-records of our last University President. Instead, I'm sitting at home typing one-handed and nursing a broken elbow (on my dominant arm, of course) sustained in a cycling crash yesterday on my way to work. So this post will be pretty short, and if anything catches your interest feel free to email me at khomo [at] uoregon [dot] edu for more information.

First, a little background about me: I'm the Electronic Records Archivist at the University of Oregon. I'm responsible for the ingest and preservation of e-records in Special Collections and University Archives. My current most pressing projects:

• Processing raw video files of oral histories documenting the Latino experience in Oregon. I had a plan, and then Final Cut X threw a wrench in the works: the raw video footage created in Final Cut 7 can't be read with Final Cut X. This is a textbook example of why proprietary digital formats are not a preservation solution.
• The aforementioned presidential e-records. Organic file trees are a nightmare, and trying to sort out the records designated as non-permanent by state law only adds to the confusion.
• Creating an inventory of all our audio-visual material in order to prioritize both for digitization and for traditional preservation/conservation.
• Doing some pre-emptive work with a couple of faculty members who are teaching oral history classes this academic year and who want their students to deposit the audio files in the archives. I'm happy to take the content, but if I work with the professors ahead of time and get myself scheduled to do some basic instruction in the class I'm far more likely to get usable files at the end of it.
• Archiving University web content, both from the official UO pages and from other sources (social media, news outlets, etc.)
• Working with our data management team to create ingest and cataloguing workflows for acquired digital data.

I think that's it. Now excuse me while I take some pain medication.

McLuhan's Understanding Media: "The [digital] medium is [no longer] the [only] message."

Happy Day of Digital Archives

David Kay, nydawg [New York Digital Archivists Working Group]
October 6, 2011
McLuhan: “The Medium Is the Message?” or “The [digital] medium is [no longer] the [only] message.”

This year marks the 100^th anniversary of the birth of “the new spokesman of the electronic age”, Marshall (Understanding Media) McLuhan, and digital archivists should take a moment to think about how media, digital and analog, hot and cool, and in many different formats change our jobs, lives and responsibilities. With threats of technological obsolescence, vendor lock-in, hardware failure, bit rot and link rot, non-backwards compatible software, and format and media obsolescence, digital archivists need a system to accurately describe digital objects and assets in their form and function, content, subject, object and context. If we miss key details, we run the risk of restricting access in the future because, for example, data may not be migrated or media refreshed as needed. By studying and understanding media, digital archivists can propose a realistic and trustworthy digital strategy and implement better and best practices to guarantee more efficiency from capture (and digitization or ingest) and appraisal (selection and description), to preservation (storage) and access (distribution).

Over the last ten, forty, one hundred and twenty thousand years, we have crossed many thresholds and lived through many profound media changes-- from oral culture to hieroglyphic communications to the alphabet and the written word, and from scrolls to books, and most recently transiting from the Atomic Age (age of atoms) to the Information Age (era of bits). While all changes were not paradigm shifts, many helped shift currencies of trust and convenience to establish new brand loyalties built on threats of imminent obsolescence and vendor lock-in. As digital archivists, we stand at the line separating data from digital assets, so we need to ensure that we are archiving and preserving the assets and describing the content, technical and contextual metadata as needed.

Want to read the rest? Check out the New York Digital Archivists Working Group [#nydawg] wordpress blog. Thanks for your interest in #DigitalArchivesDay #DoDA.

A Digital Dark Age?

Was just reading this article about a 700 year old manuscript found here in our stacks at the University of Virginia's Albert and Shirley Small Special Collections Library. Listen to this quote:

In his essay, Greenia describes how rare it is to find medieval documents. He estimates that for every volume that survived during the Middle Ages, nine more were lost. Archivists have documented the disappearance of parchments from the 13th and 14th centuries, “some perhaps rolled into tubes to make fireworks.”

During the Spanish Civil War (1936-1939), the contents of city archives were sometimes stuffed into the barricades. Since they didn’t have sandbags, said Greenia, books were used instead.

And before the municipal archives were transferred to Madrid in the late 1970s, they “were still being kept in a top floor space subject to damp air and, when it rained, dripping water.”

Now think about digital documents in our current age. How many have you lost to a hard drive crash? To the damage done when you dropped your iPhone or laptop? How many of them have you forgotten were on your computer at all? How many will you back up or move to your next computer?

Some have said we might be living in a digital dark age if we don't come up with better ways to manage our digital content, and fast. What do you think?

Surveying "Hidden" Electronic Media

As a part of my position, I implemented a Condition Survey of the archival collections maintained by the three collecting repositories at Bobst Library. This survey will provide data on our collections that will inform the Preservation Department's strategic preservation planning for the Archives Preservation Program at NYU. This post at The Back Table (the NYU Special Collections collective blog) describes this survey in more detail. You can also view a poster about the findings of this survey to date (Summer 2011) presented earlier this year at the Society of American Archivists Annual Conference in Chicago.

The survey identifies preservation issues on a unit (or box) level as well as, in aggregate, at the material level. During the course of the first year of surveying, we uncovered a significant amount of electronic media stored on various magnetic, optical and flash storage devices. Our goal this fall is to target and survey collections, known or suspected, that contain electronic media for the purposes of identifying, quantifying and, later, prioritizing preservation for these items. Building a survey tool that can accommodate the appropriate information for identifying and quantifying electronic media storage required some research that took me to view presentations from last year's Rare Books and Manuscript Pre-Conference (scroll down to the seminar I. Born-Digital Manuscripts: A Primer) as well as Wikipedia. I am not afraid to say, as I have heard former professors admit and use heavily during my coursework, that Wikipedia is an excellent source for computer and technology history. From my research, I developed a guide for identifying various type of magnetic, optical and flash storage devices that the survey assistants will be able to refer to while surveying. Also, I reconstructed a portion of the survey tool to accommodate collection of information necessary for identifying and quantifying electronic media. For instance, manufacturer, disk format, label information and commercial/non-commercial status will help preservation staff determine migration requirements, potential file types, and storage capacities. This information will also help curators determine priorities for preservation in the future.

While this survey gets underway, I am sure tweaks will be made to the survey tool as well as our surveying techniques. If anyone is undertaking a similar project, I would be happy to hear about your experiences, discuss successes and failures, and long term planning.

Jennifer Waxman
jennifer.waxman@nyu.edu
@jwax55

Playing hide & seek with digital collections

Happy Day of Digital Archives, everyone!

My name is Gwen Glazer, and I was a Google Policy Fellow with the American Library Association in 2010. While at ALA, I worked on a paper about digitizing hidden collections in small and medium-sized public libraries.

Now, we’re talking about not only how to help libraries get their materials online, but how the resulting digital collections might be linked together so they can be accessed by the greatest number of people.

What are hidden collections?

They’re materials in public libraries that are hard to access, for many reasons: the limitations of their physical locations, backlogs in cataloging and processing, vulnerability to accidents, theft, and neglect, and many more.

Hidden collections are attracting more and more attention; in the age of global information, libraries need to increase access to locally specific materials that can’t be found anywhere else. These could include business directories and town maps from the 18th century, for example, or photographs and handwritten letters in a library’s physical collection.

Most of these hidden collections’ primary content isn’t born digital, but in an ideal world, some collections would be flexible and user-driven so that people could add relevant content of all kinds. Plus, some of the power of these materials comes from crowdsourcing and the extra information people add online, via Flickr tagging, commenting, and more.

Who uses these hidden collections, and why bother digitizing them?

Public library patrons of all stripes — researchers, students, casual browsers, genealogists, and more.

But if these materials were not only digitized but linked together, that number would grow exponentially. People who might live thousands and thousands of miles outside any given library’s jurisdiction would also have access to them.

Just by being online, these materials gain value. Digital archives are increasingly vital for libraries, and they’re a part of our core mission: bringing previously hidden information to light.

But how can people find digitized hidden collections?

Exactly! One of the biggest challenges facing small and medium-sized public libraries that are trying to digitize their materials — aside from funding, of course — is how to make their collections accessible to the most people.

Smaller organizations may have the greatest digitized collections in the world, but if no one can find them, they may as well not exist.

Discoverability is the key. Libraries and consortia are looking at many different solutions to increase discoverability and link their content together, including:
·      State-level initiatives like Ohio Memory
·      Federal initiatives like the Library of Congress’ National Digital Newspaper Program
·      Nonprofit initiatives like Lyrasis Mass Digitization Collaborative

A lot of people are working on pieces of the puzzle, but no one quite has an answer yet, and that’s part of what we’re trying to figure out now.

ALA is hosting a free webinar on Wednesday, Nov. 2  from 1 p.m. to 2:30 p.m. EDT to discuss this exact question. 

I’ll be hosting the webinar with Jason Kucsma from the Metropolitan New York Library Council. We’ll be talking with digital library project managers and practitioners about the challenges and rewards inherent in managing small and medium-sized digitization projects and sharing success stories, best practices, and the lessons learned to encourage other successful projects.

We’d love to hear what you have to say. Join us and continue the conversation about digital archives; you can sign up for the webinar at this URL -- https://www2.gotomeeting.com/register/322790778 -- and you’ll receive a confirmation email with more details after registering.

Thanks for reading, and thanks to Gretchen for creating this project!

<i>See this post and more about ALA's digitization work on the District Dispatch blog.</i>

The To-Do List of a Digital Archivist

It's immensely gratifying to see today coming together. I hope that everyone who is reading is enjoying the blog and learning a lot. While this project wasn't an incredible amount of overhead to put together ("Hi, my name is Gretchen. Could you write a blog on October 6th? Thanks!" ...okay maybe a bit more swearing at Blogger was involved...), I haven't been able to do a lot of thinking about what I would write today. But since the point is to capture a "slice of life" in the world of digital archives, I thought the best illustration for me would be to post my (annotated) To-Do List. So, in all it's glory, here it is:

• start to draft blog for D0DA
  (woo-hoo! it doesn't say "finish blog" so I can check this one off!)
• check on saturday shift
  (As a member of the Special Collections staff I am still required to work on the reference desk each week and one Saturday each month. Although I don't consider myself "a people person" and working on the desk slightly stresses me out, I'm generally happy to do this. It helps me to understand how people use our collection, which in turn helps me design better interfaces and tools. It also helps me to learn about what's in the collection. However, I just started here in May and some things that are specific to our department are still a mystery to me. This task is related to my shift this coming Saturdy. The schedule says that my shift this Saturday is only from 1-5, but I know the building is open all day. I need to figure out what's going on...before Saturday.)
• correct Warner EAD
  (this item and the next one are all related to the AIMS grant project. I started here at UVa in May to replace the former Digital Archivist on the project, which is exploring the management of hybrid collections — those containing both digital and analog materials. One of the collections that was to be processed for the grant was the Papers of Senator John Warner. The collection included 35 CDs of scanned correspondence. There ended up being far too many issues surrounding copyright and to be able to do much with this collection at this point other than to image the discs and put the content into our long-term storage space. This fulfills part of our mandate to steward these materials because they will be "preserved" in this way, but we still need to process them to some level (i.e. track what's actually ON the disks...maybe) and make them accessible.
  [As an aside, the idea for the D0DA project came out of an unconference held by the AIMS team. You can read all about it on our AIMS blog if you are interested.]) 

Is This Bit Rot?

The term "bit rot" gets batted around a lot, but isn't so easy to nail down. Wikipedia's bit rot entry is surprisingly short and contains multiple definitions: decay of storage media, decay of data on storage media, and degradation of software programs. The entry is flagged as needing citations to reliable sources.

So maybe I'm not alone in wondering what bit rot looks like in a word processing file. At the Hoover Institution Archives, where I work, one manuscript collection contains hundreds of 5.25-inch floppy disks. The disks present many problems, but today I'm looking at one containing files that open on a PC. They suffer from some malady, as indicated by this sample from a letter dated January 21, 1986:

-------begin sample text------

Thankó verù mucè foò á lovelù luncheoî anä somå splendiä views® Wå 
imaginå yoõ no÷ iî Indiá anä wondeò iæ yoõ arå listeninç tï somå oæ thå 
samå Indianó witè whoí wå talkeä yearó ago® Thå artistó anä economistó 
werå quitå remarkable¬ buô thå politicaì scientistó useä tï talë abouô 
atomiã bombó foò Indiá witè eager¬ burninç eyeó whilå beinç verù carefuì 
noô tï kilì anù insects® (Severaì haä theiò beardó covereä iî whitå 
silk so that no insect would get caught and be stifled there.)

-------end sample text------

The file name, Enid, has no file extension, so it is difficult to determine what software was used to create the file. The sample above is from the rendering in MS Word with Windows character encoding, but no matter what software I open the file in, I get some gibberish. But it isn't all gibberish--the last line is completely legible. You'll see that sometimes a letter displays correctly, like the "n" in "insect," but not in other cases, like the "n" that should end "luncheoî" in the first line. Is this bit rot?

Regardless of the diagnosis, the next question is what to do with it. Because we can infer what many of the corrupted characters should be, we can match them to their actual counterpart, like this sample:

Corrupted version	True character
ë	k
ì	l
í	m
î	n
ï	o
ð	p

Then, we could use the find and replace feature to fix all the corrupted characters. But it is more difficult to infer the correct characters for corrupted numbers. In addition, I assume that matching corrupted to true characters also varies from one file to the next--after all, this decay probably doesn't occur in the same, predictable manner and standard rate. Furthermore, we've got to deal with thousands of corrupted files on hundreds of disks. Lastly, if we did restore all of the files, how can we ensure researchers studying them understand our restoration process and its implications for the authenticity and reliability of the content?

If you can answer my questions, I think Wikipedia needs you to enhance its bit rot entry.

How does the State Library of North Carolina celebrate the Day of Digital Archives?

October 6 is the first annual Day of Digital Archives. It's an "initiative to raise awareness of digital archives. . . What are digital archives? Who uses them? How are they created and managed? Why are they important?" ( more here).

Read more about what the State Library of North Carolina does, why it's important, and what you can do to take some simple steps to save your digital content.

There'll be posts here at the DoDA blog and Tweets (#digitalArchivesDay) by folks from many different organizations throughout the day.

Still need more information about digital preservation? Check out information for burgeoning digital archivists or seasoned preservationistas on the State Library of North Carolina's digital preservation website at http://digitalpreservation.ncdcr.gov/.

We sure know how to party, don't we?