Day of Digital Archives

Day of Digital Archives: Confidentiality and Collaboration

2012-02-29T16:47:00.003-05:00

Hi--my name is Gabby Redwine, and I'm an archivist at the Harry Ransom Center, which is a literary collecting archives, humanities research library, and museum at the University of Texas at Austin. My job has three components: developing the Ransom Center’s digital preservation program; reviewing and managing xml finding aids; and processing paper and hybrid holdings. I also find ways to participate in the intellectual life of the University (particularly the English and History departments), facilitate research related to the Center’s African Studies materials, and am involved with projects that fall under the digital humanities umbrella. On a typical day, these different activities merge in ways that I find productive and inspiring. I had intended to write a “day in the life” post on October 6 to share some of my excitement about what I do.

Instead, I spent that whole week working on digital projects that I couldn't discuss outside of the walls of my institution. Some of these were genuinely confidential – for example, research related to the digital media in a prospective acquisition. Other work, like drafting policy documents or planning for an ongoing collaborative project with colleagues from other institutions, simply wasn’t far enough along to discuss in a public forum.

Collaboration is a necessity for digital archivists. None of us alone has the skills or time to become a master of every media type, format and tool, particularly since the technological terrain keeps shifting beneath our feet. Pooling expertise is the only way many of us can move forward. Since so much of my work is collaborative and involves scholars as well as archivists, curators, and librarians at other institutions, I often struggle with how to maintain an appropriate level of confidentiality, while also sharing enough information to make a given collaborative effort worthwhile--or contribute to an outreach effort like the Day of Digital Archives.

I find this a particularly hard line to toe when working with scholars and doctoral students. These two parties come to the table with research agendas that are sometimes at odds with the responsibility I feel to the authors whose personal papers and digital media I process, to my institution, and to my profession. Respecting an author’s privacy, building trust with current and future depositors, and abiding by professional ethics and in-house policies regarding confidentiality are of paramount importance. These priorities limit the amount of information I can share with researchers who want to study digital archivists, digital preservation programs, and born-digital collections.

But one of the great benefits of cross-disciplinary work is being forced to re-examine existing policies and practices and re-articulate them to a diverse audience. Collaborative projects encourage archivists, curators, and librarians to theorize our hands-on work; give us the opportunity to ask faculty and student researchers how they want to engage with born-digital materials; and help us build a multidisciplinary community of colleagues with a range of skill sets, agendas, and institutional powers. These and other benefits make it worth our while to figure out how to share information in a way that furthers collaborative work, while also respecting a depositor’s privacy and upholding professional ethics and internal policies.

24 Hours: The Day of Digital Archives

2011-10-07T13:56:00.003-04:00

I'm Mark Matienzo. I work as a Digital Archivist in Manuscripts and Archives at the Yale University Library and as the Technical Architect for the ArchivesSpace project. I worked with Gretchen on the AIMS project. I wrote about my Day of Digital Archives on my blog, thesecretmirror.com, by describing a fairly busy 24 hours I had on Thursday, October 6.

Analog-to-digital video preservation and a site visit to a repair facility at the NASA Ames Research Center

2011-10-07T00:12:00.003-04:00

Hello everyone, happy day of digital archives! I'm Lauren Sorensen, Preservation Specialist working at Bay Area Video Coalition, a technology non-profit devoted to inspiring social change by enabling the sharing of diverse stories through art, education and technology. Our preservation department specifically works to preserve and provide support to archives with film, video, and moving image and audio material in their collections. We are one of the only non-profit vendors for high-quality preservation of video and audio in the country.

My blog entry can be found on our home site blog here, and involves a site visit from earlier this year, when I met in person for the first time Ken Zin, who works on the NASA Ames Research Center campus (at the site of a former MacDonald's there!) repairing obsolete reel-to-reel videotape machines. His work is essential to us because he is one of the only experts left in the country doing this specialized type of work. Because these decks feature heads (the part of the machine that reads the magnetic waveforms on the tape) that are proprietary to the companies that made them in the 1970s and 1980s, it is a real challenge keeping them in proper working order for high quality preservation. Realignment and regular maintenance is important in maintaining a facility that is appropriate for preservation services; we maintain these decks as we would museum artifacts because they are some of the last working machinery that is available to transfer 1/2" open-reel machines; after these decks are no longer operational, any magnetic recordings held in archival collections will be lost.

Please enjoy the photos!

Souding Meaning in Archives

2011-10-06T23:30:00.001-04:00

I've posted my Day of Digital Archives over at my own blog.

Day of Digital Archives

2011-10-06T18:37:00.004-04:00

My Day of Digital Archives is not going at all as planned. I meant to be working on web archiving and training my students in processing the e-records of our last University President. Instead, I'm sitting at home typing one-handed and nursing a broken elbow (on my dominant arm, of course) sustained in a cycling crash yesterday on my way to work. So this post will be pretty short, and if anything catches your interest feel free to email me at khomo [at] uoregon [dot] edu for more information.

First, a little background about me: I'm the Electronic Records Archivist at the University of Oregon. I'm responsible for the ingest and preservation of e-records in Special Collections and University Archives. My current most pressing projects:

Processing raw video files of oral histories documenting the Latino experience in Oregon. I had a plan, and then Final Cut X threw a wrench in the works: the raw video footage created in Final Cut 7 can't be read with Final Cut X. This is a textbook example of why proprietary digital formats are not a preservation solution.
The aforementioned presidential e-records. Organic file trees are a nightmare, and trying to sort out the records designated as non-permanent by state law only adds to the confusion.
Creating an inventory of all our audio-visual material in order to prioritize both for digitization and for traditional preservation/conservation.
Doing some pre-emptive work with a couple of faculty members who are teaching oral history classes this academic year and who want their students to deposit the audio files in the archives. I'm happy to take the content, but if I work with the professors ahead of time and get myself scheduled to do some basic instruction in the class I'm far more likely to get usable files at the end of it.
Archiving University web content, both from the official UO pages and from other sources (social media, news outlets, etc.)
Working with our data management team to create ingest and cataloguing workflows for acquired digital data.

I think that's it. Now excuse me while I take some pain medication.

Missing first post-AIMS born-digital workshop (SULAIR)

2011-10-06T18:16:00.004-04:00

Several staff from Special Collections and elsewhere are attending Peter Chan's first (of four) workshop on working with born-digital materials. I think I am the only one who couldn't attend today!
This is a prelude to several projects that will be concurrently processing analog and born-digital materials and part of our build out now that the AIMS project is wrapping up. One of these projects is near and dear - belonging to the Manuscripts Division. It is part of our current NHPRC grant-funded project re the records (analog & digital) of the Stop Aids Project, a still-active organization based in San Francisco.

Digital archives are having their day!

2011-10-06T17:08:00.009-04:00

WOW. This DoDA thing has really taken off. Tons of terrific posts, people, projects, progress. Zipping through everything that has come in so far today, I've felt like a bit of a kid in a candy store (nerd alert! but I guess I'm in good company today). Unlike everybody else who is weighing in, however, I have no experience whatsoever managing or preserving any digital archival stuff. (That's assuming you don't count my stuff at home that's backed up on Icloud. Nope, I didn't think you would want to count that.)

Regardless, as an archivist, I've been obsessed with the born-digital deluge for a couple of decades--out of necessity rather than ability or innate interest. When I was a University Archivist, I had the embarrassing stack of physical media sitting on my desk waiting for the day when we would "have time." At least we were clever enough to check off "digital media" on our accession forms, so when survey day comes, it'll all be easy to find.

In my current role in OCLC Research, where we undertake projects that will press research libraries forward in addressing today's Big Challenges, we work actively with members of our OCLC Research Libraries Partnership to identify appropriate topics and do good work that will help. A year ago we published Taking Our Pulse, the report of a survey of special collections and archives in more than 150 research libraries (universities, colleges, museums ...) across the U.S. and Canada, the data from which demonstrated how little is going on in management of born-digital in research libraries. Though 80% of respondents said they have at least some born-digital materials, less than one-third could say how much. Fewer than half have assigned responsibility for addressing the born-digital realm. Publicly-accessible metadata is rare. More than 80% admitted that they need education and training. And--no surprise--born-digital is one of the three most often-named "most challenging issues" that special collections and archives must address.

In the wake of the survey report, my colleague Ricky Erway and I are now in the midst of what we like to call our "born-digital baby steps" project, the objective of which is to help research libraries that have yet to take any action get off the dime. In addition to outlining the briefest possible (so as not to intimidate) list of basic (really basic) steps that should be taken to get a handle on what you have (survey!) and bring materials under initial control (server space! bit imaging! simple accessioning!), we want to demonstrate that getting started truly is feasible, including for those who don't have special resources (or staff) and for geezer archivists like me who have spent years being frozen in place like deer in the headlights.

In our research reports we're inclined to consider library directors a core audience, since they ultimately set the priorities and control the purse strings. Some of those directors are counting on their archivists to deal with born-digital (particularly for content that's beyond the scope of an institutional repository), but they don't know what that means, or why it's so hard, or the extent to which their IT people are going to be key players. At the other end of the spectrum, however, some directors think that born-digital has nothing whatsoever to do with special collections and archives (um, that's just the "rare" and the "artifacts," isn't it?).

So in the baby-steps report we're also going to talk about how "born-digital" fits into research libraries in general, and how it intersects with special collections and archives. We'll outline the wealth of skills and expertise that archivists have to contribute to the mix. We'll discuss how the range of born-digital content and media dovetails with the types of material that special collections and archives have traditionally collected. Remember, library directors and other higher administrators are a key audience: even though many archivists already know these things, few directors know how to think about them. So, we hope we'll be able to help them get a grip so that they'll understand what their archivists are up against.

We have a small army of colleagues in the wings (including some of you), all of whom have various sorts of expertise in the born-digital realm, standing by to give us feedback on drafts, answer our baby-steps questions, and in general keep us from embarrassing ourselves by saying things that aren't quite true. And all these DoDA postings are suggesting some other people with whom we might want to talk at some point.

Speaking of which, if you have any reactions to our project, please comment! Brickbats, tomatoes, garlands of flowers are all welcome. Or get in touch with Ricky or me to chat.

Congratulations to everybody who has weighed in today, as well as those listening in, for being part of the community of born-digital archives and archivists. Hey, Gretchen, you better be planning to preserve this blog! It's going to be a fantastic resource.

Day in the Life: Digital Archives Educator

2011-10-06T16:43:00.004-04:00

On the Day of Digital Archives, I am

1) Working with local archives to develop projects for my Digital Archiving and Preservation class in the spring semester;

2) Planning with my TA a project to set up in our “Digital Archaeology Lab”several canonical systems for testing significant properties and how original bitstreams pulled from old media might interact with a coeval computing environment—to include research in historic popular computing publications of the relevant available programs, peripherals, and other sundries and drawing on a project last year researching the preservation of Ultima II using an Apple IIc from the Goodwill Computer Museum;

3) Discussing with project partners at the Museum several additional research projects going forward in the form of student capstones and individual studies;

4) Considering progress on my own research about archival organization of preserved born-digital materials;

5) Planning advising sessions for spring 2012 to assist students especially with getting experience in hands-on work with born-digital archival material—hardware, media, and content.

Here is the virtual archives: https://pacer.ischool.utexas.edu/

Here is the physical lab (or parts of it):

Organizing canonical environments

Inventorying

BLACKOUT

2011-10-06T16:38:00.001-04:00

I don't have a sleep schedule - I have periods of blackouts in between work sprints. Today's Day of Digital Archives happened for me after one of these blackouts, where I was up around midnight and looking to get a bunch of stuff online, and prepare the rest of it for being online. Online is where a lot of my stuff goes because the first and main priority is to share what I acquire, not sit it somewhere waiting for grant funding or hugs to make it better.

Here's part of 200 hours of Game Developers Conference footage I'm digitizing for their GDC Vault site - they're having me put everything that makes sense up, for free, for the world to see. These are conference recordings dating from 1996 up through to 2008, and there's a lot more besides this. Historically fascinating, one of the leading industries in the world has some of the brightest minds at this conference, not to mention introductions of major platforms like the Dreamcast, XBOX, Playstation (2 and 3), and so on. Most of this has been accomplished using this old thing:

..and this is just for the BetacamXP format tapes - I have HDCAM, MiniDV, VHS, and Audio tapes as well. All are being digitized, all are going up. A lot of work!

I blew a bunch of material up to the Archive Team collection at archive.org; mostly captures of websites that decided to go down that had been around a long time, like the social gaming site Akoha.com or the art project Word Count Journal and even some pieces of the big boys, like Google's now defunct Google Friends Newsletter they decided to deep-six this year. I'm now doing some checks on the Archiveteam Friendster Snapshot, where we put millions of Friendster accounts up for later study by academics and historians. We didn't get all of it, not by a long shot, but we definitely got a good sample for people to use.

A few years back, I did a documentary on text adventures called GET LAMP, and over time have been uploading raw footage sets from that movie into this collection. Well, among the things I shot was a ton of footage inside Mammoth Cave, Kentucky, and as we speak, 11 gigabytes of .m2t (hi-definition) footage is making its way up. That'll probably be all day, at least.

Just yesterday, my talk about Shareware came up! A massive, profanity-filled rant of fun, that talk is here.

Heck, if we want to mention talks, this is the way to go: Archive Team: A Distributed Preservation of Service Attack.

Until my next blackout!

2011-10-06T16:23:00.002-04:00

Written by Steven Szegedi, Archivist and Special Collections Librarian, Dominican University

This post was originally published on the Dominican University Library Blog.

This year's Society of American Archivists' 75^th anniversary annual gathering, held in our very own city of Chicago, was positioned as a reflection upon the development of the both society as well as of the profession as a whole. Naturally this included a fair number of historically minded sessions reflecting on accomplishments and challenges met. Ultimately, however, a very welcome focus on digital archives emerged over the three days of sessions as archivists, historians, programmers, and records managers addressed collective present and future challenges facing our institutions.

The panels and sessions were incredibly energizing, proving time and again that archivists of all stripes are willing and able to shoulder the responsibility for our digital cultural heritage. Archivists of old could content themselves with managing documents already consigned to history (excepting of course those working within the registratur system…there's always an exception); no so with today's archivists who are active in content creation, and with managing digital collections from the point of creation onward.

The quality of the open source tools that are emerging for all kinds of repositories is awe-inspiring: ArchivesSpace (which will merge the Archivists' Toolkit and Archon), Archivematica, Denver University's Records Authority, and Tuft's Taper project, to name only a few. The only dispiriting refrain common to all of these software systems is their proposed launch dates of 2013 and beyond…promises of a brighter future unavailable today.

As a lone arranger, for me what was most useful was hearing real-life reports about the practicality of existing tools for harvesting and managing digital content for our repositories: HTTrack, TreeSize Pro, Heretrix, Firefly SSN finder, TeraCopy… Hopefully by next year I will be able to report on our successes and learning curves with some of these tools at Dominican University.

Personally, I had to create a new tab on my Netvibes dashboard for all of the presenter's blogs that are now my required daily readings. Apart from my hard-won Deranger ribbon, I am left with an abiding joy from meeting so many inspiring and engaging archivists unafraid of the luminous glow enveloping our digital horizon.

Elsewhere in the Blogosphere IV: The Next Bloggeration

2011-10-06T16:21:00.001-04:00

The über-cool Rhizome has posted today about digital art and anti-aliasing and Columbia Center for Oral History blogs about their digital preservation initiatives. Enjoy!

McLuhan's Understanding Media: "The [digital] medium is [no longer] the [only] message."

2011-10-06T16:07:00.007-04:00

Happy Day of Digital Archives

David Kay, nydawg [New York Digital Archivists Working Group]
October 6, 2011
McLuhan: “The Medium Is the Message?” or “The [digital] medium is [no longer] the [only] message.”

This year marks the 100^th anniversary of the birth of “the new spokesman of the electronic age”, Marshall (Understanding Media) McLuhan, and digital archivists should take a moment to think about how media, digital and analog, hot and cool, and in many different formats change our jobs, lives and responsibilities. With threats of technological obsolescence, vendor lock-in, hardware failure, bit rot and link rot, non-backwards compatible software, and format and media obsolescence, digital archivists need a system to accurately describe digital objects and assets in their form and function, content, subject, object and context. If we miss key details, we run the risk of restricting access in the future because, for example, data may not be migrated or media refreshed as needed. By studying and understanding media, digital archivists can propose a realistic and trustworthy digital strategy and implement better and best practices to guarantee more efficiency from capture (and digitization or ingest) and appraisal (selection and description), to preservation (storage) and access (distribution).

Over the last ten, forty, one hundred and twenty thousand years, we have crossed many thresholds and lived through many profound media changes-- from oral culture to hieroglyphic communications to the alphabet and the written word, and from scrolls to books, and most recently transiting from the Atomic Age (age of atoms) to the Information Age (era of bits). While all changes were not paradigm shifts, many helped shift currencies of trust and convenience to establish new brand loyalties built on threats of imminent obsolescence and vendor lock-in. As digital archivists, we stand at the line separating data from digital assets, so we need to ensure that we are archiving and preserving the assets and describing the content, technical and contextual metadata as needed.

Want to read the rest? Check out the New York Digital Archivists Working Group [#nydawg] wordpress blog. Thanks for your interest in #DigitalArchivesDay #DoDA.

Adventures in web archiving

2011-10-06T15:51:00.009-04:00

As a web curator, quality control of archived websites is part of my everyday work. With the help of our able student assistants, we review each archived site to determine whether or not we have successfully captured the site. Basic quality control includes making sure that the archive site resembles the live site, internal site links work, we can download files (for example, PDfs of reports), and so on. More advanced quality control involves evaluating crawl reports and adjusting (and re-adjusting) the settings for a particular site. The advanced quality control happens when a site is not successfully captured using our default settings. Most of the time, crawl issues can be tackled and resolved by the combined brilliance of the curators, library programmers, and the unflappable Archive-It support staff. But occasionally, there are sites that seem designed to test your skills and your patience. Just when you think you’ve solved the problem and the road to a perfect capture seems clear, the site is redesigned, or changes URLs, or disappears only to reappear months later with a new host of issues.

Web Curator Nightmare: an incredibly valuable site disappears before it can be captured, then reappears months later at a completely unrelated URL, redesigned completely in Flash and Java and a programming language that didn’t exist five minutes ago, with content hosted on eight different servers inexplicably returning HTTP 404 codes all day except for three hours during the vernal equinox.

The Trials of Web Archiving: A Saga in Six Screenshots...

Victory! Almost!

Formatting is *slightly* off.

Defeat


This screen appears when something was not captured and/or is missing from the web archive

More defeat.


We originally captured the Arabic content, and then the URL changed...

Crushing defeat.

This shows the URL of a PDF that was not captured.

Victory at last! Again!

The new website, captured and functional.

Almost.

There should be content here...

Elsewhere in the Blogosphere, pt III: the Bloggening

2011-10-06T15:35:00.001-04:00

Check out the Library of Congress and NDIIP's digital preservation blog The Signal for a post on LC's many digital preservation resources!

Digital Archives at IIT

2011-10-06T15:26:00.003-04:00

Having only recently launched an institutional repository at the Illinois Institute of Technology (in fact, our first major digital archives initiative), I used this Day of Digital Archives as an opportunity to introduce the IR and some related concepts to the IIT community via our library blog: http://galvinlibrary.wordpress.com/2011/10/06/share-iit-and-the-day-of-digital-archives/.

Dana Lamparello
Metadata & Digitization Librarian
Illinois Institute of Technology

Sharing Our University Mission

2011-10-06T15:19:00.001-04:00

On September 27, Dominican University held its 2nd annual Caritas Veritas Symposium. This year's theme, "From Motto to Mission," explored how truth (veritas) and love (caritas) lead to action for “the creation of a more just and humane world.” (Dominican’s motto is Caritas et Veritas and our mission is to prepare students to seek truth, to give compassionate service and to participate in the creation of a more just and humane world.) Presentations included formal papers, panels, round-table discussions, and creative dance, and involved faculty, staff, students, alumnae/i and trustees. Our library offered to publish the proceedings in our new open access repository, Constellation. Last year, select proceedings were published in print and distributed mostly to the immediate Dominican community. This, year we wanted to share the proceedings with the world. After all, sharing our research and service projects with the world is itself an act of caritas that allows others to freely pursue the truth and to use that information to make the world more just. We’re sharing, for example, a theater professor’s work to end the death penalty in Illinois, a theology class’s trip to Native American reservations to teach children about climate change, and a Dominican Sister’s research into service learning and service work in the early years of the University. This unique collection of projects will now be preserved, accessible to the Dominican community and to anyone outside the University. We’re publishing the papers, slides and other materials as they are sent to us by the authors. Find them here.

An English Professor Among the Archivists

2011-10-06T15:07:00.008-04:00

My job is actually pretty simple. When people ask me what I do, I tell them I’m an English professor who teaches archivists about computers. What’s not to understand?

Okay, there’s more to it than that. The official story is that I’m an Associate Professor of English who also has a research and administrative role at a digital humanities center, the Maryland Institute for Technology in the Humanities at the University of Maryland. But that’s arguably even less illuminating.

Let me try a third tack. Here’s a picture of my office. Apologies for the mess, I’m actually kind of a neat freak, but this is an interesting mess. Maybe one way to talk about what I do is just to walk you through it.

So, we’ll start over on the left-hand side. That’s an Apple IIe, my first machine. My parents bought it for me (yeah, I was that lucky) back around 1982-3. It still boots today. Among other things, I keep it around to show students. Most of them have never used a computer that didn’t have a hard drive, so first having to select a disk and pop it into the drive before the computer does anything is a new experience for them. It’s signed by two people I have enormous respect for, Bruce Sterling (famous for, among much else, the Dead Media project) and Jason Scott, aka @textfiles. Both of them have been here to give talks, but the signatures also reinforce the way individual pieces of hardware become artifacts in their own right, something we’ve attempted to document in a more formal way. This computer, along with some Commodore 64s that Doug Reside used to keep around MITH when he was here, became the impetus for other people to begin offering us machines to add to our vintage collection.

Which brings us to the next machine over, an Osborne “portable,” which came into the shop in exactly that fashion. A friend--

Okay, not kidding, I was just this moment interrupted by a staffer from university relations who wanted to record an interview about Steve Jobs. We chatted about my early experiences using the Apple technology and he recorded video and audio of the old Disk II unit spinning up.

Anyway, the Osborne came to us courtesy of a friend in another library department who knew of MITH’s interest in vintage machines. As we’ve gotten more and more offers like this we’ve realized the importance of developing something like a collections policy, so that we’re not taking in lots of equipment we don’t really have any use for. The Osborne, though, is a real prize, and it too starts right up, though I was initially stymied by the fact that it wanted to boot from its B drive and I had to do a little online research to change the configuration settings. One thing I always impress upon my students when I co-teach (with Duke’s Naomi Nelson) the Born-Digital course at Rare Book School is the extent of the knowledge base that exists online in the computer enthusiast community.

Directly above the Osborne is a foam-board reproduction of a newspaper clipping. This in fact is the story and photograph that inspired Michael Joyce’s Afternoon, widely regarded as the first full-length piece of hypertext fiction (it was first released back in 1987). The original is in the collections at the Harry Ransom Center, where Joyce’s literary papers are on deposit. The “papers,” however, also consist of born-digital materials, including several laptops and several hundred diskettes. This image is a reminder of the complex relationships that emerge around hybrid digital/analog collections, something I wrote about extensively in my first book, Mechanisms.

The small wooden box in the background next to the Osborne? That’s an omnibus edition of Oregon Trail, one of the games in our case set as part of the Preserving Virtual Worlds II project. This is a multi-institutional effort funded by the IMLS that includes the University of Illinois, Stanford, RIT, and MITH at Maryland; we’re attempting to establishes a methodology for evaluating the “significant properties” of games and complex virtual objects, something we expect to become increasingly important to collecting institutions.

Now we get to some really interesting stuff. The old Sun tower that you see is there as a power supply for the 5 ¼” floppy disk drive sitting on top of it. This drive is what I use to create “images” of data stored on old magnetic media. As Jason Scott has powerfully stated, it may well already be too late for much of this generation of computer history, as the floppies are already well past their expected lifespan. But using a floppy disk controller like the FC5025 or the Software Preservation Society’s KryoFlux (which just arrived the other day), I’m able to tether the floppy drive to my current laptop and image the disk as a bitstream representation of the original data. The disk image can then be used as the basis for extracting individual files, or it can be loaded into an emulator. Right now I’m working my way through my stockpile of personal diskettes from the Apple IIe. We’ve also used the FC5025 to image materials for the Preserving Virtual Worlds work. What’s interesting about the floppy controller technology is that it represents a grassroots effort outside of formal academic or collecting institutions, something my colleague Kari Kraus has written about compellingly.

By the way, the blue box in the background behind the Sun tower is a Maxell Optical disk cartridge, a gift from Jason Scott. Visually it resembles a greatly enlarged 3.5” disk, and at Rare Book School I carry it on the first morning so our students will be able to tell our class from, say, the descriptive bibliography crew.

Which is one way of bringing me back to my vocation as a professor of English. Having been privileged enough to grow up with a piece of technology like the IIe, I’ve never really recognized the “two cultures” divide that supposedly separates the humanities from technology. I’d spend an afternoon reading a novel and then that evening hacking my way through an Infocom text adventure—or maybe even making a ham-handed attempt to program one of my own. My interest in digital preservation, however, comes from the realization, largely a product of the intellectual culture at the University of Virginia where I did my graduate work, that computers, like old books, are rich and multi-faceted objects, with their own unique stories to tell. Digital forensics thus becomes the analog to a vocation like descriptive bibliography, one that focuses obsessively on the material characteristics of its object of study. Already it’s obvious that anyone interested in a writer or other public figure from the 1980s forward will likely find themselves working with born-digital material of one sort or another as part of that individual’s cultural legacy. I see my work as a kind of descriptive bibliography for the 21^st century, bringing the kind of sensibilities we’ve cultivated with regard to books and other printed matter to bear on the objects and artifacts of our digital cultural inheritance.

There. Does that make sense now?

--Matthew Kirschenbaum

First steps: mapping out a new territory

2011-10-06T14:46:00.007-04:00

by Meg Tuomala

I started as digital archivist at Washington University in St. Louis’ University Archives in August. So I guess you could say I’m a digital archives newbie. And today, on this first Day of Digital Archives I thought I’d write about my first digital archives project in this position. So here goes--

Like I said, I started as digital archivist at Washington University in St. Louis’ University Archives in August. In this new position (for both me and the Archives) my first task was to assess the amount, types, and provenance of the digital objects and materials we already have in our collection. Born digital stuff, digital surrogates of traditional archival materials… A way to get a handle on the lay of the digital land. A survey of things as they are now, in order to map out how we see them (would like to see them?!) existing in the future.

So I poked around files, dug into directories, and explored the larger digital landscape of the libraries and the university. I didn’t look through the stacks for those disks hidden in boxes… yet (that’s another project) but I did do some serious surveying in places where I know digital assets do, should, and well, maybe-or-maybe-not live.

Of course I found a lot of great stuff we already have--recordings of campus lecture series, a huge amount of scanned images from the Archives, born digital administrative files...

But after taking a better look at what we do have, what I really started to think about what we do not yet have, what we’d like to have (digital University publications would be just one thing that tops that list). Because chances are if it’s not in the Archives, it’s probably not being properly cared for. And this is especially true for digital materials.

I believe that my preliminary survey of the digital assets we have will be useful. First of all, it introduced me to what I’ll be working with right off the bat. Secondly, it highlighted unexplored territories and places that need more work. And finally, it can and will help us make a case for resources that we’ll need to get in order to begin responsibly managing and preserving what we already have, and what we hope to have in the future. Exciting!

So if you’re just looking for a place to start I’d suggest doing your own survey. It’s a small step, but it’s a step in the right direction.

Of Resources and Reading Lists

2011-10-06T14:26:00.003-04:00

I have posted my 'Day of Digital Archives' post over on Spellbound Blog. A high points list of what I have spent my attention on in the past week, it includes links to resources and reading lists related to digital archives and digital curation.

Enjoy!
Jeanne of SpellboundBlog

Conversations Between Archives and their Users

2011-10-06T14:08:00.004-04:00

At the University of Wyoming's American Heritage Center, we've found that one of the biggest challenges we face is getting our potential donors to understand the importance of born-digital issues and the challenges we face in documenting contemporary cultural heritage. With that in mind, our contribution to the Day of Digital Archives is not about the mechanics of digital archivy, but a post on our institutional blog designed to speak to our users, to our potential donors. The post is hopefully the first of many future efforts aimed at educating our public. You can read it at: http://americanheritagecenter.wordpress.com/2011/10/05/archives-in-the-digital-age/

A Digital Dark Age?

2011-10-06T13:20:00.000-04:00

Was just reading this article about a 700 year old manuscript found here in our stacks at the University of Virginia's Albert and Shirley Small Special Collections Library. Listen to this quote:

In his essay, Greenia describes how rare it is to findmedieval documents. He estimates that for every volume that survived during theMiddle Ages, nine more were lost. Archivists have documented the disappearanceof parchments from the 13th and 14th centuries, “some perhaps rolled into tubesto make fireworks.”

During the Spanish Civil War (1936-1939), the contents of city archives weresometimes stuffed into the barricades. Since they didn’t have sandbags,said Greenia, books were used instead.

And before the municipal archives were transferred to Madridin the late 1970s, they “were still being kept in a top floor space subject to damp air and, when it rained,dripping water.”

Now think about digital documents in our current age. How many have you lost to a hard drive crash? To the damage done when you dropped your iPhone or laptop? How many of them have you forgotten were on your computer at all? How many will you back up or move to your next computer?

Some have said we might be living in a digital dark age if we don't come up with better ways to manage our digital content, and fast. What do you think?

The Challenge of Choices

2011-10-06T13:11:00.008-04:00

This post originally appeared on the Dominican University Library blog here.

written by Margaret Heller, Web Services Librarian, Dominican University

Like a lot of places, the library at Dominican University has a mix of systems we regularly use, and it’s often a challenge to determine what we want to use going forward.

For instance, we have our digital collections of images in a ContentDM repository, and are starting an insititonal repository for scholarly works in DSpace. These systems are great for what they do, but as we create more hybrid digital collections in the future, we will need to use additional systems for display and access. Where do we go from here?

I will use the example of a Civil War sheet music collection to discuss how we are looking at this challenge. This collection was digitized ten years ago as part of a grant. The original system for display and access was a simple series of static HTML pages and links to image files all in a directory. At one point it was on the library’s website, but as the library transitioned to a content management system and personnel changed over the years, the site and its files ended up in a folder on a server that was no longer online. I knew the files were there, since I’d taken a look at everything on the server, but didn’t know a whole lot else about it. One day a few months ago, however, someone asked me if I’d ever seen the files: since no one had seen anything of the collection for awhile, they wondered if the files had been lost. I showed them where the files were, but now came the next problem–how to make them available to the public again?

We could, of course, just transfer everything to the new library website
as it was.
.
But as you might imagine, web design and practices from 2001 look pretty dated in 2011. It seemed like a good chance to examine our digital collections practices. Our first thought was to install Omeka and use that. The problem with that idea is that our campus doesn’t have any Linux servers, which Omeka requires to run. The solution in that case would be to outsource that particular server need–our campus IT department, quite understandably, would rather not run servers they don’t have the expertise to fix when they break. We have held off on this, because another option was suggested–the whole campus was switching content management systems again, to a mix of Drupal and Sharepoint. Why not try using Drupal? I personally really liked that idea, as the more tightly integrated our digital collections were with other campus systems, the easier it would be for the rest of the campus community to find and use these collections.

But, as is hardly surprising, we find that this project is hard to find the time and resources to complete, so at this point we are in the same boat–though at least the files have been moved to be with the rest of the digital collection files. We hope to have something in place soon to experiment with, but until then we are researching our options. We like having lots of choices when it comes to what system to use, but it’s all too easy to suffer from analysis paralysis.

A Day in the Lives of Two Archivists Gone Digital - Vancouver, BC

2011-10-06T13:05:00.003-04:00

Go to AuthentiCity to check out today's Day of Digital Archives offering from the City of Vancouver Archives. Cindy McLellan and Courtney Mumma share some of their day-to-day duties handling digital materials.

<----Photograph by Mark Freeman (Free-ers on flickr). Hybrids, at first glance may seem to be cute and cuddly creatures, upon further investigation however, these unpredictable monsters, born of uncontrolled digital environments are actually quite frightening. Approach with caution and a team of knowledgeable people.

Creating an Online Collection with Born-Digital Content at Ball State University Libraries

2011-10-06T13:02:00.010-04:00

Hello all, and happy Day of Digital Archives! My name is Amanda Hurford and I am Metadata and Digital Collections Developer at Ball State University Libraries in Muncie, Indiana. I work closely with archivists developing and uploading content to the Libraries’ Digital Media Repository (DMR), which we use to provide online access to digitized versions of various primary source materials (photos, manuscripts, audio/video recordings, etc.). As of Oct. 1, we’ve created 113 digital collections, consisting of 115,837 assets and 343,011 individual digital files.

More and more, the source materials that arrive in our department are born-digital. To provide a glimpse into the life of a digital initiatives librarian (me), I’ve broken down one particular project that has been a big part of my day-to-day work for the last several months: the Roger Conatser Aerial Photographs Collection. The collection currently includes over 1,500 digital aerial photographs of Muncie and Delaware County, Indiana, taken by Muncie resident Roger Conatser in 2005. It recently went “live” at the end of July, but work continues on this ongoing project.

My share of working with this digital archive (and others) has consisted of several steps:

· Organization of the files: The digital photographs came to us on several CDs, and have since been copied to a backed-up server where our staff can easily access the files while we work with them. We also renamed the files. While keeping a log of the changes we made, we altered the file names from generic numbers to a naming schema that falls in line with our other digital projects. Each file now has a unique identifier which includes the archival number that was assigned to the collection.

· Upload into the content management system: Our team inputs the files into the DMR using CONTENTdm’s Project Client. This process includes auto-populating the “Digital Identifier” field with the file name, making it possible to track the master image down from the online version.

· Create and edit metadata: We spent a lot of time thinking about what kind of information would be useful in searching and browsing these images, and ended up with the following fields, among others. Many thanks go out to our local team of Delaware County experts for creating such amazing, useful records!

o Landmarks

o Identified Streets

o Thesaurus for Graphic Materials Subjects

o Area Type

o Date

· Maintain controlled vocabularies: All of the landmarks and streets our team identified contributed to the development of a hefty local controlled vocabulary that is frequently being tweaked.

· Add other materials: This is the step we are currently working on. In addition to the large archive of digital images, the collection consists of around 200 slides, over a thousand negatives of varying size, and countless prints. The slides and negatives have been digitized and are currently being described in CONTENTdm.

· Finalize the collection for archiving: Once we finish the remainder of the collection, all the master files, access files, and documentation (including an export of the metadata from the DMR) will be nicely packaged for long-term archiving. Next the collection gets passed to our technology team, who will redundantly archive the collection off-line.

This has been an interesting collection to work with, and an introduction to the inevitable evolution of the role of a digital initiatives librarian. As more born-digital materials need to be made available online, some of the more traditional aspects of my work will taper off (no sense scanning a print of a photograph if the original digital file is available). I will however, need to keep up-to-date with new digital formats and learn how to best work with them.

Amanda Hurford

Metadata and Digital Collections Developer

Ball State University Libraries... A destination for research, learning, and friends

AAHurford@bsu.edu

Seeing the Forest for/and the Trees, or a Day in the life of a Digital Archivist

2011-10-06T12:50:00.010-04:00

Greetings all, Sam Meister here to share my tale....

On this Day of Digital Archives I’m going to use a metaphor that seems appropriate to the new regional and geographical context to which I have recently migrated. I’m just starting month number three as Digital Archivist in the Mansfield Library at the University of Montana, a new position that did not exist before my arrival. After two months, there is still some definition of roles and responsibilities occurring, but more and more a clear path is being drawn.

Today I will be continuing the work I have been doing most recently: Project Planning. One of my main responsibilities is developing and implementing a workflow to collect, manage, and provide access to born-digital materials. At the moment, we are in the midst of creating the plan and structure which will frame the process of developing this workflow.

http://www.flickr.com/photos/johanroed/3163039837

This current mode is a bit like standing at the top of a mountain ski slope, getting ready to take the plunge down to the bottom. Before I take the plunge (or I should say we because there are definitely a number of others involved) I’m attempting to look down the mountain and chart the course. There are some obvious obstacles (Risks) that should be avoided if possible, (think large rocks, cliffs, giant trees, avalanche zones) but the perfect path is not entirely clear. I’m sure that once we start the descent there will be a number of switchbacks, diversions, and course corrections that will occur over time. We will likely need to stop at various points down the path and reflect on how far we’ve come and learn from any mistakes we’ve made.

At this point I have an idea of the type of skis (Tools: Archivematica, Curator’s Workbench) to choose from but haven’t tested too many of them out. Some might be good for certain areas of the mountain, such as the Accession Forest, while others may be better for other areas like the Meadow of Discovery and Access. Luckily, there are a number of tracks from those that have navigated down this slope before (AIMS Project) that I can utilize when negotiating the difficult and unpredictable terrain. There are a few hand-painted signs marking danger zones which others have left to warn those that would follow. I will do well to heed their warnings and avoid where others have fallen.

http://www.flickr.com/photos/82995349@N00/111054536/

For now, I am standing at the top of the mountain, charting the course that will hopefully lead to a successful journey to the Valley of Sustainable Stewardship.

At the same time we are drawing the map to ski the Born-Digital Descent, we are also charting the course to navigate the rapids of Institutional Repository River, explore the Cave of Web Archives, and scale the massive cliffs of Sustainable Digital Preservation Strategy. As Digital Archivist I am directly involved in all of these adventures, and am tasked with integrating all efforts into a cohesive Map of Managing Digital Content. The beginnings of this map are starting to surface on my wall, growing more and more as each day goes by…

Planning for the future, now, is a key aspect of working in the realm of digital archives. Being able to balance the big picture with the granular details of testing tools and understanding the risks of technical file format details is a important skill set to master. It’s an exciting time to be doing this work, because while the solutions are still unclear, there are more and more passionate individuals and organizations who are engaged and collaborating to develop the latest attempt to manage, preserve, and provide access to the huge amounts of digital content being created in our contemporary society.

If all else fails I can always start my new career as a creative writer (see above for potential as huge success)....

Surveying "Hidden" Electronic Media

2011-10-06T12:41:00.005-04:00

As a part of my position, I implemented a Condition Survey of the archival collections maintained by the three collecting repositories at Bobst Library. This survey will provide data on our collections that will inform the Preservation Department's strategic preservation planning for the Archives Preservation Program at NYU. This post at The Back Table (the NYU Special Collections collective blog) describes this survey in more detail. You can also view a poster about the findings of this survey to date (Summer 2011) presented earlier this year at the Society of American Archivists Annual Conference in Chicago.

The survey identifies preservation issues on a unit (or box) level as well as, in aggregate, at the material level. During the course of the first year of surveying, we uncovered a significant amount of electronic media stored on various magnetic, optical and flash storage devices. Our goal this fall is to target and survey collections, known or suspected, that contain electronic media for the purposes of identifying, quantifying and, later, prioritizing preservation for these items. Building a survey tool that can accommodate the appropriate information for identifying and quantifying electronic media storage required some research that took me to view presentations from last year's Rare Books and Manuscript Pre-Conference (scroll down to the seminar I. Born-Digital Manuscripts: A Primer) as well as Wikipedia. I am not afraid to say, as I have heard former professors admit and use heavily during my coursework, that Wikipedia is an excellent source for computer and technology history. From my research, I developed a guide for identifying various type of magnetic, optical and flash storage devices that the survey assistants will be able to refer to while surveying. Also, I reconstructed a portion of the survey tool to accommodate collection of information necessary for identifying and quantifying electronic media. For instance, manufacturer, disk format, label information and commercial/non-commercial status will help preservation staff determine migration requirements, potential file types, and storage capacities. This information will also help curators determine priorities for preservation in the future.

While this survey gets underway, I am sure tweaks will be made to the survey tool as well as our surveying techniques. If anyone is undertaking a similar project, I would be happy to hear about your experiences, discuss successes and failures, and long term planning.

Jennifer Waxman
jennifer.waxman@nyu.edu
@jwax55

Day of Digital Archives

2011-10-06T12:04:00.000-04:00

I have written a piece about the personal journey I have taken in the last two years and placed it on the AIMS blog - I hope people find it interesting

Simon Wilson
Senior Archivist (formerly Digital Archivist - AIMS Project)
Hull University Archives

How we do it at the State Library of North Carolina

2011-10-06T11:58:00.002-04:00

October 6 is the first annual Day of Digital Archives. It's an "initiative to raise awareness of digital archives. . . What are digital archives? Who uses them? How are they created and managed? Why are they important?" ( more here).

Read more about what the State Library of North Carolina does, why it's important, and what you can do to take some simple steps to save your digital content.

There'll be posts here at the DoDA blog and Tweets (#digitalArchivesDay) by folks from many different organizations throughout the day.

Still need more information about digital preservation? Check out information for burgeoning digital archivists or seasoned preservationistas on the State Library of North Carolina's digital preservation website at http://digitalpreservation.ncdcr.gov/.

We sure know how to party, don't we?

13 Million Documents and Counting

2011-10-06T11:52:00.002-04:00

I manage the Legacy Tobacco Documents Library (LTDL, http://legacy.library.ucsf.edu), a digital archive of over 13 million documents, and I scanned every one of them myself! (NOT) Most of the documents were spidered from websites that the major US tobacco companies are required by court order to host. In 1998 the companies agreed to this as part of a settlement of lawsuits from 46 states trying to recover money they’d spent treating people with tobacco-related diseases. And, in 2006 a federal judge, when she found them guilty under the Racketeer Influenced Corrupt Organization Act for creating an illegal enterprise for the purpose of defrauding the public, ordered them to continue making documents available until 2021.

Above: Typical document http://legacy.library.ucsf.edu/tid/dup28d00

As an archivist I’d had plenty of experience keeping track of things – photographs, financial records and textiles, for example, but I’d never worked with digital documents until I started as a project archivist with LTDL in 2004. It wasn’t until I started managing LTDL in 2006, though, that I learned that having a great computer programmer is the key to a successful digital archive. While some people evaluate success by the numbers of visitors to the website (we get an average of 15,000 visits each month) I look at our ability to correctly spider, parse and ingest hundreds of thousands of pages of documents, each with constantly changing metadata fields and values, from at least three websites each month as the measure of our competence.

The other key component to our success is a strong relationship with some of our users which include academic researchers, journalists, lawyers, public health advocates, government agencies and students. My office is in the same building as the Center for Tobacco Control Research and Education and I frequently consult with faculty and post-docs there about their research. I go to presentations about their work with the documents and am able to make decisions about LTDL with their needs and desires in mind. In the past month, LTDL had visitors from 132 countries and I am able to communicate with some of them via GlobaLink, the international tobacco control listserv. While we get many visitors who look at a document and leave, many of our users spend countless hours researching the tobacco industry’s advertising, manufacturing, marketing, sales, lobbying and scientific research activities. In fact, close to 600 peer-reviewed articles using the documents have been published.

My biggest headaches come from the donors of our material – the tobacco companies that do not want the world to see how they’ve manipulated science, lied to the public and covertly marketed cigarettes to youth, who are at greater risk of becoming addicted than adults. Whether from a lack of attention to detail, incompetence, or outright resistance, the companies continually fail to produce documents that should be uploaded to their sites, post downloadable metadata files that are missing fields or contain values that are different than those showing on the website, clutter up the corpus with duplicate documents and give incorrect document dates in the indexing information. There is no easy ingest of new documents, each download contains numerous subsets of documents, each requiring specialized handling. There’s always a tradeoff between just ingesting the whole batch as is, with duplicates, incorrect metadata, etc. and spending the time trying to fix the most egregious anomalies.

But, thank goodness that the Court ordered the cigarette companies to make documents and metadata available, warts and all. I also manage the Drug Industry Document Archive (DIDA, http://dida.library.ucsf.edu), a digital archive of pharmaceutical industry documents from lawsuits and government hearings. No court has ordered a drug company, even Merck which continued to sell Vioxx even after they knew of its increased risk for deadly heart attacks, to make their documents with indexing information available digitally. So, we’ve had to create our own metadata for the DIDA documents, something which is extraordinarily expensive when you’re talking about thousands of documents. With no original order, series or file folders, it’s item level description for these documents and as a result we’ve been unable to add many documents to DIDA. Crowd sourcing the creation of metadata seems like the only possible solution to this problem. But, meanwhile, it’s back to the task of spidering thousands of new tobacco company documents. It’s all in a day’s work!

Gilderoy Lockhart's Guide to Archiving the Sugar Quill

2011-10-06T11:27:00.000-04:00

I have been conscious of digital archives for a long time, even before I began my position as the Manager of Digital Collections at the University of Maryland Libraries. My concern and angst has been focused more on the born-digital materials that I have created over the years in my personal life, and the one that perhaps means the most to me is The Sugar Quill <http://www.sugarquill.net>.

The Sugar Quill: A History

In 2000, a close friend and I, both avid readers, simultaneously discovered the world of online fan websites and, specifically, fan fiction. We met on a website called "The Republic of Pemberley" <http://www.pemberley.com> as fans of each other's stories, both imagining alternate-universe happenings to the novel Pride and Prejudice. We also both shared a fascination with a new book series focused on a boy wizard named Harry Potter, and began to search for something like The Republic of Pemberley for the Harry Potter novels. It is difficult to believe now, but in 2001, the Harry Potter offerings on the Internet were slim, especially websites geared towards more civilized, intelligent conversation. Over the course of a bleak autumn in 2000, we conceived and designed the Sugar Quill. It was an opportunity for me to brush up on my HTML skills, and to examine some emerging technologies. We launched the Sugar Quill on January 5, 2001, with a garish gold background created by me after-hours with Paintshop Pro, the aim of which was to appear magical and Gryffindor-like. The website would have two major components: fan fiction and discussion forums. The fan fiction would be peer-reviewed - we would accept only a limited number of stories, and every author would have to submit to our "beta-reading" process, by which site volunteers would work with authors to correct grammar and spelling, and to counsel on matters of plot and character development. All stories also had to fall within what we considered the "canon" of Harry Potter, and for those that took place after the most recently released book in the series, Harry Potter and Goblet of Fire, the stories had to be what we considered "canon-compliant." We chose a system called EZBoard to host our discussion forums. For a small fee, we could have gold status on EZBoard, which hosted all of the forums, and allowed for unlimited discussion, different user groups, and moderation activities.

If you build it, they will come. We did not expect a lot of traffic. We told a handful of friends about the website and used it as our own personal playground. The initial costs were low - the EZBoard fee and a Yahoo hosting fee of around $10/month. The time investment was large, but it was a hobby, and for hobbies, time is not a concern. Soon, however, word spread. We made a few vague announcements on different forums, such as the then-dominant "Harry Potter for GrownUps" and soon passers-by started joining into the discussions. Within a year, we had so many fan fiction submissions that we had to close the open submission process and instead designate official "submission windows" for new authors. We would generally only accept a quarter of the authors who sent us their stories. The discussion forums remained relatively small, never more than 100 or so members active at one time, and the community grew strong, perhaps best evidenced by the reactions and postings on September 11, 2001.

By 2002, the site had grown exponentially. One day, I received an email from a bright fifteen-year-old in England, who had observed that it must be very time-consuming and cumbersome for me to update the Sugar Quill, and that he had developed a basic content management system using MySQL and PHP that he thought could help me manage the fan fiction archive. He also did not like EZBoard, and recommended that we obtain more server space and install an instance of Invision Board, an open source discussion board software, that we could manage ourselves. He was correct that the site was taking a large chunk of my time to manage; I uploaded every story manually, had to update links in various places, and was spending approximately 30 hours/week maintaining everything, even with a team of thirty volunteers. Not only had the British whiz kid programmed a new website for us, but he also had a friend, who, at age eighteen, was running his own webhosting company and who would be able to provide us with more support, server space, and bandwidth than Yahoo.

Early Archival Attempts

So we switched. What does this whole long history have to do with digital archives? I'm getting there. But bear in mind that, with the exception of a few hand-drawn images of quills that my friend and I had created and scanned to use as site logos, the Sugar Quill's entire existence was digital. Early on, we printed a few key planning documents, but most everything resided on my Toshiba laptop, a monster of a machine that had the loudest fan ever heard on a laptop computer and that displayed the blue screen of death at least once a day, on a few backup CDs, and in my own and various friends' email accounts, on a Yahoo server, and on EZBoard's servers. Being an archivist, backups and preservation were on my mind, but I did not have the tools to really manage all of this properly. There was no way, for example, to extract data from the EZBoard. If I wanted to "archive" notable conversations, I had to save the HTML pages, which is precisely the approach I took with the September 11 posts, and this is why these still exist on my computer.

Approximately a year after we switched to the new content management system and new server host, two things happened. First, I awoke one morning to several frantic emails to my personal email account, that the Sugar Quill was "down." I turned on some instant messaging software, and sent a message to my British server host. Apparently, all of the servers he had been using to host the websites were located someplace in Texas and the owner of the servers had absconded in the middle of the night with them. They were gone. Turned off. What could we do? I had asked repeatedly for backups of the MySQL database that now contained most of the Sugar Quill's information. I had saved what I could in my archaic, slow manner, but I did not have backups of any of the actual data. Luckily, my host did, and within a few days, everything was up and running again. The second crisis had to do with EZBoard. When we migrated from EZBoard to InvisionBoard as a platform for our discussion forums, we did not migrate the EZBoard discussions, but rather, linked to them from a link on our new site. EZBoard suffered some sort of huge server attack/crash and made a decision NOT to restore data for a large number of their defunct forums. With the exception of discussion topics that I manually saved, those old EZBoard discussions are now lost to history.

The Sugar Quill peaked sometime around the release of the seventh and final book in the Harry Potter series, Harry Potter and the Deathly Hallows in 2007. By that time, the original core of volunteers had all drifted away. Many of us had become and remain to this day, close in-person friends, but life and interests had moved us away from the daily interactions on the Sugar Quill. Many began writing their own original stories. People married, had children, returned to school. Besides, the series was over. While J. K. Rowling left much to the imagination, many questions were also answered, and it had been the speculation that fueled our obsession. New fan sites were everywhere, many with more advanced searching features than we could offer. My fifteen-year-old whiz kid was in college, and while he helped when he could, he also had moved on. Sites such as LiveJournal made it possible for anyone to post a work of fan fiction easily, without being at the mercy of a dictatorial fan fiction site and its hefty rules. We made a decision to "freeze" the Sugar Quill in time. At the time of the freeze, the Sugar Quill contained 2,634 stories and 1,168 fan art works by 958 authors and artists. These items collectively had close to 133,000 reviews. The discussion forums had 9,329 registered members and 411,406 posts.

How do you "freeze" a website? Around the same time, our British web host announced that he was moving towards focusing on providing high-speed internet services and discontinuing his web hosting. Our bandwidth needs had reduced significantly by 2008, and we determined that a basic Yahoo business web hosting plan would suit us well. He helped us move everything to the new server, and I put a notice on the website that it was in "Read Only" mode. I knew that there were still people who read the stories and the discussions and I did not want to take that pleasure away from them. I decided that I would continue to pay to host the Sugar Quill until it could be adequately archived elsewhere and interest was truly dead. That was three years ago. I am still paying.

Impedimenta! Impediments to Archiving a Website

Wayback Machine

What are the impediments to archiving? The Internet Archive's Wayback Machine has archived parts of the Sugar Quill. The part that most concerns me is the fan fiction archive. However, for any story that contains more than one chapter, the Wayback Machine fails. Why? We use a very basic pull-down menu that requires that the user make a selection to navigate from chapter to chapter. Because of this, the Wayback Machine does not follow the links. The Wayback Machine also has not captured any of the fan art. The reason for this I do not know. The Wayback Machine does not like our background image. We have a robots.txt file that indicates that the forums should not be crawled and I have not had the time to investigate the effect on the site if I remove that restriction. For there are other issues that go beyond simple backups...

PHP

In 2009, Yahoo updated the version of PHP that it was running on its servers. Specifically, two PHP configurations were deemed to be result in insecure code, and Yahoo disabled them. Unfortunately, my genius British whiz kid created the Sugar Quill's PHP code in 2002. I began to receive emails from concerned visitors to the now read-only site that certain pages were no longer accessible. Luckily, I connected with my whiz kid (now aged 52 and living with his seventeen children on a farm in Wales) on Facebook, and he corrected the problem, adding an amusing footnote in the remarks: "/* before you start reading this code, please let me apologise for the world of pain you are about to endure. ;) in my defence, this was my first major project, and it clearly shows. still, what a lesson in legacy code eh?"

Database corruption

Something is amiss with the MySQL database. Every few months, the database-driven portions of the Sugar Quill crash, and attempting to view them provides one with an "IBF forum error." Sometimes it would be weeks before a frustrated user would finally get through to me via one of half a dozen abandoned email addresses and social media outlets. Now, there is a Facebook group created by former Sugar Quill members who alert me quickly when a problem occurs. At that time, I log in to my handy Yahoo Web Hosting control panel and push the "Repair Database" button, which runs "myisamchk" on the MySQL database and somehow repairs it. I have no idea if I am losing data. I am not 100% sure what is getting corrupted, although I am suspicious of the Invision Board "posts" table, which is over 650 MB in size. I do not possess the skills, the time, nor the motivation, to investigate further. This is not, of course, how I would choose to manage an archival collection that I had vowed to preserve into perpetuity, but I also do not have at my disposal, a team of talented developers and engineers to work with me on solving these problems.

Passwords

It takes me at least ten minutes to successfully remember any of the correct user name/password combinations for the various administrative portions of the Sugar Quill. I should write them all down on paper and hide them in my house somewhere. In fact, right now would be a good time to do that. But where has all the paper gone? And will I remember where I've hidden them?

Security

And speaking of security... I thought that I had turned off all of the posting features for the Sugar Quill back in 2009. However, a quick glance at the forums today shows me that in fact, the last post was by someone named "Hematite" on July 21, 2011. I was amazed that people were still trying to post messages. I learned about this from Facebook, where giddy "Quillers" shared the information that posting was still possible. I've chosen to leave the loophole active, partially because I am not sure how to fix it. In addition, there have been numerous upgrades and patches to Invision Board since we "closed," and I have not kept up with the times. Security holes are everywhere, if someone were feeling particularly malicious. We are being spammed hourly. As a result of this posting, I discovered 33,000 pending registrations, all from what looked like fraudulent email addresses.

Conclusion

I wish I had some neat conclusions. But the problems that I have illuminated here are not unique, and I shudder to think about what is being lost by people who are even less technology-savvy than I am. I *tried* to archive. I am still trying. What I need is to set aside a week of my life, hire a technologically-savvy consultant, and address the many issues that I have observed that are stopping me from fully archiving this website. I have not even stuck a toe in the waters of discussing things like emulation and environment. After all, in my mind, users won't really get a true Sugar Quill experience, until they can view it with a laptop computer fan loud enough to wake the dead, just as I did when I was creating the site. New tools are being developed every day, and the way that people create web content are changing. But many of the issues are the same. Companies fail, servers crash, people hit "delete" and don't have backups. The first step is to be aware of the issues and to be able to work with creators of these resources to have preservation in the front of their minds. Just as many paper records were lost to neglect, fire, flood, and other disasters, so will electronic records be lost to neglect, crashes, failures, and other disasters. The goal, however, is to ensure that as much content is saved as is possible.

Digital Roundtable

2011-10-06T11:00:00.000-04:00

Last year, a colleague visiting Austin for a conference took me out for drinks after work. At some point she asked what Austin-area archivists were doing to get together. Well, I didn’t have a good answer, and she told me to get to work on that.

So I started thinking about what would bring us together as a professional group. (Can you guess where this is going?) We’d already toured each other’s repositories, and meeting up at the bar didn’t carry the intellectual rigor that would draw busy professionals. At the time I was just finishing up work on a digitization project that was funded by IMLS and administered by the Texas State Library and Archives. It had been a three-year project with a strong focus on training and collaboration across institutions and across the disciplines of librarianship, museum studies, and archives. So I was also thinking about ways to continue and cultivate the relationships built during the project. The project had brought together various institutions and their materials based on a common, usually regional, theme. It showed us that we could share resources and talk about things like scanning, metadata, storage, and display with our peers outside our organizations—even if we were not experts.

So that’s what we’ve done. I contacted Jennifer Hecker of the Archivists of Central Texas, the group that got those repository tours and happy hours happening, and we set up a roundtable discussion. We got a couple of folks to commit to sharing their recent projects to get the conversation started, and some people promised to pipe in and keep the discussion going. Turns out we didn’t need the ringers, and the discussion was lively and engaged. Everyone got to speak some, and we listened to each other’s projects and some challenges and successes. Our next meeting is today, October 6, and we plan to continue meeting regularly.

We decided on the roundtable discussion format rather than inviting speakers to give presentations because we felt there is enough training and information out there. What we needed was a place to talk and get to know each other and our projects, and network and build relationships, so we’d have resources and help when we needed it. Many of us were not trained in digital archives, but most of us have to deal with digital objects in some aspect of our jobs.
Right now we’re calling the meetings the Digitization Roundtable, and we chose that name on purpose. We wanted to create a certain comfort level, and felt that the words digital archives, born digital, and digital humanities would scare off people who were just getting started, perhaps without all of the resources they’d like. I expect the name to change at some point as the group grows and our conversations evolve.

And that’s what the Archivists of Central Texas are doing on the Day of Digital Archives.

Digital Preservation at the Parliamentary Archives

2011-10-06T10:54:00.000-04:00

At the Parliamentary Archives we’re working to develop a digital preservation capability for the UK Parliament. For the Day of Digital Archives, I’ve highlighted some of the things we’re working on this week, which will hopefully be representative of the kinds of issue many digital archives are addressing.

Our ambition is not only to be able to provide an equivalent level of service – in terms both of collection care and user access – for Parliament’s digital records as we have always done for paper and parchment, but also to support innovative ways of creating, managing and exploiting those information resources. We are very much a corporate archive: we preserve and make available the records of our own organisation or, in this case, the two separate institutions of the House of Commons and the House of Lords. As with most memory institutions, our digital preservation activities are being driven by, and hopefully anticipating, new digital ways of working, and the new kinds of information resource which result from these. For example, having the capability to preserve digital records with the same assurance as paper is a prerequisite for moving to digital-only publication and management of information.

We formally started work on our digital preservation project in 2008. As well as the major task of setting-up the technical infrastructure required - at the heart of which will be our digital repository - this is also tackling softer, but equally-important challenges, such as developing procedures and standards, training staff and raising awareness.

We’ve recently completed procurement of our repository software, and are now in the midst of detailed design work. The initial version of our repository will be operational by mid-2012, with further enhancement planned over the following years, moving to a fully business-as-usual basis by 2015.

So what are we doing this week?

· Ingest and delivery: We’re currently doing a lot of analysis to understand exactly how we’ll ingest different types of content into the repository, and how to present that content back to our users. Our collecting scope is essentially defined as any information created by Parliament which is considered to have permanent historical value; much will be the digital equivalent of information previously produced on paper – business records, published information about the work of both Houses, building plans etc. – but we will also receive entirely new kinds of information, such as websites. As well as born-digital, we’ll be collecting digitised surrogates of physical items. In each case, we need to identify how it will come to us, with what metadata, and how users will expect to see it. Subject to technical limitations, we aim to make all the open records we hold available online.

· Describing content: Closely related to this, we’re doing a lot of work to identify the metadata we need to describe the objects in our catalogue and manage them in the repository, and to map this to the metadata we can expect to receive on ingest. This will also enable us to identify any gaps, and think about how to fill them. We’re also thinking about some of the challenges of describing digital records. All archival records which we store in our digital repository will be catalogued as part of our wider collection, through our online catalogue Portcullis. We follow ISAD(G), the international standard for archival cataloguing, and we don’t want to use different approaches to describe analogue and digital records: our catalogue should be format-agnostic. However, ISAD(G) was developed in a pre-digital world, and its application to digital objects requires some careful thought.

· Characterisation tools: I’m in the middle of preparing a presentation on characterisation (and especially file format identification) tools for the GOPORTIS Digital Preservation Summit in Hamburg later this month. I would be really interested to know what people think about the current crop of tools and, in particular, where the major gaps are, in terms both of format coverage and tool functionality.

· Staff seminar: Today we’re starting to plan a seminar on our digital preservation project for all staff at the House of Lords. Setting-up a functioning repository is only half the story: we also need to ensure our potential depositors and users are aware of it, and understand its implications. We’re therefore putting a lot of effort into telling people about the project and what we’re aiming to achieve, and working with them to understand how it can help them. If anyone has useful tips on approaches which have worked for them, we’d be very interested to hear them – perhaps relating the institutional issues to personal data such as music or photos, or getting people to think about the digital information they rely on day to day.

· Web archiving: We’re in the midst of QA-ing the latest crawl of the Parliamentary web estate. We’ve been archiving this since 2009, taking thrice-yearly snapshots of c. 30 websites, and the resultant collection is now available online. The results of the QA process are invaluable in feeding back to our web team, so that we can prioritise improvements to the sustainability of the site.

· Demonstrating the benefits: We have a conundrum. Many of the benefits we expect our repository to deliver will only be demonstrable over timescales measured in decades, and indeed centuries. By definition, we can only show that we are able to maintain long term access to our records... well, over the long term. However, we need to be able to demonstrate some evidence of success much more quickly, not least to those who are funding our activities. So how do you demonstrate that a long-term repository is fit-for-purpose over the short-term? Perhaps the best we can do today is to use some of the emerging Trusted Digital Repository standards (such as TRAC, DRAMBORA and the Data Seal of Approval) to show that we meet current best practice. We’re therefore looking at how we might apply such standards.

Follow us @UKParlArchives or via #digitalarchivesday if you’d like to know more.

digitization: land of moving parts, self-discovery, and . . .

2011-10-06T10:53:00.005-04:00

Hi everyone! My name is Caryn Radick and I am a digital archivist at Rutgers University Libraries.

I hesitated a bit before asking to join the posting activities for the Day of Digital Archives (and thank you, Gretchen, for organizing). Although I fully expect that I will be called upon to help tackle managing born digital materials at my institution, thus far my experience has been in the realm of the digital to analog. I thought I would share some of my thoughts of different aspects of digitization that I've encountered thus far. All of these aspects will be important to keep in mind when moving into working with "born digital" material.

The Moving Parts Aspect

I haven't run into a digital project yet that didn't involve collaborative effort. My work on digital projects has provided me with wonderful opportunities, put me in touch with innovative, enthusiastic colleagues, and enabled me to be a part of a community of digital archivists and curators.

I am constantly amazed that there are so many moving parts to a digitization project. There are the different people (curators, conservators, the people working on scanning and inputting metadata, programmers and web designers) who are all juggling other responsibilities (finding a good meeting time for all of the players can be a challenge). The material may be simple to digitize, or may need special handling, special equipment, or reformatting. Levels of information about material vary (as does metadata). And the project may have a deadline. It's a challenge to have so many strands to pull together and kept track and yet not allow myself to be tugged too far into their differing directions.

The Prioritization Aspect

In some ways, setting priorities for digitization is easy (e.g., wanting material digitized because of an anniversary or event). Once we move beyond issues of timeliness, a number of factors play a role. Amount of material. Whether the material has been processed/described. What types of material are in the collection? How long will it take to get through the process? Do we have the right to digitize this material? And of course, what will it mean to researchers that this collection is available online? The latter is a driving force. This also brings us to..

The Time Management Aspect

Before embarking on my archival career, I worked in publishing where there was always a deadline. With digitization, sometimes a project comes with an absolute deadline, sometimes the deadline is more of a range (next semester…). I have found this aspect to be a particular challenge (especially in light of the moving parts and prioritization aspects). I'm finding my way on this one.

The "Self Discovery" [or, "What I've Learned About Myself"] Aspect

I constantly learn and relearn the lesson that I am not alone when it comes to questions I have regarding digitization (or other parts of my responsibilities). I will spend more time than I should trying to work something out that I think I should know, and then bring myself to the point where I will ask someone to explain only to learn that the same issue confuses them as well, or that my grasp on a situation is pretty much all there is to know and the issue I've brought up needs to be explored. So, I'm learning to spend less time wondering what I'm missing and more time trying to track down someone who can help me figure it out. I feel fortunate that there are many wonderful resources and people to turn to.

The "Things Change" Aspect

The words "evolving," "iterative," and "moving target" come up a lot when your work involves archives and technology. I knew that in taking on the position of digital archivist (a recent career development), that I'd be committing to seeking education and accepting that just because something had been done a certain way was no guarantee it would continue in the same manner. This is the aspect that reminds me that I may one day (and sooner than I think) write about my work with "born digital" or about something else on the horizon I'm not even aware of yet!