On Monday, 2nd December 2002, a meeting was held to demonstrate and discuss CAMiLEON's work in preserving the BBC Domesday project (a social record of UK life in the 1980s). The BBC Domesday project was created to celebrate the 900th anniversary of the 1086 Domesday book, but is now in danger of being lost through technological obselescence.The meeting brought together some of the original BBC Domesday videodisc developers: including Peter Armstrong from the BBC Interactive Unit, the Ecodisc's Roger Moore (who brought an original glass master of one of the Domesday discs), two of the editors, and some ex-schoolchildren who had contributed some of the content; experts on digital preservation; individuals interested in developing modern interfaces to the original Domesday data; some Acorn enthusiasts, including reporters - notably John Cartmell from Acorn Publisher and RISC OS Foundation's editor, Richard Hallas; as well as representatives from the local and national press.
The presentation
The first speaker was Peter Armstrong, who was heavily involved in the production team of the BBC Domesday project. He gave some interesting anecdotes and background material, about the highs and lows making of Domesday. (Read more about this below).
Then came Dr Tom Graham from CURL (the Consortium of University Research Libraries) who talked about digital preservation. He explained it was our duty to future generations - for both historical and technological reasons. Digital preservation needs to be understood by those at all levels, from data creators to end users whether national, institutional or individual.
The final speaker was David Holdsworth from CAMiLEON, who has worked in IT since the mid 60s, and is now an expert in digital preservation and storage. He explained how Domesday had been chosen as a test case to demonstrate the many problems in digital preservation and how CAMiLEON has developed strategies to solve these.
There then followed demonstrations of the BBC Domesday system running on the original hardware, and also of CAMiLEON's modern emulation that provides an accurate reproduction of almost all the original functionality.
A brief time of history
Peter Armstrong was 20 years a film maker and Head of Department at the BBC. He set up the BBC's Interactive Unit to make educational multimedia. In 1983, Armstrong wondered if it would be possible to celebrate the 900th anniversary of the original Domesday Book by producing a modern day equivalent. It was an ambitious idea, but it captured the imagination. It was planned that the first copy would be given to Prince William, the "poetic successor to William the Conqueror".
Funding for the project - an estimated budget of £2million - was relatively easy to source. Multimedia was an exciting, upcoming technology; people involved in education, national archiving as well as computing were keen to push it forward. The BBC put together a team of around 60 staff to develop the project and recruited pupils from over half of all the schools in the country to help produce the content. In all, around a million children were involved from 14,000 schools.
Resource gathering
The map of the UK was divided into blocks, each measuring 4:3km - it is no coincidence that this is the ratio of a television screen. Each map block was adopted by a school. The UK consists of over 25,000 blocks but it was only practical to cover around half of these. It was difficult to find schools that were in the more remote areas of Scotland and Wales, but the majority of England was well counted for. Pupils investigated the land use, counted numbers of doctors, post offices and so on, and wrote articles about the people and buildings there. Each area was allocated 20 screens of BBC text and three photographs.
Back in the classroom, the school computers - a large user base of BBC Micros - were used for data entry. The articles were typed in and sent on floppy discs to the BBC. The text was left unedited - any spelling mistakes or typing errors remained in the final print. The only alterations were prompted by the lawyers. They found that some descriptions of local characters "could cause us some problems".
The hardware
Two years were spent developing the hardware. Philips, the only manufacturers of videodisc players in Europe, were approached to produce the laserdisc player. This is actually a SCSI device - the original SCSI specification had only just been confirmed. This meant a SCSI interface had to be developed for the BBC Master - the player looked like a large and slow hard disc to the computer. The BBC Master had a special read-only version of its DFS called VFS (Videodisc Filing System). You could control the laserdisc player using standard *Commands.
The laser videodisk player produced PAL video, and the BBC Micro also produced a PAL-like video signal. The player carried a genlock and video mixing board to combine the computer and disc pictures.
Other hardware was developed for the BBC Master, including a co-processor and a trackerball. The trackerball featured three buttons - Acorn had the right idea even then! The original Acorn trackerball in CAMiLEON's office has a label 'Action' over the left button, the middle button was 'Change', and the right button is unlabelled (the right button is unused in the BBC Domesday program).
The software was written by Logica using BCPL, a forerunner of C. In total, over 70,000 lines of custom code were written.
Into production
"If we'd known the problems involved we would never have attempted it" said one of the staff. The project was completed on time and on budget, thanks to the remarkable work of the team. Over 24,000 maps and 200,000 photos were processed. Remember that there were no digital copies to work from - the paper originals of the maps were quite literally "cut and pasted" together. Each map and photo was captured as a single frame of continuous video tape. These then had to be captioned, and have their copyright cleared. As well as this, over 8000 data sets - traffic congestion, radiation levels, etc - were stored.
The size of the Domesday project is overwhelming. The budget of £2million sounds a lot - but the real cost must be far, far more than that when the dedicated work of all the school children and volunteers is considered. It has been estimated that if you worked a 40 hour week viewing Domesday, it would take seven years to see all the information. One source calculated that it should have cost a quarter of million pounds for institutions to access that amount of data, which made the price tag of the Domesday system sound like a bargain.
When the first plans for the project were announced the estimated price was £1100. When the Domesday came to market this had turned into over £4000. This was too expensive for most libraries and schools and the Domesday became a commercial flop. The first set of discs were presented to the keeper of records at the Public Record Office, to be placed alongside the original Domesday book.
Life went on. The BBC Interactive Unit developed a few other ideas, but eventually folded when the Director General decided there was no future in multimedia. Armstrong and a group of colleagues bought out the department and set up the MultiMedia Corporation. Reworkings of the Domesday ideas appeared in other forms: the 3D World Atlas (Domesday on a global scale) sold over a million copies; OneWorld.net features Another Domesday which focuses on global justice issues, and is one of Kofi Annan's favourite web sites. The BBC Domesday project became an icon, the granddaddy of interactive multimedia. And then it became obsolete.
How to preserve a time machine
The CAMiLEON project - Creative Archiving at Michigan and Leeds Emulating the Old on the New - has spent three years developing strategies for digital preservation and testing them with materials such as the BBC Domesday system. The BBC Domesday project encapsulates many difficult problems encountered by those working in the field: a huge amount of multimedia data, technological complexities, and the intellectual property rights (IPR) issues.
There are several aspects to preserving the BBC Domesday. There is the decay of the media - discs get scratched during use and become less reliable. The hardware to read these discs is rare, and the few remaining laserdisc players are prone to break down (and require very specialist repair). All of the hardware is long past its shelf life - BBC computers have always been durable, but not many were produced with the special Domesday extras. The Domesday system also had a particular 'look and feel' which requires preserving in addition to the actual content.
Rescuing the resource
CAMiLEON obtained access to a semi-working Domesday system donated by the School of Geography at the University of Leeds. One of the first tasks to preserve BBC domesday was to transfer the data files from the 12 inch laserdiscs to bytestreams accessible on modern hardware. A Linux PC could be connected to the laserdisc player using a SCSI cable, allowing the PC to read the text articles and database. Images, including still-frame video, were transferred to a PC using a standard video frame-grabber card at maximum resolution. These images were stored in an uncompressed format to avoid quality loss or the introduction of artefacts (as can occur with JPEG compression). In total, around 70GB - yes 70 gigabytes - of data was transferred per side of each laserdisc. The correct choice of abstract form for representing this data is crucial for accurate and efficient preservation over time.
The next step was to develop software that emulates the adapted BBC Master computer and the laserdisc player on which the original BBC Domesday system ran. An open source emulator - (BeebEm - was used as the starting point for this software. Emulation of the specific Domesday system hardware had to be incorporated by CAMiLEON - which included the co-processor, SCSI communication, and the many functions of the laserdisc player.
Preservation strategy
CAMiLEON's philosophy is to preserve the data in its original, unmodified, format (ie. the original abstract bytestream, not on the same physical medium). Software can then be written to use this data: perhaps an emulation of the original system; perhaps a tool that reformats it into a modern format; or software that provides a new interface to the data. For the BBC Domesday project, CAMiLEON developed an emulation of the original system - and knowledge of how the original system works is encapsulated in the emulation software. The emulation software, together with the abstracted data, provides a record of the original BBC Domesday system.
To avoid problems of the emulation software becoming obselete, it is important to ensure that the software is not chained to any specific operating system or machine architecture. Careful development with a clear focus on the goal of longevity will make it easier to run this software on a future (as yet unknown) computer, needing only a few simple (and well documented) modifications. This also means that it should be possible to port the emulation software to any current machine.
Currently the CAMiLEON BBC Domesday Emulator only runs on Windows machines. This is because, due to time constraints, a Windows based emulator, BeebEm, was used as the starting point. This was not written to follow software longevity guidelines, so it is tied to the Windows platform.
However, the developments that CAMiLEON have added are much more portable, so there is no technical reason why these could not be incorporated into a RISC OS based BBC emulator, or for the Windows-specific parts of the emulator to be made portable. In fact, one developer of a RISC OS BBC emulator has already contacted CAMiLEON to discuss making Domesday available for the platform.
Distribution and copyright
Sadly, it is unlikely that Domesday will become available for the general public to use. The contents of the discs are heavily tied up in copyright - parts are owned by the BBC, the Ordinance Survey, and possibly the Local Education Authorities and schools. However, it may be possible for owners of original BBC Domesday laserdiscs to gain access to the preserved data, and that the emulator software can be made publically available. CAMiLEON's updated Domesday pages offer more detailed information about these IPR issues.
The recent auction of a BBC Domesday system on eBay has revived a lot of interest in the project. There is already talk of extracting the data from the laserdiscs and combining it with freely available modern maps (such as those from streetmap.co.uk). There are a couple of people working to produce modern interfaces to the original Domesday data - they met each other for the first time at the CAMiLEON meeting. At the end of the CAMiLEON project, the emulation software and data will be deposited at the Public Record Office to go alongside the 12 inch videodisc system.
CAMiLEON are also recording the experience of using the Domesday project. They are writing an account of the original system - including descriptions of what the hardware was and how it was used, because trackerballs and even keyboards may become unfamiliar peripherals. They are hoping to co-ordinate the preservation of documentation associated with BBC Domesday - if you possess any documentation or interesting related publications please contact CAMiLEON via Paul Wheatley at p.r.wheatley@leeds.ac.uk
Further reading
- The CAMiLEON web site has lots of useful information about its work with BBC Domesday and other digital preservation research
- Andy Finney's detailed account of BBC Domesday
- Pictures from the CAMiLEON meeting taken by Gareth Babb
- Digital Domesday unlocked - BBC news
- Experts rescue BBC Domesday project knowledge - Ananova
- Digital Longevity - the lifespan of files
- Acorn Publisher will have an article about Domesday in their December 2002 issue
