From giant website tables to a usable historical archive
Burra History Group website and BDA v2 content analysis · Working report · 13 November 2026
Status: Architecture and content recommendation. Based on the completed July website audit, the 28 September 2026 BDA v2 analysis, the deterministic BDA-002 census of 22 original HTML sources, and the BDA-003 Burra in the News overlap audit. This report proposes future content restructuring; it does not claim that the record-by-record migration is already published.
Executive summary
The existing Burra History Group website is much richer than its page count suggests. A rebuild audit identified roughly 150 substantive HTML content pages and approximately 500 MB of retained website assets. But a subset of just 22 legacy Genealogy / Databases pages contains 20,883 candidate entries, names or inscription lines. These include thousands of cemetery records, school admissions, signature-book entries and soldiers' research dossiers. Many are squeezed into giant HTML tables because the old website had no better way to present them.
The separate BDA v2 investigation broadened the old Databases navigation to 28 items and found 21 clearly collection-shaped, or 23 if memorial structures are included. Some remaining items are historical essays with embedded evidence rather than databases at all.
The proposed answer is not to copy each enormous HTML page unchanged, nor to treat every table row as a person. We should preserve the original pages and source collections, extract the smallest historically meaningful records, publish independent collection landing pages, and offer public browsing views that can combine records across sources. Authored history articles should be able to incorporate these records without swallowing or rewriting them.
This also gives a useful lesson for the Burra Digital Archive: an archive's intellectual size may be measured in tens of thousands of research objects even when it occupies only a few hundred megabytes on disk.
1. The website that was audited
The BHG site audit, content map and migration checklist document the earlier public-site rebuild work.
The old site was a hand-built collection of static HTML pages, iframe hubs, mobile duplicates, maps, tables, images, books and documents. The sitemap contained 153 URLs; the retainable content was estimated at around 150 HTML pages, with approximately 500 MB of media and documents. Separate mobile pages did not cover many important genealogy tables.
The July rebuild decisions were sensible for website continuity: one responsive layout, topic hubs instead of deep sidebar navigation, a retained interactive town map, removal of shop and accommodation material, and normal links to church and Town Hall pages. The migration checklist records a hand-pass of those site pages as done, including 22 Genealogy / Databases pages and around 139 place pages.
Important distinction: that website migration made the content readable within a modernised site. It did not automatically turn the giant tables into individual BDA archival assets. The later BDA v2 analysis is a separate and deeper content-conversion programme.
2. The discovery: the old Databases menu is a latent archive
The 28 September BDA v2 analysis identified four fundamentally different structures behind the same old website label:
- Source/record collections: independently meaningful burial entries, school admissions, newspaper references, signature-book entries, property assessments and mine records.
- Curated dossiers: compiled research about soldiers, pioneers and notable people, where each entry is richer than a table row.
- Articles with structured evidence: narrative arguments that incorporate datasets, such as Origins of People in Burra, Chinese in Burra and council history.
- Single historical objects or indexes: one memorial with a list of names, or a bare index of early residents that does not yet justify inventing biographies.
Across 28 old Databases navigation items the original classification was:
| Type | Items | Preferred treatment |
|---|---|---|
| Clear source/record collections | 18 | Collection landing page plus granular records |
| Curated research/dossier collections | 3 | One compiled dossier per meaningful subject, with methodology |
| Article + structured-evidence hybrids | 3 | Preserve authored essay and separately extract evidence |
| Memorial/object structures | 2 | Preserve object and ordered inscriptions; link to related people |
| Mainly article or index material | 2 | Retain intact unless sufficient evidence supports a finer record grain |
| Total | 28 | 21 clear collections, or 23 including memorials |
The mechanical census below covered the 22 HTML pages in the BHG genealogy migration script. The six additional database-nav items are War Memorials & Honour Rolls, Germans in Burra, Chinese in Burra, Councillors, Other Mines and Mongolata Mines. They were analysed conceptually, but are outside that 22-source census.
3. What the mechanical census actually counted
BDA-002 ran a deterministic, read-only examination of the original source HTML, using the local historic website snapshot. It preserved original-byte SHA-256 hashes and reported headers, HTML rows, continuation rows, uncertainty, links, introductory prose and provenance.
This was not an LLM guess based on filenames. The exact source-by-source results are in the census report.
| Legacy source | Source data rows or items | Candidate meaningful unit |
|---|---|---|
| Burra Cemetery Register 1858–1899 | 2,335 | 2,335 interment entries |
| Burra Cemetery Register 1900–1988 | 2,650 | 2,650 interment entries |
| Headstone transcriptions 1970–1999 | 288 | 288 transcription rows; monument boundaries uncertain |
| Booborowie Cemetery | 213 rows | 194 interments; 19 letter headings |
| Soldiers WW1 and prior | 1,658 | Compiled service-person dossiers |
| Fallen Soldiers memorial | 102 names | One memorial with 102 inscription lines |
| Origins of People in Burra | 355 rows | 349 origin/birth observations, plus authored article |
| Weddings | 1,428 | Newspaper marriage-reference entries |
| Burra Pioneers | 104 | Compiled person dossiers |
| Burra Record of Births | 903 | Newspaper birth-notice reference entries |
| Burra Notables | 20 | Compiled person dossiers |
| School Boys register | 1,937 | School admission events |
| School Girls register | 1,818 | School admission events |
| Redruth Girls Reformatory | 161 | Admission/file entries |
| NAB Signature Book 1 | 869 | Signature entries |
| NAB Signature Book 2 | 1,434 | Signature entries |
| NAB Signature Book 3 | 1,722 | Signature entries |
| 1873 North Ward assessment | 39 rows | 34 numbered assessments |
| 1873 South Ward assessment | 36 rows | 35 numbered assessments |
| 1873 Kooringa Ward assessment | 480 rows | 410 numbered assessments |
| 1873 Redruth Ward assessment | 875 rows | 763 numbered assessments |
| Miners and early residents to 1877 | 1,669 | Bare index names; not 1,669 demonstrated dossiers |
| All 22 sources | 20,883 candidate records, names or inscription lines, mixed kinds |
The 20,883 is a useful measure of the site's information granularity, not a clean count of unique people or independent historical documents. It deliberately includes 102 names on one memorial and 1,669 bare names in an index. Repeated admissions and references are retained rather than deduplicated.
Some useful sub-totals: 4,985 Burra cemetery interments; 4,025 bank signature entries; 3,755 school admission events; 1,242 grouped 1873 assessment entries. Even these should not be treated as numbers of distinct people or places.
4. Why one HTML row is not always one archival record
The census exposed real irregularities in the original sources:
Cemetery registers: interment, not plot or person identity
Several individuals may share the same plot. In the 1858–1899 register, 499 plot numbers were repeated across 1,340 rows. Some entries lack names, dates or plot identifiers. Preserve their original values and uncertainties. A plot is a relationship or browsing key, not the primary identity of each burial entry.
School registers: admission event, not unique child
The Boys register contains 1,937 admission rows but 1,723 distinct normalised child names; the Girls register has 1,818 admission rows and 1,583 names. A child may return to school, change residence or move between schools. Identical names can also describe different children. Keep each admission as recorded; link people only when supported.
Weddings and births: newspaper references, not civil events
The Weddings page has 1,428 references, including 77 normalised couples with repeated references. The same marriage can be announced on different dates or pages. Likewise the 903 Births rows are references to published notices. Preserve newspaper date, microfilm page and column, and do not claim the row is a civil certificate.
Assessment books: numbered entries may span rows
The Kooringa table has 480 data rows but 410 assessments; the Redruth table has 875 rows but 763 assessments. Continuation rows with blank numbers belong to the preceding assessment. A naive per-HTML-row import would corrupt property descriptions and inflate counts.
Memorials, dossiers and bare indexes
The fallen-soldiers material is one memorial object with 102 inscribed names, not 102 freestanding biographies. The 1,658 Soldiers entries, however, are substantive compiled research dossiers with distinct editorial value. The 1,669 early residents/miners names are mostly a bare index; creating 1,669 thin person pages would imply research that has not actually been done.
These rules should be part of the conversion acceptance tests, not left to downstream cleanup.
5. A better model: source, record, lens, article
The BDA analysis proposes four distinct concepts:
| Concept | Example | Purpose |
|---|---|---|
| Source collection | Burra Cemetery Register 1858–1899 | What work or holding was created, by whom, with what gaps and provenance |
| Record asset | One dated burial/interment entry | Smallest independently meaningful unit to search, cite and link |
| Browsing lens | Cemeteries & Burials | Human-facing view spanning different original sources |
| Authored article | History of Chinese settlement in Burra | Narrative and interpretation that can cite or compose record sets |
A collection landing page should include creator/compiler, custody, date coverage, source format, field definitions, abbreviations, errors and omissions, transformation method, and links to originals wherever possible.
A record should retain its source collection ID and original coordinates — for example page/column, book/page, row or assessment number — and include original text alongside clearly marked derived fields. Stable identifiers need not imply that two records referring to the same person are the same historical object.
One source collection can contribute to several public topics, and one public topic can combine several sources. This is the crucial separation the old site could not express.
6. Proposed website and archive browsing structure
Instead of a top-level Databases dumping ground, provide coherent ways into the historical material:
| Public entry point | Source material it can combine |
|---|---|
| Cemeteries & Burials | Burra registers, headstones, Booborowie, grave-related material |
| Births, Marriages & Families | Birth-notice index, wedding references, origin observations, family histories |
| Schools & Institutions | Boys and Girls admissions, Reformatory records, school archival documents |
| Military Service & Memorials | Service-person dossiers, memorial and honour-roll objects, related BITN references |
| Property & Assessment Records | Four 1873 ward assessments, later relevant assessment and council material |
| Pioneers & Notable People | Curated biographies with references, not bare name stubs |
| NAB Signature Books | Three source books in one public browsing experience |
| Mining & Industry | Other Mines, Mongolata and later mine/claim records under the mining history topic |
| Browse Collections | Direct access to original collections, creators and provenance |
| Research Guides & Sources | How to interpret the material, bibliographies, external institutional links |
Historical topic pages remain important. A Chinese in Burra page might contain explanatory narrative plus views of burial, marriage, assessment, newspaper, court and photography records. Origins of People in Burra should preserve Eric Fuss's argument about settlement patterns and expose its 349 observations as inspectable supporting evidence. Council/Councillors should separate its narrative, election/term evidence and researched people.
The town map remains a parallel spatial navigation tool to the same underlying place records. The July rebuilding strategy already provides a model: approximately 134 map hotspots, a list of about 142 places and around 148 history modals were to become individually addressable place pages while preserving the map.
7. The Burra in the News overlap: preserve relationships, not artificial independence
BDA-003 compared the legacy database census with 129,480 structured Burra in the News records. This confirmed that some BHG “databases” are later indexes or enriched dossiers built from the same newspaper observations represented in BITN.
| Legacy family | Exact or strong BITN links | Interpretation |
|---|---|---|
| Weddings | 904 / 1,428 (63.3%) | Derived newspaper index rows can be linked to BITN observations |
| Births | 468 / 903 (51.8%) | Separate birth-notice index rows and newspaper-note records |
| Soldiers | 194 / 1,658 dossiers linked | Some dossiers aggregate several newspaper records; 46 one-to-many cases |
| Pioneers | 65 / 104 dossiers linked | Obituary evidence plus later research and burial metadata |
| Notables | 6 / 20 dossiers linked | Enriched biographies with partial correspondence |
All five hand-checked examples were recovered; 277 surname-only matches were rejected. Origins and Miners were correctly left without mechanical row-level links.
Never erase the later BHG index or dossier merely because its newspaper source exists in BITN. Both are historical artifacts. The compiled work may add editorial judgement, correction, references and coordinates unavailable in the underlying source. The right relationship is derived from, cites, enriches or associated with, with evidence and confidence, not forced deduplication.
Conversely, a dossier based on BITN is not independent confirmation of BITN's newspaper observation. The public interface should make provenance and derivation intelligible.
8. Conversion and publishing approach
The original website HTML, media and books remain immutable source evidence. Extraction and rendering should be separate, reproducible layers:
- Inventory and secure originals. The 22-source census is complete. Preserve source bytes and hashes. Extend the census to the six additional database-nav items and other giant research pages as appropriate.
- Define collection families and record grain. Use verified parsers for cemeteries, school events, NAB entries, property assessments and birth/wedding reference rows.
- Use LLM assistance for irregular and authored material. Extract dossier boundaries, section structure, citations, tags and image relationships with versioned prompts and explicit validation. Preserve authored text rather than casually rewriting historians' prose.
- Generate reviewable BDA asset plans. Each output needs provenance, source position, original/derived distinction, deterministic ID, references and relationship proposals.
- Run source-specific quality checks. Counts reconcile to the census, and no source text is silently lost. Continuation rows, repeated dates, malformed rows, incomplete sources and rights caveats require tests.
- Publish incrementally. Begin with one or two clean record families, then dossier and hybrid pages, while maintaining the existing site and its editorial process. Present collection landing pages and cross-collection lenses as the corpus grows.
- Cross-link to BITN where justified. Keep strong links explicit; leave weak or unmatched records searchable via ordinary related-item discovery rather than inventing precision.
- Retain human review and site autonomy. BHG members must be able to approve research representation, provenance and public exposure. An agent can help discover, organise and edit, but does not become the source of historical authority.
BDA v2 deliberately separates wf-store's storage/search capabilities from higher-level organisation, interpretation and rendering. A visitor-facing archive should not require a rigid relational data model before content can become useful.
9. Concrete next decision and scope boundary
The BDA-004 conversion ticket was Ready in the documentation examined for this report. It is a parallel content workstream; it is not a prerequisite for the main BDA v2 platform work with BITN and Ian Auhl. Its specified output is versioned asset plans, not immediate production writes or publication.
A practical first delivery could be:
- The two Burra cemetery register sources, retaining incomplete interments and plot relationships.
- The Boys and Girls school admission registers as dated events.
- The three NAB books, keeping book/page coordinates.
- The 1873 ward assessments, with continuation-row grouping.
- The Weddings and Births indexes with their already-measured BITN relationship candidates.
Then tackle the richer Soldiers, Pioneers, Notables, Origins, Chinese and Council materials with editorial review. Importing the old HTML as an intact source asset remains useful even before granular conversion succeeds.
Do not confuse three milestones: (a) the legacy website was copied and modernised in the BHG rebuild; (b) its database sources were counted and analysed in BDA-002/BDA-003; (c) record-level BDA publication remains subsequent implementation work.
10. Consequences for understanding the size of Burra's information
There are now at least two grounded scales: 129,479 BITN website records, plus a 20,883-item mixed candidate census hiding within 22 BHG legacy pages. These are not 150,362 unique historical facts or independent source documents; some overlap, and the counts include indexes, names and inscription lines. Nevertheless they reveal why counting “web pages” badly understates a local archive's searchable richness.
The record-extraction work is likely to add many small textual/search assets while consuming relatively little raw storage compared with scanned newspaper pages, photographs, oversized maps and audio. An archive's physical digitisation budget, number of addressable records and intellectual discovery value must therefore be estimated separately.
This strengthens the larger community-archive argument: valuable local records already exist, often hidden inside old websites and researcher compilations. We can make them accessible without throwing away the character, provenance or scholarly contribution of the original work.
References and implementation trail
- BDA v2 — Legacy Databases as collections and browsing lenses, 28 September 2026: full 28-item classification and proposed model.
- BDA-002 — Census the legacy BHG database sources: completed deterministic source audit, parser hazards and evidence.
- BDA content — complete 22-source legacy census: 20,883 candidates and per-source counts.
- BDA-003 — BITN overlap audit: verified correspondence methodology and rates.
- BDA-004 — BHG content conversion: next proposed conversion workstream.
- BHG site audit, content map, migration checklist, and navigation model.
- BHG interactive map strategy: hotspot/detail content separation.
- Companion working report: Burra's historical information — sources and size estimates.