How It Is Made
To enable the creation of the catalogue (a database of all artwork details), we used a web service/website called Airtable. This allowed us to have multiple people on the team add to the database at the same time (mostly just myself and the archivist). It also featured easy to use UI for creating and modifying tables allowing for an incremental DB design enabling a more agile approach as we discovered needs or problems. Airtable also had a nice feature which allowed us to upload images and have them be visible as thumbnails as we were working on the catalogue. This feature made the project much easier as we had pictures of the artwork we could reference when needed.
From the start, I knew that writing a program to generate the HTML would require a database I had full control over — Airtable alone would not be sufficient. Airtable is a closed system: it provides a polished UI, but it does not allow direct database access or SQL queries. Being able to write and run arbitrary SQL was essential for developing and testing the more complex parts of the system. So I created a database migration system to move the data from Airtable to a PostgreSQL database. The system is comprised of 3 programs:
I then wrote a C# console application that reads the PostgreSQL database and generates the complete set of static HTML files that make up this website — including the very page you are reading now. Each time it runs, it regenerates all pages from scratch, pulling the latest artwork data, image references, and metadata from the database. The result is a fully self-contained static website with no server-side code required to view it. The date and time the pages were last generated is recorded at the bottom of every page.
The pictures of the artworks were taken by a professional photographer and color processed by him then uploaded to web cloud storage using file names stored in the DB. (This allowed the HTML generator to reference them in the page generation.) In a similar way the scans were performed by professional scanners and stored on the web cloud storage. The scanners provided a metadata record for each scan containing: the filename, sketchbook number, page number, date (when readable via OCR), and a notes field — used so far to flag pages that were attachments taped into the book rather than bound pages. This metadata was uploaded to the database and is what allows the HTML generator to correctly organize and display the sketchbook pages.
In addition to the sketchbook scans, we also have a collection of scans of various documents related to Keith's life and work. These include: slides, photos, drawings, letters, notes, clippings, and other miscellaneous items.
Our internal team also scanned some of the smaller items using a high-resolution personal tabletop scanner. We created software to convert the tif files from the scanner to jpg format.
The final step of the process is to upload the generated HTML files and images to cloud storage and make them available on the web. We used AWS S3 for storage and CloudFront for content delivery. The upload process is automated using a script that syncs the generated local files with the S3 bucket and then clears the CloudFront cache.
| Cloud Storage (images)... | AWS S3 |
| Cloud Storage (website)... | AWS S3->Cloud Front |
| SQL DB Hosting... | Amazon Aurora/RDS |
| Version Control... | GitHub |
| Code generation "AI"... | Claude Code (~v4.6) |
| Languages used... | C#, HTML, JavaScript, (unknown AirTable script) |
All the code for the database migration and HTML generation is available on GitHub: Hogan's Archive System This project includes all the technical documentation and source code for the project. Nothing is hidden or proprietary.