ADM 8 Database Project Update: Major Changes Coming

I’m glad to say that since the last time I’ve blogged about this project, I’ve decided to throw most of the plans away- or at least the intended implementation has been shoved aside as I’ve decided to make larger changes.

Larry Hartzell and I have been moving our focus from the end of Volume 1 (1687/1688), where we had been making very smoother progress, to the earlier years, where there were many more complicated things. One of the issues – relating to the very beginning of the project- is that because of the way I designed things, we could not entirely reflect the accuracy of the way things were written in the reports- particularly, where there were reports with more than one set of columns on the same page, or where different sections of reports had different numbers of columns or different column labels. Here are a few examples.

ADM 8/1 f30v & 31r.

So here’s one example- on the  verso page you see that there is a distinct difference between the main body of report- which has the columns were’ accustomed to- and the “Ships in Ordinary” section.

ADM 8/1 f 35 verso and 36 recto

Here is another example of differences within one report- a structure similar to the recto page from the first example and then the “regular” columns to follow.

ADM 8/1 f41 verso and 42 recto

And here’s yet another example.

The problem with the original/current structure of the reports- that the flags for what columns were used were set at the report level, so that the entirety of a report had those columns visible whether they were used or not. Similarly, there was no mechanism for accommodating multiple sets of columns such as the “Ships in Ordinary” sections. LIkewise, even though we recorded the label titles for each independent report- these did not accommodate multiple sections with different titles- and they were only displayed in a list in the report information- they were not actually in the table columns with the data.

A few weeks ago, I finally made the decision (for mental health reasons) to just make the important changes- so that the project can better represent the documents in our tables. The major change so far is that I’ve added another layer to the report construction/deconstruction in the database- a “Section” layer. I’m moving all the flags for what columns are used- and for the column labels to these section rows- and a report can be composed of multiple sections. (most reports will be- I’ve decided to make Abstracts sections as well so they can be displayed interspersed with other data- which will improve the Fleet List reports (which need to be fixed). 

These past two weeks, I’ve been busy writing the code that will allow Larry and I (mostly Larry, if I’m honest. He’s amazingly wonderful and if he’s not addressed as the Amazing and Wonderful Larry Hartzell  then somebody’s missing something) to convert all the existing reports that are live- or transcribed, and had been ready to go live- to the news structure. 

Thankfully, there’s not really *that* many changes. I created two new tables- a much smaller one for reports, and a new one for Sections (At the moment, just for deployment reports- I’ve leaving the other report types for later).  The report lines, subtitles and abstracts tables had to have two, two and one new columns added respectively- “Section key” and “Column number” for the first two, and just “section line number” for the abstracts (so that the latter can be slotted into the right order within the sections). Of course, the existing line numbers will remain intact- it doesn’t matter if the first row in a section isn’t line 0, of course). A good side thing is that since I’ve been writing lots of new code, I’ve been also improving the code. For example, functions that were doing too many things have been simplified in the new version- and I’ve also been separating the “functions” into three different files (from a single nearly 8000 line long file- much too long). This was one of the improvements I had discussed in the previous update this spring.

I’m currently working on the code that will display the reports in the new much more variable and flexible structure- that should be done in the next week or so however it may not be made live until we’ve converted all of the existing reports- that way we don’t have two different methodologies which can get confused (or frustrated the researcher).

The other major change (At a friend’s suggestion) – not implemented yet- is that when researchers pull up a report, or call for the report lines associated with a single officer, or ship, or location, that the final request will use $_GET instead of $_POST. This means that the information passed through a link can be found in the URL- and will mean that those individual reports can be bookmarked/saved, so that researchers don’t have to go through all the steps again to find the previous results (although results may change because the table/results wouldn’t be saved- just the query. So results will change if more reports are added, for the non-single-report queries).

There are still other coding things that I will be working on- like I really want to improve the maps- but they will be coming later after I’ve finished this coding agenda. I look forward to the next update in a few months when hopefully we’ll have some more interesting details.