Admin Notes

The idea here is to provide a lot of specific detail on how the website is set up, how the software works, and any other information that one needs to know to keep it working and/or to set up future polls. Work on this was cut short of time. It probably needs to be redesigned from scratch (perhaps as part of a complete overhaul of the website).

Setting Up the Website

The top-level website directory is, as it has been since 2014, at https://hullworks.net/jazzpoll/. At some point it should be moved under its own domain name. Underneath it are directories with names like 24/, for the 2024 year-end poll, and 24my/ for the 2024 mid-year poll. So the first thing to do is to create a new 25my directory. Then, add a link from the top-level index page to the new directory, and add an entry in the Makefile to make an archive of the new directory. There is one such entry for each previous poll, so copy one and edit the year. The website is designed to be updated year-by-year -- rather than as a whole -- so there is also a top-level make entry.

After that, the easiest way to get started is to copy all of the files from 24my into 25my. I would normally use the previous year's files, as they were more recent than the previous mid-year's files, but I figured I'd want to keep more of the specific mid-year edits than the year-end ones. The one exception here was the critics file. I wanted to use the latest critics list, so I copied it over the one from 24my/.

After that, a bunch of changes need to be made. There's an old text file called HOWTO that explains the necessary year-to-year changes, so I started by consulting it. But I also figure I need to carefully re-read everything, and make extra edits for clarity. This is a record of my edits (superseding HOWTO, which I will also edit with an eye to future years).

Here's a detailed list of file edits. Anything marked TBD needs to be checked and edited as necessary.

• Data Files: albums, ballots-in, critics, footnotes, log-ballots, oldvotes: These need to be reset to start a new poll: zero these files out, except for a couple cases where we want to pick up previous year's information when needed:

  • critics: Replace mid-year file with end-year file, as it's more recent, and we want to preserve critic numbers/info from poll to poll; then comment out all lines by putting a # at the start of the line. When a critic votes, you can uncomment the line and edit it as needed.
  • log-ballots: Empty the file, then copy critics into it. When a critic votes, you can uncomment the line and add a cut-and-paste of the ballot, formatting with line tags per spec.

• Temporary Files: These files can be deleted or ignored, as they will be regenerated when make is run: albums.out, albums.sort, ballots-*.php, b.tmp, critics.count, critics.menu, critics.out, critics.urls, menu.inc, totals-*.inc

• Unnecessary Files: These are files that existed in the source directory, but are not needed here. It is best to delete them now, and refer to the original directory if necessary. There is no need to publicly list these files, as they won't pose future editing problems. Beware though that similar files reside in most previous poll directories.

  Note that while the Mid-Year Poll does not track Debut, Latin, or Vocal, and links to the totals files for those categories (totals-debut.php, totals-latin.php, totals-vocal.php) have been deleted from the index.php page and from the drop-down menus, the files have not yet been deleted. The shell script mkball.sh still generates the intermediate files, even if they are empty and don't get picked up by the totals and ballots code.

• Helper Files:

  • Makefile: Used by make for targets: all (default, invokes mkball.sh and mkcrit.sh to rebuild everything); clean; clean-temps. No edits, but clean and clean-temps are not routinely used.
  • excludes: Used by make to exclude some files from website update. No edits.
  • t: Template for adding entry to ballots-in: Edited for 5 Rara Avis slots instead of 3.

• admin-notes.php: One of several -notes files, each for a specific audience. The hope is that they'll provide comprehensive documentation of the various pieces that make up each poll.

  • Changed title from "Notes for Administrators" to "Admin Notes."
  • Commented out old questions list, as new structure will be topical.
  • Wrote new introduction.
  • Added this section on "Setting Up the Website."
  • Obvious a lot more can be done here.

• album.php: TBD.

  • Changed "New Releases" to "New Jazz Albums."

• alist.php: TBD.

  • Changed default "Albums List" to "Albums Lists."
  • Changed "Albums List:" to "List of" for non-default cases.
  • Changed "New Releases" to "New Jazz Albums."

• critics.php: Basically just an index of who has voted, each with a credentials line. Added an introductory paragraph.

• diffd.sh: Shell script, written new for this poll. It takes an argument pointing to another directory, then runs diff on all of the files in both this and the other directory. I wrote it to figure out what I had already changed before it occurred to me to write this section.

• faq.php: File is currently nearly empty, so much to be done here. TBD.

  • Need tag #points to explain point scheme.

• HOWTO: Old cribsheet for edits needed to set up a new poll directory. Edits:

  • std.inc: added note to update timelock.

• index.php:

  • Changed head/title to "Francis Davis Jazz Critics Poll: Mid-Year 2025."
  • Minimal edit to "Temporary notes" needed for deadline date. More extensive edits to better explain state of the poll.
  • Update dates for main body.
  • TBD.
• invite.php: Voter invitation file, edited to relect dates and changes in rules. This is nominally the "long-form" of the invite, same as was sent out on June 13 to the jazzpoll email list. The email version has minimal links, but the website version could be edited to provide links to more detailed explanations of rules, etc., as well as files referenced, so TBD. In the future, I'd like to make this the reference version, and use the alternative "short version" for mailings.

• invite2.php: This is the "short-form" invitation, simplified for late email invitations. TBD (still needs to be edited to reflect changes in "long-form" invite).

• std-ballots.inc:

  • Changed "NEW RELEASES" (24my) or "NEW ALBUMS" (24) to "NEW JAZZ ALBUMS."

• totals-new.php:

  • Added temp-note about points, FAQ links [TBD].
  • Intro rewritten to explain voting rules, including distinction from Rara Avis. Includes links to faq.php, which are currently dangling.
  • Footnotes section emptied. Need to add new footnotes. (Need to research this, as there is a data file involved.)
  • Notes section cleaned out. A section on 2024 releases, with combined votes for 2024-2025(mid-year) added. What else? Carryover votes do not apply to mid-year poll.

• totals-old.php: TBD.

  • Intro needs to be changed to be more consistent with totals-new.php.
  • Footnotes section, including heading, commented out. I need to review how footnotes are generated. (Used to be jotted down, but now there is a data file involved.)
  • Notes section (but not heading) commented out. The reference to "oldvotes" is not applicable, but we may still wish to note albums which received votes in 2024.

Voter List

By far, the hardest part of administering the poll is voter list management: qualifying voters, managing mailing lists, sending out invites and notices, trying to figure out who to nag and/or accept late votes from. Actually counting the ballots and compiling the results is relatively easy (almost a pleasure). Still, a big part of the gratification that comes with the poll is interacting with the voters, who I've developed immense respect and appreciation for.

I manage the voter list in a non-public file voter-log. This maps onto an email list managed by my local Thunderbird mailer, and onto an email list on my server, jazzpoll@hullworks.net. (The server has a second email list for administrators/helpers: jpadmin@hullworks.net. (There is also a spreadsheet that Paul Medrano set up, which is useful for sharing information in the jpadmin list, but I'm restarting with voter-log. To update these lists, I did the following:

1. I started with the 2024 (not 2024my) file, and commented out all the entries. Each entry is a call to a PHP function, which explodes a text string and prints out a table entry. The string has fields [0 .. 7], which I slightly adjusted. The fields tell me: name and credits, whether voted in 2024 (or -N for number of years lapsed since last vote), mailing list(s), invite sent (and how)?, invite ack?, ballot received.
2. I looked at the 2024 critics and uncommented and marked everyone on the list who had voted in 2024. Anyone else will have their lapsed times incremented.
3. I looked at the jazzpoll membership list, and noted it for everyone who was subscribed. I also noted several subscribers who had died, and unsubscribed them. (Most notably Francis Davis, but there were some who had died in previous years.)
4. I looked at the jpadmin membership list, and noted it for everyone who was subscribed. This wasn't necessary for the task at hand, but is useful information. (This includes a few people who have never voted but are interested parties or at least personal gurus.)
5. I went through my mail to flag voters who acknowledged receipt, and noted whether they intended to vote.
6. I checked my mail from the previous jazzpoll mailing to identify bounces. I dropped those names from the list (two were RIP; I tried contacting Marcela Breton via JJA message).
7. I looked at my Thunderbird Jazzpoll email list, noting names and collecting info for names not in server Jazzpoll list. This yielded an additional 84 critics names, of which 34 had voted in 2024 (and should be added to the Jazzpoll list, as they had not yet received invitations).
8. In retrospect, I erred in starting with the 2024 year-end voter-log file. It had been greatly expanded, especially with the addition of international critics. I then had to reconcile the files. I divided the additions into three groups: 2024 voters; people who had been invited in 2024 but who didn't vote; and other names, briefly considered but uninvited. The first two should get personalized invites ASAP.
9. Once the Thunderbird mailing list has been updated, create a template file, with: To: field set to "{{DisplayName}} <{{PrimaryEmail}}>"; Subject: "Francis Davis Jazz Critics Poll: Mid-Year 2025 Invite"; first text line: "Dear {{DisplayName}},". Save the template. Click on MailMerge, and specify: Address Book: Jazzpoll; Delivery Mode: Save as Draft. Click on Send, then Start. MailMerge then created 287 messages in 38 seconds. They are now in the Thunderbird Drafts directory.

What is mkball.sh and how does it work?

It is a shell/awk script, which is run by make to create the ballots and totals files. The documentation below was written to help understand where a bug is coming from, but it will be helpful if the program has to later be modified (or further debugged).

Its input files are:

• critics: one line per critic, with vbar-separated fields: [0] critic id; [1] name; [2] affiliations. The file is read at BEGIN, and stored in C1[], keyed by id, with the other information formatted for the PHP critic() function call in the ballot pages.
• albums: one line per album, with vbar-separated fields: [0] album id; [1] artist name; [2] title; [3] label; [4] unused?; [5] artist/album sort key. The album data is stored in three arrays, each keyed by album id: A1[] formats { id, artist, album, label } for PHP; A2[] formats vbar-separated artist|album|label; As[] saves sort key. Additional arrays for each album's votes are initialized to zero. The number of albums is counted and stored as acnt.
• oldvotes: This is used for carryover votes from last year. The votes are added into the vote counters before we process the ballots-in table. This file is empty at present, but it could be used to award extra points to R albums that received votes as M.
• ballots-in: The votes file, fields vbar-separated, where the first field is the record type: C for the critic with the following votes; M or M= for main (new releases), followed by up to 10 album ids (= for unranked lists); R or R= for rara avis, followed by up to 5 album ids (= for unranked lists); N for a text note to be added to the end of the ballot. The main loop of the awk script parses these lines and writes output to b.tmp.
• footnotes: Defines text to be used for inline notes in totals files. This is not used by mkball.sh, but is picked up in the various totals-*.php files, which call the PHP function init_acs($k) to read in the file, then check_acs($id) to see whether an album id has a note associated with it. The file consists of lines, each vbar-separated for fields: [0] key, any combination of [MRVDL]; [1] album id; [2] comment.

Its output files are:

• totals-* where * is { old, new, latin, vocal, debut }. These are intermediate files, one line per album, vbar-separated fields: [0] total points; [1] votal votes; [2] album id; [3] artist name; [4] title; [5] label; [6] semicolon-separated vote tuples, like A:B, where A is critic id, and B is points (or ? if unranked); [7] artist/album sort key.
• totals-*.inc, where * is as above. This is the intermediate file, sorted and reformatted to be included as PHP code. Each line is a call to function p(), with arguments: [0] album id; [1] total points; [2] total votes; [3] artist name; [4] title; [5] label.
• b.tmp: This is an intermediate file with all of the ballots, before they are cut into chunks of 20 ballots each.
• ballots-%.php, where % is a zero-filled two-digit number for a page of ballots. These pages are presentable, or can be reduced to a single ballot by calling them with a selector.

The main awk loop reads lines from ballots-in. Each non-blank, non-comment line begins with a record type code { C, M, M=, R, R=, N }. (The code also allows for V, D, and L, but these are unused in the Mid-Year Poll.) The code switches on record type:

• C: Identifies the critic subsequent lines are applied to. Each critic starts a new ballot. The ballots are counted for chunking in the ballots-%.php files, with extra PHP code generated by the bpage() and epage() functions. That code includes markers for cutting b.tmp into multiple ballot pages, as well as boilerplate code for the top and bottom of each page. The current critic id is saved as ccr, which can be used to look up stored critic info in C1[]. The variable cc is set = 1 when we have a current critic, and cleared by a C record with no critic id (does such a thing ever happen?).
• M or M=: Ranked or unranked votes, which differ only in the way points are allocated. The rest of the fields are album ids, which are processed in order. For each vote: let i be the position of the vote { 1 . . 10 }; set n to the album id $(i+1); increment the album vote total Mn[n]; add points to the point total Mt[h] (for ranked lists, points = Mtv[i]; for unranked lists, points = 1); add a code string to the vote records M[n]: this code consists of the album id, a colon :, and the points (the formatting is %s:%.1f; to force ranked points to include the .0; unranked points don't have the decimal point.
• R or R=: Ranked or unranked votes for rara avis. Same as with M, except that the variables are now Rn[n], Rt[n], and R[n], and the ranked votes come from Rtv[i]. Also, the vote code stored in R[n] has a final letter r, so we can tell that this is a Rara Avis vote.
• N: This immediately prints out a line with a note, under the current ballot.

The code also allows for { V, D, L } cases, but these are unused. It also allows for record types M! and R! for cases where there is no vote in a category. (Are these necessary?)

When the awk script finishes turning ballots-in into b.tmp, the END code writes out the various totals files.

After the initial awk script has created the file b.tmp,