Update pledge page template to show elected candidates.

.gitignore

@@ -0,0 +1 @@
+*.swp

Readme.md

@@ -2,6 +2,30 @@
 
 SPL tools is a collection of tools to assist in building the Streets People Love website.
 
+## csv-normaliser
+
+A PHP script for converting the raw .csv files received from volunteers into a .csv file that can be used by the php-template tool. Ensures that ward names are capitalised correctly, and identifies which picture to use for each candidate.
+
+## map-generator
+
+A webpage built using Mapbox for showing the boundaries of the wards in a particular council. Also contains some JS scripts for generating a .jpg image of each council's map.
+
+## php-template
+
+A PHP script for generating the HTML to use in each council page on the Streets People Love website. Handles reading each of the relevant configuration files and exposing them in a format that can easily be consumed in a regular PHP template. This is used by the make-council-page.sh script.
+
+## VEC Data
+
+This folder contains the raw data downloaded from the VEC Map website.
+
+## bulk-upload-media.sh
+
+A script for uploading all images in the spl-data folder to WordPress. Uses the `upload-media.sh` script to handle uploading each image.
+
+## copy-config.sh
+
+A script for splitting up `council_names.json` into separate `config.json` files for each council. Shouldn't need to be used again.
+
 ## council_names.json
 
 Contains the name of each council, a "short" name, and the names of each ward in the council.
@@ -11,7 +35,15 @@ The "short" named is created by taking the electorate name and removing the word
 The file can be generated using the `jq` tool and the VEC data:
 
 ```
-jq '[.[] | {name: .electorateName, electorateId: .electorateId, shortName: .parentElectorateName | match("(.*?)(?:(?: Rural)?(?: City| Shire) Council)").captures[0].string, parentElectorateId: .parentElectorateId, councilName: .parentElectorateName }] | group_by(.parentElectorateId) | map({shortName: .[0].shortName, slug: .[0].shortName | ascii_downcase | split(" ") | join("-"), councilName: .[0].councilName, wardNames: . | map(.name) }) | sort_by(.shortName)' "VEC Data\wards.json" > council_names.json
+jq '[.[] | {name: .electorateName, electorateId: .electorateId, shortName: .parentElectorateName | match("(.*?)(?:(?: Rural)?(?: City| Shire| Borough) Council)").captures[0].string, parentElectorateId: .parentElectorateId, councilName: .parentElectorateName }] | group_by(.parentElectorateId) | map({shortName: .[0].shortName, slug: .[0].shortName | ascii_downcase | split(" ") | join("-"), councilName: .[0].councilName, wardNames: . | map(.name) }) | sort_by(.shortName)' "VEC Data\wards.json" > council_names.json
+```
+
+## lga-links-filter
+
+This is a jq filter that will output a HTML anchor tag for each council. Can be used like this:
+
+```
+jq -f -r .\lga-links-filter .\council_names.json
 ```
 
 ## make-council-pages.sh
@@ -21,3 +53,8 @@ This is a bash script for creating a page in WordPress for each council.
 If a page for a council already exists, the page will be updated instead. The source of councils for this script is the "council_names.json" file. 
 
 The script needs the [`jq`](https://jqlang.github.io/jq/), [`php`](https://www.php.net/) and [`wp`](https://wp-cli.org/) tools.
+
+## upload-media.sh
+
+Tries to upload a file to WordPress and stores the media ID and URL in a .json file next to the file. If the .json file is already present, the upload will be skipped.
+

Updating SPL Website.md

@@ -0,0 +1,16 @@
+# Updating SPL Website
+
+1) Download any candidate pictures provided by volunteer to the appropriate council folder in the `spl-data` repo
+1) Ensure that the file name of each picture contains the candidate's full name (eg. for a candidate named `Joe Bloggs`, the file could be named `JoeBloggs.jpg` or `WardName_Joe_Bloggs.png`, etc)
+1) Resize the pictures to 400px x 400px
+1) Commit the pictures to the `spl-data` repo
+1) Download CSV provided by a volunteer containing the candidate scores
+1) Normalise the CSV using the "CSV Normaliser" tool (eg. `php csv-normaliser/main.php --input ~/Downloads/COUNCIL_NAME.csv --media ../spl-data/council-name --output ../spl-data/council-name/candidates.csv`)
+1) Commit the candidates.csv file to the `spl-data` repo
+1) Push commits in the `spl-data` repo to the Git server
+1) SSH into the Streets People Love WordPress server
+1) Pull the latest changes into the `spl-data` repo
+1) Run the `bulk-media-upload.sh` script (eg `sudo ./bulk-media-upload.sh ../spl-data`)
+1) Commit any added media .json files in the `spl-data` repo
+1) Push commits in the `spl-data` repo to the Git server (use the `spl` git user)
+1) Run the `make-council-pages.sh` script to regenerate all pages (eg `sudo ./make-council-pages.sh`) OR regenerate a single page (eg `sudo ./make-council-pages.sh "Council Name"`)

council_names.json

@@ -786,6 +786,14 @@
       "Beaufort"
     ]
   },
+  {
+    "shortName": "Queenscliffe",
+    "slug": "queenscliffe",
+    "councilName": "Queenscliffe Borough Council",
+    "wardNames": [
+      "Unsubdivided"
+    ]
+  },
   {
     "shortName": "South Gippsland",
     "slug": "south-gippsland",

csv-generic/gen-generic.php

@@ -0,0 +1,144 @@
+<?php
+
+require_once("parse_generic_csv.php");
+
+$options = getopt("", ["generic-csv:", "config-files:"]);
+
+if (isset($options['generic-csv'])) {
+    $generic_csv = $options['generic-csv'];
+} else {
+    error_log("Error: Missing required option '--generic-csv'.");
+    exit(1);
+}
+
+if (isset($options['config-files'])) {
+    $config_files = $options['config-files'];
+} else {
+    error_log("Error: Missing required option '--config-files'.");
+    exit(1);
+}
+
+$config_files = explode(" ", $config_files);
+$candidate_data = parse_generic_csv($generic_csv);
+
+$lga_list = [];
+/* Generate dictionary of LGAs and Wards */
+foreach ($config_files as $config_file) {
+    $config_string = file_get_contents($config_file);
+    if ($config_string !== FALSE) {
+        $config = json_decode($config_string, true);
+    } else {
+        error_log("Error opening config.json.");
+        exit(1);
+    }
+    $config['config-file'] = $config_file;
+    $lga_list[] = $config;
+}
+
+/* Match user typed LGA/Ward to our database */
+match_lga($candidate_data, $lga_list);
+
+/* Calculate score for candidate */
+foreach ($candidate_data as $key => $candidate) {
+    $score = 0;
+
+    if ($candidate['Pledge'] === "y") $score++;
+    if ($candidate['q1'] === "Yes") $score++;
+    if ($candidate['q3'] === "Yes") $score++;
+    if ($candidate['q4'] === "Yes") $score++;
+    if ($candidate['q7'] === "Yes") $score++;
+
+    $candidate_data[$key]['Score'] = $score;
+}
+
+$header = ["Ward", "Candidate Name", "Rating", "Pledge", "Picture"];
+
+/* Generate candidates-generic.csv */
+foreach ($lga_list as $lga) {
+    $lga_candidates = array_filter($candidate_data, function ($candidate) use ($lga) {
+        return $candidate['match_lga'] === $lga['slug'];
+    });
+
+    if (count($lga_candidates) === 0) continue;
+
+    remove_duplicates($lga_candidates);
+
+    $dir = dirname($lga['config-file']);
+    $dir_files = scandir($dir);
+    $output_file = $dir."/candidates-generic.csv";
+    $override_file = $dir."/candidates-override.csv";
+
+    if (($handle = fopen($output_file, "w")) === FALSE) {
+        error_log('Error opening output file');
+        exit(1);
+    }
+
+    if (fputcsv($handle, $header) === FALSE) {
+        error_log('Error writing headers to output file');
+        exit(3);
+    }
+
+    $lines = [];
+    foreach ($lga_candidates as $candidate) {
+        /* Add extension to photo hash */
+        if (strlen($candidate['Photo'])) {
+            foreach ($dir_files as $file) {
+                if (preg_match("/\.json$/", $file)) continue;
+                if (strstr($file, $candidate['Photo'])) {
+                    $candidate['Photo'] = $file;
+                }
+            }
+        }
+
+        $lines[] = [
+            $candidate['match_ward'],
+            $candidate['Name'],
+            $candidate['Score'],
+            $candidate['Pledge'],
+            $candidate['Photo'],
+        ];
+    }
+
+    /* Apply overrides if they exist */
+    $overrides = [];
+    if (file_exists($override_file)) {
+        if (($ovr_handle = fopen($override_file, "r")) !== FALSE) {
+            $headers = fgetcsv($ovr_handle);
+            while (($data = fgetcsv($ovr_handle)) !== FALSE) {
+                $override = [];
+                foreach ($headers as $key => $value) {
+                    $override[$value] = $data[$key];
+                }
+                $overrides[] = $override;
+            }
+            fclose($ovr_handle);
+        } else {
+            error_log('Error opening overrides file');
+            exit(3);
+        }
+    }
+
+    foreach ($overrides as $override) {
+        foreach ($lines as $line_key => $line) {
+            $match_index = array_search($override['Match Field'], $header); 
+            $replace_index = array_search($override['Replace Field'], $header); 
+            if ($line[$match_index] === $override['Match Value']) {
+                if ($replace_index !== false)
+                    $lines[$line_key][$replace_index] = $override['Replace Value'];
+                else /* If 'Replace Field' is not matched - delete this entry */
+                    $lines[$line_key]['Delete'] = 'y';
+            }
+        }
+    }
+
+    foreach ($lines as $line) {
+        if (isset($line['Delete'])) continue;
+        if (fputcsv($handle, $line) === FALSE) {
+            error_log('Error writing candidate to output file');
+            exit(3);
+        }
+    }
+    fclose($handle);
+}
+
+exit(0);

csv-generic/gen-image-map.php

@@ -0,0 +1,54 @@
+<?php
+
+require_once("parse_generic_csv.php");
+
+$options = getopt("", ["generic-csv:", "config-files:"]);
+
+if (isset($options['generic-csv'])) {
+    $generic_csv = $options['generic-csv'];
+} else {
+    error_log("Error: Missing required option '--generic-csv'.");
+    exit(1);
+}
+
+if (isset($options['config-files'])) {
+    $config_files = $options['config-files'];
+} else {
+    error_log("Error: Missing required option '--config-files'.");
+    exit(1);
+}
+
+$config_files = explode(" ", $config_files);
+$candidate_data = parse_generic_csv($generic_csv);
+
+$lga_list = [];
+/* Generate dictionary of LGAs and Wards */
+foreach ($config_files as $config_file) {
+    $config_string = file_get_contents($config_file);
+    if ($config_string !== FALSE) {
+        $config = json_decode($config_string, true);
+    } else {
+        error_log("Error opening config.json.");
+        exit(1);
+    }
+    $config['config-file'] = $config_file;
+    $lga_list[] = $config;
+}
+
+/* Match user typed LGA/Ward to our database */
+match_lga($candidate_data, $lga_list);
+
+$image_map = [];
+foreach ($candidate_data as $candidate) {
+    if (strlen($candidate['photo_url'])) {
+        $map['url'] = $candidate['photo_url'];
+        $map['match_lga'] = $candidate['match_lga'];
+        $image_map[$candidate['Photo']] = $map;
+    }
+}
+
+$json_data = json_encode($image_map);
+
+print_r($json_data);
+
+exit(0);

csv-generic/parse_generic_csv.php

@@ -0,0 +1,107 @@
+<?php
+
+function parse_generic_csv($generic_csv) {
+    $candidate_data = [];
+    if (($handle = fopen($generic_csv, "r")) !== FALSE) {
+        $headers = fgetcsv($handle);
+        while (($data = fgetcsv($handle)) !== FALSE) {
+            $candidate = [];
+            $question_no = 0;
+            $is_question = false;
+            foreach ($headers as $key => $value) {
+                /* Override key name for questions */
+                if ($value === "Verified") {
+                    $is_question = false;
+                }
+
+                if (strstr($value, "candidate photo"))                  $value = "Photo";
+                if (strstr($value, "In which Local Government Area"))   $value = "LGA";
+                if (strstr($value, "In which Ward"))                    $value = "Ward";
+                if (strstr($value, "Political Party"))                  $value = "Party";
+
+                if ($value === "Pledge") {
+                    if (strstr($data[$key], "I pledge")) $data[$key] = "y";
+                    else $data[$key] = "n";
+                }
+                
+                if ($value === "Photo") {
+                    $candidate['photo_url'] = $data[$key];
+                    $data[$key] = preg_filter("/.*id=/", "", $data[$key]);
+                }
+
+                if ($is_question) {
+                    $candidate['q'.$question_no++] = $data[$key];
+                } else {
+                    $candidate[$value] = $data[$key];
+                }
+
+                if ($value === "Pledge") {
+                    $is_question = true;
+                }
+            }
+            $candidate_data[] = $candidate;
+        }
+        fclose($handle);
+    } else {
+        error_log('Error opening candidates file');
+        exit(1);
+    }
+
+    return $candidate_data;
+}
+
+function match_lga(&$candidate_data, $lga_list) {
+    foreach ($candidate_data as &$candidate) {
+        /* Match user typed LGA/Ward to our database */
+        $max_score = 0;
+        foreach ($lga_list as $lga) {
+            $aa = preg_split("/[^a-z]/", strtolower($candidate['LGA']));
+            $bb = preg_split("/[^a-z]/", $lga['slug']);
+
+            $score_sum = 0;
+            foreach ($aa as $a) {
+                foreach ($bb as $b) {
+                    similar_text($a, $b, $score);
+                    if ($score > 70) $score_sum += $score;
+                    else $score_sum -= 1;
+                }
+            }
+
+            if ($score_sum > $max_score) {
+                $max_score = $score_sum;
+                $match_lga = $lga;
+            }
+        }
+
+        $max_score = 0;
+        foreach ($match_lga['wardNames'] as $ward) {
+            similar_text(strtolower($ward), strtolower($candidate['Ward']), $score);
+            if ($score >= $max_score) {
+                $max_score = $score;
+                $match_ward = $ward;
+            }
+        }
+
+        $candidate['match_lga'] = $match_lga['slug'];
+        $candidate['match_ward'] = $match_ward;
+    }
+}
+
+function remove_duplicates(&$candidate_data) {
+    $names = [];
+    $duplicates = [];
+    foreach ($candidate_data as $candidate_key => $candidate) {
+        /* If we've already had this name, remove the old entry */
+        foreach ($names as $name_key => $name) {
+            similar_text(strtolower($name), strtolower($candidate['Name']), $score);
+            if ($score > 90) {
+                $duplicates[] = $name_key;
+            }
+        }
+        $names[$candidate_key] = $candidate['Name'];
+    }
+    $duplicates = array_unique($duplicates);
+    foreach ($duplicates as $duplicate) {
+        unset($candidate_data[$duplicate]);
+    }
+}

csv-normaliser/main.php

@@ -1,5 +1,31 @@
 <?php
-$options = getopt("", ["input:", "output:", "media:"]);
+$options = getopt("", ["folder:", "input:", "output:", "media:"]);
+
+if (isset($options['folder'])) {
+    $folder = $options['folder'];
+    if (is_dir($folder)) {
+        $expectedInputFileNames = [];
+        $expectedInputFileNames[] = str_replace("-", " ", strtoupper(basename($folder))) . ".csv";
+        $expectedInputFileNames[] = strtoupper(basename($folder)) . ".csv";
+        if (!isset($options['input'])) {
+            foreach ($expectedInputFileNames as $expectedInputFileName) {
+                $expectedInputFile = $folder . DIRECTORY_SEPARATOR  . $expectedInputFileName;
+                if (is_file($expectedInputFile)) {
+                    $options['input'] = $expectedInputFile;
+                }
+            }
+        }
+        if (!isset($options['output'])) {
+            $options['output'] = $folder . DIRECTORY_SEPARATOR . "candidates.csv";
+        }
+        if (!isset($options['media'])) {
+            $options['media'] = $folder;
+        }
+    } else {
+        error_log("Error: Specified folder is not valid.");
+        exit(1);
+    }
+}
 
 if (isset($options['input'])) {
     $inputFile = $options['input'];
@@ -51,50 +77,91 @@ if (($handle = fopen($inputFile, "r")) !== FALSE) {
             if ($currentWard == "Coastal-promontory") {
                 $currentWard = "Coastal-Promontory";
             }
+            if ($currentWard == "Alpine Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Ararat Rural City") $currentWard = "Unsubdivided";
+            if ($currentWard == "Benalla Rural City") $currentWard = "Unsubdivided";
+            if ($currentWard == "Campaspe Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Colac Otway Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "East Gippsland Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Gannawarra Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Glenelg Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Golden Plains Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Hepburn Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Indigo Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Mansfield Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Melbourne City") $currentWard = "Unsubdivided";
+            if ($currentWard == "Moira Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Moorabool Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Moyne Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Queenscliffe Borough") $currentWard = "Unsubdivided";
+            if ($currentWard == "Southern Grampians Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Strathbogie Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "Swan Hill Rural City") $currentWard = "Unsubdivided";
+            if ($currentWard == "Towong Shire") $currentWard = "Unsubdivided";
+            if ($currentWard == "West Wimmera Shire") $currentWard = "Unsubdivided";
         }
-        if ($data[0] == "Candidate") {
+        if (trim($data[0]) == "Candidate" || trim($data[0]) == "") {
             if ($currentWard == null) {
                 error_log("No ward found, skipping data on line " . $currentLine);
                 continue;
             }
 
-            $candidateName = $data[1];
+            $candidateName = trim($data[1]);
 
-            if ($candidateName == " example name") {
+            if ($candidateName == "example name" || $candidateName == "") {
                 error_log("Skipping line ". $currentLine);
                 continue;
             }
 
-            print("Adding candidate " . $candidateName . " to ". $currentWard . "\n");
+            //print("Adding candidate " . $candidateName . " to ". $currentWard . "\n");
 
-            $name_split = explode(" ", $data[1]);
+            $name_split = array_values(array_filter(explode(" ", str_replace(",", "", str_replace("'", "_", $data[1]))), function($value) { return !is_null($value) && $value !== ''; }));
 
             $name_patterns = [
-                implode(".*", $name_split),
-                implode(".*", array_reverse($name_split)),
+                ".*" . implode(".*", $name_split) . ".*",
+                ".*" . implode(".*", array_reverse($name_split)) . ".*"
             ];
 
-            $regex_groups = array_map(function($x) { return "(?:.*" . $x . ".*)"; }, $name_patterns);
+            $first_name = $name_split[0];
+            $last_name = $name_split[array_key_last($name_split)];
+
+            if (strlen($first_name) > 3) {
+                $name_patterns[] = "^" . $first_name . ".*";
+            }
+            if ($last_name != $first_name && strlen($last_name)) {
+                $name_patterns[] = "^" . $last_name . ".*";
+            }
+
+            $regex_groups = array_map(function($x) { return "(?:" . $x . ")"; }, $name_patterns);
 
             $regex_pattern = "/" . implode("|", $regex_groups) . "/i";
 
             $picture = "";
             foreach ($mediaFiles as $mediaFile) {
+                if ($mediaFile == ".") continue;
+                if ($mediaFile == "..") continue;
                 if (preg_match($regex_pattern, $mediaFile)) {
                     $picture = $mediaFile;
                     break;
                 }
             }
             if ($picture === "") {
-                print("Failed to identify picture for " . $candidateName . "\n");
+                print("\033[31mFailed to identify picture for " . $candidateName . "\033[0m\n");
             }
 
+            $rating = $data[2];
+            if ($rating == "score" || $rating == "") {
+                // Don't include candidates who haven't been given a score
+                continue;
+            }
+            $rating = (int)$rating;
+
             array_push(
                 $candidates, 
                 [
                     "Ward" => $currentWard,
                     "Candidate Name" => $candidateName,
-                    "Rating" => $data[2],
+                    "Rating" => $rating,
                     "Picture" => $picture
                 ]
             );
@@ -139,6 +206,6 @@ if (($handle = fopen($outputFile, "w")) !== FALSE) {
     exit(1);
 }
 
-print("Data written to " . $outputFile);
+print("Data written to " . $outputFile . "\n");
 
 exit(0);

get-generic.sh

@@ -0,0 +1,62 @@
+#!/bin/bash
+
+#rclone sync --progress bikewest:spl_generic_survey_2024 $DATA_LOC/google-data
+
+GENERIC_SURVEY=../generic-survey/responses.csv
+IMAGES=../generic-survey/images
+
+DATA_PATH="../spl-data"
+
+echo "Fetching latest responses to generic survey."
+rm -f $GENERIC_SURVEY # Force re-fetch
+rclone -v copyto --drive-export-formats csv 'bikewest:spl_generic_survey_2024/Streets People Love council election candidate pledge and survey  (Responses).csv' $GENERIC_SURVEY
+
+config_files=()
+for folder in "$DATA_PATH"/*; do
+    if test -f "$folder"/config.json; then
+        config_files+=("$folder"/config.json)
+    fi
+done
+
+image_map=$(php csv-generic/gen-image-map.php --generic-csv $GENERIC_SURVEY --config-files "${config_files[*]}")
+
+img_list=()
+for key in $(jq -r 'keys[]' <<< $image_map) ; do
+    if [ -f $IMAGES/$key ] ; then
+        continue
+    fi
+    img_list+=($key)
+    img_list+=($IMAGES/$key)
+done
+
+if [ ${#img_list[*]} -gt 0 ] ; then
+    echo "Downloading $((${#img_list[*]}/2)) image(s)..."
+    rclone -v backend copyid bikewest: ${img_list[*]}
+fi
+
+for key in $(jq -r 'keys[]' <<< $image_map) ; do
+    format=$(identify $IMAGES/$key | awk '{print $2}')
+
+    case $format in
+        PNG )  suffix=.png ;;
+        JPEG ) suffix=.jpg ;;
+        HEIC ) suffix=.jpg ;;
+        WEBP ) suffix=.png ;;
+        *)
+            echo "Error: Unknown image format: $IMAGES/$key"
+            ;;
+    esac
+
+    lga=$(jq -r ".[\"$key\"][\"match_lga\"]" <<< $image_map)
+    dst="$DATA_PATH/$lga/$key$suffix"
+
+    if [ -f $dst ] ; then
+        continue
+    fi
+
+    echo "Resizing $dst"
+    convert $IMAGES/$key -resize 400x400 $dst
+done
+
+echo "Generating candidates-generic.csv files."
+php csv-generic/gen-generic.php --generic-csv $GENERIC_SURVEY --config-files "${config_files[*]}"

lga-page.php

@@ -0,0 +1,50 @@
+<?php
+
+$options = getopt("", ["candidates-files:", "lga-files:"]);
+
+if (isset($options['candidates-files'])) {
+    $candidates_files = explode(" ", $options['candidates-files']);
+} else {
+    error_log("Error: Missing required option '--candidates-files'.");
+    exit(1);
+}
+
+if (isset($options['lga-files'])) {
+    $lga_files = explode(" ", $options['lga-files']);
+} else {
+    error_log("Error: Missing required option '--lga-files'.");
+    exit(1);
+}
+
+$lgas_with_data = [];
+foreach ($candidates_files as $file) {
+    $lgas_with_data[] = basename(dirname($file));
+}
+
+print('<!-- wp:list --> <ul class="wp-block-list">' . "\n");
+
+foreach ($lga_files as $config_file) {
+    $config_string = file_get_contents($config_file);
+
+    if ($config_string !== FALSE) {
+        $config = json_decode($config_string, true);
+    } else {
+        error_log("Error opening config.json.");
+        exit(1);
+    }
+
+    if (array_search($config['slug'], $lgas_with_data) === false)
+        $font_weight = 300;
+    else
+        $font_weight = 700;
+
+    print('<!-- wp:list-item {"style":{"typography":{"fontStyle":"normal","fontWeight":"');
+    print($font_weight . '"}}} -->' . "\n");
+    print('<li style="font-style:normal;font-weight:' . $font_weight . '">');
+    print('<a href="' . $config['slug'] . '">' . $config['councilName'] . "</a></li>\n");
+    print('<!-- /wp:list-item -->' . "\n");
+}
+
+print('</ul> <!-- /wp:list -->');
+
+exit(0);

make-council-pages.sh

@@ -7,7 +7,7 @@
 DATA_PATH="../spl-data"
 
 # Controls the flags that are passed to every usage of the wp command.
-WP_FLAGS="--allow-root --path=/var/www/html"
+#WP_FLAGS="--allow-root --path=/var/www/html"
 
 function create_or_update_page() {
   local council_block="$1"
@@ -31,7 +31,20 @@ function create_or_update_page() {
 
   jq -n '[inputs | { (input_filename | sub("\\.json$"; "") | sub("^.+/"; "")): . }] | reduce .[] as $item ({}; . + $item)' "${media_inputs[@]}" > "$DATA_PATH"/$slug/media.json
 
-  content=$(echo "$council_block" | jq -c | php php-template/main.php --council-file "php://stdin" --candidates-file "$DATA_PATH"/$slug/candidates.csv --media-file "$DATA_PATH"/$slug/media.json )
+  # Community groups get priority
+  if test -f "$DATA_PATH"/$slug/candidates.csv; then
+    candidates_file="$DATA_PATH"/$slug/candidates.csv
+  else
+    candidates_file="$DATA_PATH"/$slug/candidates-generic.csv
+  fi
+
+  if test -f "$DATA_PATH"/$slug/candidates-elected.csv; then
+    candidates_elected_file="$DATA_PATH"/$slug/candidates-elected.csv
+    echo "Found candidates-elected.csv"
+  fi
+
+
+  content=$(echo "$council_block" | jq -c | php php-template/main.php --council-file "php://stdin" --candidates-file "$candidates_file" --media-file "$DATA_PATH"/$slug/media.json --candidates-elected-file "$candidates_elected_file" )
 
   if [ $? -eq 0 ]; then
 
@@ -40,7 +53,7 @@ function create_or_update_page() {
       echo "$content" | wp post update "$page_id" --post_content="$content" $WP_FLAGS -
     else
       echo "Create page $short_name"
-      echo "$content" | wp post create --post_type=page --post_title="$short_name" --post_status=publish $WP_FLAGS -
+      echo "$content" | wp post create --post_type=page --post_title="$short_name" --post_status=draft $WP_FLAGS -
     fi
 
   else

make-lga-page.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+# This script uses the jq, wp, and php commands, make sure they are installed before running this script.
+
+# The folder containing data for each council.
+# Includes the list of candidates and any media.
+DATA_PATH="../spl-data"
+
+# Iterate over folders in data path
+candidates_files=()
+for folder in "$DATA_PATH"/*; do
+    if test -f "$folder"/candidates-generic.csv; then
+        candidates_files+=("$folder"/candidates-generic.csv)
+    elif test -f "$folder"/candidates.csv; then
+        candidates_files+=("$folder"/candidates.csv)
+    fi
+    if test -f "$folder"/config.json; then
+        lga_files+=("$folder"/config.json)
+    fi
+done
+
+content=$(php lga-page.php  --candidates-files "${candidates_files[*]}" \
+                            --lga-files "${lga_files[*]}")
+
+echo "$content" | wp post update 465 -

make-pledge-page.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+# This script uses the jq, wp, and php commands, make sure they are installed before running this script.
+
+# The folder containing data for each council.
+# Includes the list of candidates and any media.
+DATA_PATH="../spl-data"
+DEFAULT_IMAGE="../spl-data/default.png.json"
+
+# Iterate over folders in data path
+candidates_files=()
+for folder in "$DATA_PATH"/*; do
+    if test -f "$folder"/candidates-generic.csv; then
+        candidates_files+=("$folder"/candidates-generic.csv)
+    fi
+    # Community groups get priority
+    if test -f "$folder"/candidates.csv; then
+        candidates_files+=("$folder"/candidates.csv)
+    fi
+done
+
+content=$(php pledge-update/pledge-page.php --candidates-files "${candidates_files[*]}" \
+                                            --default-image $DEFAULT_IMAGE)
+
+echo "$content" | wp post update 12106 -

map-generator/src.js

@@ -1,7 +1,7 @@
 import polylabel from 'polylabel';
 
 function normaliseCouncilName(str) {
-    const regex = /(.*?)(?:(?: Rural)?(?: City| Shire) Council)/g;
+    const regex = /(.*?)(?:(?: Rural)?(?: City| Shire| Borough) Council)/g;
     const matches = str.matchAll(regex);
 
     // If we get a match, convert to slug format

php-template/example-candidates-elected.csv

@@ -0,0 +1,2 @@
+Ward,Candidate Name,Elected
+Harvester,Joe Blogs,y

php-template/main.php

@@ -1,7 +1,7 @@
 <?php
 require_once "page_renderer.php";
 
-$options = getopt("", ["council-file:", "candidates-file:", "media-file:"]);
+$options = getopt("", ["council-file:", "candidates-file:", "media-file:", "candidates-elected-file:"]);
 
 if (isset($options['council-file'])) {
     $councilFileContents = file_get_contents($options['council-file']);
@@ -55,6 +55,37 @@ if (isset($options['media-file'])) {
 
 $mediaData = json_decode($mediaFileContents, true);
 
+// Merge elected data (if present) into candidate objects
+if (isset($options['candidates-elected-file'])) {
+    $candidatesElectedFile = $options['candidates-elected-file'];
+
+    if (file_exists($candidatesElectedFile)) {
+        if (($handle = fopen($candidatesElectedFile, "r")) !== FALSE) {
+            $headers = fgetcsv($handle);
+            while (($data = fgetcsv($handle)) !== FALSE) {
+                $electedCandidate = [];
+                foreach ($headers as $key => $value) {
+                    $electedCandidate[$value] = $data[$key];
+                }
+                foreach ($candidateData as &$candidate) {
+                    if ($candidate["Candidate Name"] == $electedCandidate["Candidate Name"]) {
+                        if ($electedCandidate["Elected"] == "y") {
+                            $candidate["Elected"] = True;
+                        }
+                        break;
+                    }
+                }
+            }
+            fclose($handle);
+        } else {
+            error_log('Error opening candidates elected file');
+            exit(1);
+        }
+    } else {
+        error_log("The specified candidates elected file does not exist, will not show any elected candidates for " . $councilData["shortName"] . ".");
+    }
+}
+
 $renderer = new SPLPageRenderer();
 $pageContent = $renderer->renderCouncilPage($councilData, $candidateData, $mediaData);
 if ($pageContent === null) {

php-template/template.php

@@ -4,14 +4,27 @@ function sluggify($input) {
     return strtolower(str_replace(' ', '-', $input));
 }
 
+$surveyLink = "<a href=\"https://forms.gle/gnDNyBiVC64tDo2Y7\">Streets People Love Pledge and Survey</a>";
+if (isset($config["survey"])) {
+    $surveyLink = $config["survey"];
+}
+
 ?>
 
 <!-- wp:paragraph -->
+<?php if (isset($config['header'])): ?>
+
+<p><?php echo $config['header']; ?></p>
+
+<?php else: ?>
+
 <p>The Streets People Love campaign has created scorecards for candidates in the 2024 council elections. Scorecards have been generated based on a candidate's engagement with the Streets People Love campaign, their commitment to our pledge, their responses to a survey and input from campaign members located in the local government area in which they are running.</p>
 <!-- /wp:paragraph -->
 
 <!-- wp:paragraph -->
-<p>Have candidates in your local government area not yet taken part? Send your local candidates the <a href="https://forms.gle/gnDNyBiVC64tDo2Y7">Streets People Love Pledge and Survey</a> and ask them to complete it so that local residents can vote for the candidates who want to build the streets people love.</p>
+<p>Can't see a candidate you know is running? Candidates who don't take our survey won't appear on this page. Feel free to send your local candidates the <?php echo $surveyLink; ?> and let them know it's important to local residents that they do take part, so that we can vote for those who want to build the streets people love.</p>
+
+<?php endif; ?>
 <!-- /wp:paragraph -->
 
 <?php if (isset($media["header.jpg"])): ?>
@@ -88,31 +101,42 @@ function sluggify($input) {
     <!-- /wp:paragraph -->
 <?php endif; ?>
 
-<?php foreach ($config['wardNames'] as $index => $wardName): ?>
+<?php 
+    if (isset($config['groupNames'])) {
+        $groupNames = $config['groupNames'];
+        $propertyOnCandidate = 'Group';
+    } else {
+        $groupNames = $config['wardNames'];
+        $propertyOnCandidate = 'Ward';
+    }
+?>
+
+<?php foreach ($groupNames as $index => $groupName): ?>
     <!-- wp:heading {"level":3,"className":"is-style-default"} -->
-    <?php $wardSlug = sluggify($wardName); ?>
-    <h3 class="wp-block-heading is-style-default" id="<?php echo $wardSlug; ?>"><a style="text-decoration: none;" href="#<?php echo $wardSlug; ?>"><?php echo $wardName; ?></a></h3>
+    <?php $groupSlug = sluggify($groupName); ?>
+    <h3 class="wp-block-heading is-style-default" id="<?php echo $groupSlug; ?>"><a style="text-decoration: none;" href="#<?php echo $groupSlug; ?>"><?php echo htmlspecialchars($groupName); ?></a></h3>
     <!-- /wp:heading -->
 
     <?php
-    $wardCandidates = array_filter($candidates, function ($candidate) use ($wardName) {
-        return isset($candidate["Ward"]) && $candidate["Ward"] === $wardName;
+    $groupCandidates = array_filter($candidates, function ($candidate) use ($groupName, $propertyOnCandidate) {
+        return isset($candidate[$propertyOnCandidate]) && 
+            strtolower($candidate[$propertyOnCandidate]) === strtolower($groupName);
     });
 
-    usort($wardCandidates, function($a, $b) {
+    usort($groupCandidates, function($a, $b) {
         if ($a == $b) {
             return 0;
         }
-        return (((int) $a) < ((int) $b)) ? 1 : -1;
+        return (((int) $a['Rating']) < ((int) $b['Rating'])) ? 1 : -1;
     });
 
-    if (count($wardCandidates) > 0):
+    if (count($groupCandidates) > 0):
     ?>
 
         <?php
         $columnCount = 4;
 
-        $chunkedWardCandidates = array_chunk($wardCandidates, $columnCount);
+        $chunkedWardCandidates = array_chunk($groupCandidates, $columnCount);
         ?>
 
         <?php foreach($chunkedWardCandidates as $chunk): ?>
@@ -127,6 +151,12 @@ function sluggify($input) {
                             <?php 
                                 $candidate = $chunk[$columnIdx];
 
+                                if (isset($candidate["Elected"]) && $candidate["Elected"]) {
+                                    $candidate_elected = true;
+                                } else {
+                                    $candidate_elected = false;
+                                }
+
                                 if (isset($candidate['Picture']) && isset($media[$candidate['Picture']])) {
                                     $candidate_image = $media[$candidate['Picture']];
                                 } else {
@@ -134,7 +164,6 @@ function sluggify($input) {
                                 }
 
                                 $candidate_rating = str_repeat("✔️", max(0, min(5, $candidate['Rating'])));
-                                $candidate_rating_colour = "green";
 
                                 // If string is 5 ticks, insert a zero width space entity between the 3rd and 4th ticks so that it wraps nicer
                                 if ($candidate_rating == "✔️✔️✔️✔️✔️") {
@@ -147,15 +176,19 @@ function sluggify($input) {
                             ?>
 
                             <!-- wp:image {"id":<?php echo $candidate_image['id']; ?>,"width":"200px","height":"200px","scale":"cover","align":"center","style":{"color":{}},"className":"is-resized"} -->
-                            <figure class="wp-block-image aligncenter is-resized"><img src="<?php echo $candidate_image['url']; ?>" alt="" class="wp-image-<?php echo $candidate_image['id']; ?>" style="object-fit:cover;width:200px;height:200px"/></figure>
+                            <figure class="wp-block-image aligncenter is-resized <?php if ($candidate_elected) { echo "elected-candidate"; } ?>"><img src="<?php echo $candidate_image['url']; ?>" alt="" class="wp-image-<?php echo $candidate_image['id']; ?>" style="object-fit:cover;width:200px;height:200px;"/>
+                                <?php if ($candidate_elected): ?>
+                                    <figcaption>ELECTED</figcaption>
+                                <?php endif; ?>
+                            </figure>
                             <!-- /wp:image -->
 
                             <!-- wp:heading {"textAlign":"center","className":"wp-block-heading has-text-align-center has-medium-font-size","style":{"spacing":{"margin":{"top":"1rem","bottom":"0.5rem"}}}} -->
-                            <h2 class="wp-block-heading has-text-align-center has-medium-font-size" style="margin-top:1rem;margin-bottom:0.5rem"><strong><?php echo $candidate['Candidate Name']; ?></strong></h2>
+                            <h2 class="wp-block-heading has-text-align-center has-medium-font-size" style="margin-top:1rem;margin-bottom:0.5rem"><strong><?php echo htmlspecialchars($candidate['Candidate Name']); ?></strong></h2>
                             <!-- /wp:heading -->
 
                             <!-- wp:paragraph {"align":"center","style":{"layout":{"selfStretch":"fit","flexSize":null},"typography":{"lineHeight":"1"},"spacing":{"margin":{"top":"0.5rem","bottom":"1.5rem"}}},"fontSize":"large"} -->
-                            <p class="has-text-align-center has-large-font-size" style="margin-top:0.5rem;margin-bottom:1.5rem;line-height:1;color: rgba(100%, 0%, 0%, 0);text-shadow: 0 0 0 <?php echo $candidate_rating_colour; ?>;"><?php echo $candidate_rating; ?></p>
+                            <p class="has-text-align-center has-large-font-size candidate-ticks"><?php echo $candidate_rating; ?></p>
                             <!-- /wp:paragraph -->
                         <?php endif; ?>
 
@@ -170,7 +203,7 @@ function sluggify($input) {
     <?php else: ?>
 
         <!-- wp:paragraph -->
-        <p>No candidates in this ward have completed the Streets People Love survey. Send your local candidates the <a href="https://forms.gle/gnDNyBiVC64tDo2Y7">Streets People Love Pledge and Survey</a> and ask them to take part so that local residents can vote for the candidates who want to build the streets people love.</p>
+        <p>No candidates in this ward have completed the survey. Send your local candidates the <?php echo $surveyLink; ?> and ask them to take part so that local residents can vote for the candidates who want to build the streets people love.</p>
         <!-- /wp:paragraph -->
 
     <?php endif; ?>

pledge-update/homepage.php

@@ -0,0 +1,42 @@
+<?php
+
+require_once("parse_pledge_data.php");
+
+$options = getopt("", ["candidates-files:"]);
+
+if (isset($options['candidates-files'])) {
+    $candidates_files = $options['candidates-files'];
+} else {
+    error_log("Error: Missing required option '--candidates-files'.");
+    exit(1);
+}
+
+$candidate_data = parse_pledge_data(explode(" ", $candidates_files), null);
+
+/* Select people who have taken the pledge (and have an image) */
+$pledgeCandidates = array_filter($candidate_data, function ($candidate) {
+    return $candidate['Pledge'] === 'y' && $candidate['Picture'] !== "";
+});
+
+/* Select 9 random candidates */
+$pledgeKeys = array_rand($pledgeCandidates, 9);
+shuffle($pledgeKeys);
+
+$i = 0;
+foreach ($pledgeKeys as $key) {
+    $image_url = $pledgeCandidates[$key]['image_url'];
+    $image_id = $pledgeCandidates[$key]['image_id'];
+
+    echo "s|pledge_img_".$i."|".$image_url."|\n";
+    echo "s|pledge_id_".$i."|".$image_id."|\n";
+
+    echo "s|pledge_string_".$i."|";
+    echo $pledgeCandidates[$key]['Candidate Name'].
+        " (".
+        $pledgeCandidates[$key]['Council'].
+        ") has taken the pledge!|\n";
+
+    $i++;
+}
+
+exit(0);

pledge-update/page_renderer.php

@@ -0,0 +1,28 @@
+<?php
+
+class SPLPageRenderer {
+    public function renderPledgePage($councils, $lga_pages, $candidates) {
+        ob_start();
+
+        $didError = false;
+
+        set_error_handler(function($errno, $errstr, $errfile, $errline) use(&$didError) {
+            $didError = true;
+            error_log("Error: $errstr in $errfile on line $errline");
+            return true; // Prevent default error handling
+        });
+
+        require "template.php";
+
+        restore_error_handler();
+
+        $content = ob_get_clean();
+
+        // Explictly return null if we didn't generate any content or if there was an error
+        if (!empty($content) && !$didError) {
+            return $content;
+        } else {
+            return null;
+        }
+    }
+}

pledge-update/parse_pledge_data.php

@@ -0,0 +1,110 @@
+<?php
+
+function trim_sluggify($input) {
+    return strtolower(str_replace(' ', '-', trim($input)));
+}
+
+function parse_pledge_data($candidates_files, $default_image) {
+    $candidate_data = [];
+    foreach ($candidates_files as $key => $file) {
+        $config_file = dirname($file)."/config.json";
+        $config_string = file_get_contents($config_file);
+        $elected_file = dirname($file)."/candidates-elected.csv";
+
+        if ($config_string !== FALSE) {
+            $config = json_decode($config_string, true);
+        } else {
+            error_log("Error opening config.json.");
+            exit(1);
+        }
+
+        $elected_data = [];
+        if (file_exists($elected_file)) {
+            if (($elected_handle = fopen($elected_file, "r")) !== FALSE) {
+                $headers = fgetcsv($elected_handle);
+                while (($data = fgetcsv($elected_handle)) !== FALSE) {
+                    $candidate = [];
+                    foreach ($headers as $key => $value) {
+                        $candidate[$value] = $data[$key];
+                    }
+                    $name_slug = trim_sluggify($candidate['Candidate Name']);
+                    $elected_data[$name_slug] = $candidate;
+                }
+            } else {
+                error_log('Error opening candidates file');
+                exit(1);
+            }
+        }
+
+        if (($handle = fopen($file, "r")) !== FALSE) {
+            $headers = fgetcsv($handle);
+            while (($data = fgetcsv($handle)) !== FALSE) {
+                $candidate = [];
+                $candidate['Pledge'] = 'n';
+                $candidate['Picture'] = "";
+                if (is_array($default_image)) {
+                    $candidate['image_url'] = $default_image['url'];
+                    $candidate['image_id'] = $default_image['id'];
+                } else {
+                    $candidate['image_url'] = "";
+                    $candidate['image_id'] = "";
+                }
+                foreach ($headers as $key => $value) {
+                    $candidate[$value] = $data[$key];
+                }
+                $name_slug = trim_sluggify($candidate['Candidate Name']);
+                if (array_key_exists($name_slug, $candidate_data)) {
+                    if ($candidate_data[$name_slug]['Pledge'] === 'y') {
+                        $candidate['Pledge'] = 'y';
+                    }
+                }
+                if (!empty($elected_data) && array_key_exists($name_slug, $elected_data)) {
+                    $candidate['Elected'] = $elected_data[$name_slug]['Elected'];
+                }
+                $candidate['Council'] = $config['councilName'];
+                $candidate['Path'] = dirname($file);
+                $media_desc = $candidate['Path']."/".
+                                $candidate['Picture'].".json";
+                if (file_exists($media_desc)) {
+                    $media_string = file_get_contents($media_desc);
+                    if ($media_string !== FALSE) {
+                        $media = json_decode($media_string, true);
+                    } else {
+                        error_log("Error opening image descriptor.");
+                        exit(1);
+                    }
+                    /* Get photo URL and ID */
+                    $candidate['image_url'] = $media['url'];
+                    $candidate['image_id'] = $media['id'];
+                }
+                $candidate_data[$name_slug] = $candidate;
+            }
+            fclose($handle);
+        } else {
+            error_log('Error opening candidates file');
+            exit(1);
+        }
+
+        /* Override pledge columns if pledges.csv is present */
+        $pledges_file = dirname($file)."/pledges.csv";
+        if (!file_exists($pledges_file)) continue;
+
+        if (($handle = fopen($pledges_file, "r")) !== FALSE) {
+            $headers = fgetcsv($handle);
+            while (($data = fgetcsv($handle)) !== FALSE) {
+                $candidate = [];
+                foreach ($headers as $key => $value) {
+                    $candidate[$value] = $data[$key];
+                }
+                $candidate_data[trim_sluggify($candidate['Candidate Name'])]['Pledge'] = 
+                    $candidate['Pledge'];
+            }
+            fclose($handle);
+        } else {
+            error_log('Error opening pledges file');
+            exit(1);
+        }
+    }
+
+    return $candidate_data;
+}

pledge-update/pledge-page.php

@@ -0,0 +1,64 @@
+<?php
+
+require_once("parse_pledge_data.php");
+require_once("page_renderer.php");
+
+$options = getopt("", ["candidates-files:", "default-image:"]);
+
+if (isset($options['candidates-files'])) {
+    $candidates_files = $options['candidates-files'];
+} else {
+    error_log("Error: Missing required option '--candidates-files'.");
+    exit(1);
+}
+
+if (isset($options['default-image'])) {
+    $default_image = $options['default-image'];
+} else {
+    error_log("Error: Missing required option '--default-image'.");
+    exit(1);
+}
+
+$default_image = file_get_contents($default_image);
+
+if ($default_image !== FALSE) {
+    $default_image = json_decode($default_image, true);
+} else {
+    error_log("Error opening config.json.");
+    exit(1);
+}
+
+$candidate_data = parse_pledge_data(explode(" ", $candidates_files), $default_image);
+
+/* Select people who have taken the pledge */
+$pledgeCandidates = array_filter($candidate_data, function ($candidate) {
+    return $candidate['Pledge'] === 'y';
+});
+
+$renderer = new SPLPageRenderer();
+//print_r($pledgeCandidates);
+
+$councils = [];
+$lga_pages_unsort = [];
+foreach ($pledgeCandidates as $key => $candidate) {
+    $councils[] = $candidate['Council'];
+    $lga_pages_unsort[] = basename($candidate['Path']);
+}
+$councils = array_unique($councils);
+asort($councils);
+
+$lga_pages = [];
+foreach ($councils as $key => $council) {
+    $lga_pages[$key] = $lga_pages_unsort[$key];
+}
+
+//print_r($councils);
+
+$pageContent = $renderer->renderPledgePage($councils, $lga_pages, $pledgeCandidates);
+
+if ($pageContent === null) {
+    exit(2);
+}
+
+echo $pageContent;
+exit(0);

pledge-update/template.php

@@ -0,0 +1,143 @@
+<?php 
+function sluggify($input) {
+    return strtolower(str_replace(' ', '-', $input));
+}
+?>
+
+<!-- wp:paragraph -->
+<p>The Streets People Love campaign offers council candidates the opportunity to take the following pledge:</p>
+<!-- /wp:paragraph -->
+
+<!-- wp:paragraph {"style":{"color":{"background":"#10B5B0"}}} -->
+<p class="has-background" style="background-color:#10B5B0">If elected Councillor, I pledge to allocate budget and street space to build streets people love, and ensure that residents of all ages and abilities can safely move around our council area, irrespective of whether they choose to walk, cycle, wheel, use public transport or drive.</p>
+<!-- /wp:paragraph -->
+
+<!-- wp:paragraph -->
+<p>Candidates from these councils have taken the pledge:</p>
+<!-- /wp:paragraph -->
+
+<?php 
+$councilCount = count($councils);
+?>
+
+<?php if ($councilCount > 1): ?>
+
+    <?php 
+
+    if ($councilCount > 8) {
+        $councilListChunkSize = ceil($councilCount / 2);
+    } else {
+        $councilListChunkSize = $councilCount;
+    }
+
+    $councilChunks = array_chunk($councils, $councilListChunkSize);
+
+    ?>
+    <!-- wp:columns {"className":"council-list-columns"} -->
+    <div class="wp-block-columns council-list-columns">
+
+    <?php for ($columnIdx = 0; $columnIdx < 4; $columnIdx++): ?>
+        <!-- wp:column {"verticalAlignment":"top","style":{"spacing":{"padding":{"top":"0","bottom":"0"}}}} -->
+        <div class="wp-block-column is-vertically-aligned-top" style="padding-top:0;padding-bottom:0">
+            
+            <?php if (array_key_exists($columnIdx, $councilChunks)): ?>
+                <!-- wp:list {"style":{"spacing":{"margin":{"top":"0","right":"0","bottom":"0","left":"0"}}}} -->
+                <ul style="margin-top:0;margin-right:0;margin-bottom:0;margin-left:0" class="wp-block-list">
+
+                    <?php foreach($councilChunks[$columnIdx] as $councilName): ?>
+                        <!-- wp:list-item -->
+                        <li><a href="#<?php echo sluggify($councilName); ?>"><?php echo $councilName; ?></a></li>
+                        <!-- /wp:list-item -->
+                    <?php endforeach; ?>
+
+                </ul>
+                <!-- /wp:list -->
+            <?php endif; ?>
+
+        </div>
+        <!-- /wp:column -->
+    <?php endfor; ?>
+
+    </div>
+    <!-- /wp:columns -->
+<?php else: ?>
+    <!-- wp:paragraph -->
+    <p></p>
+    <!-- /wp:paragraph -->
+<?php endif; ?>
+
+<?php foreach ($councils as $key => $council): ?>
+    <!-- wp:heading {"level":3,"className":"is-style-default"} -->
+    <?php $groupSlug = sluggify($council); ?>
+    <h3 class="wp-block-heading is-style-default" id="<?php echo $groupSlug; ?>"><a style="text-decoration: none;" href="/<?php echo $lga_pages[$key]; ?>"><?php echo htmlspecialchars($council); ?></a></h3>
+    <!-- /wp:heading -->
+
+    <?php
+    $groupCandidates = array_filter($candidates, function ($candidate) use ($council) {
+        return $candidate['Council'] === $council;
+    });
+
+    ?>
+
+    <?php
+    $columnCount = 4;
+
+    $chunkedCouncilCandidates = array_chunk($groupCandidates, $columnCount);
+    ?>
+
+    <?php foreach($chunkedCouncilCandidates as $chunk): ?>
+        <!-- wp:columns -->
+        <div class="wp-block-columns">
+            
+            <?php for ($columnIdx = 0; $columnIdx < $columnCount; $columnIdx++): ?>
+                <!-- wp:column -->
+                <div class="wp-block-column">
+
+                    <?php if (array_key_exists($columnIdx, $chunk)): ?>
+                        <?php 
+                            $candidate = $chunk[$columnIdx];
+
+                            if (isset($candidate['Elected']) && $candidate['Elected'] === 'y') {
+                                $candidate_elected = true;
+                            } else {
+                                $candidate_elected = false;
+                            }
+
+                            if (isset($candidate['Picture']) && strlen($candidate['image_url'])) {
+                                $candidate_image['url'] = $candidate['image_url'];
+                                $candidate_image['id'] = $candidate['image_id'];
+                            } else {
+                                continue;
+                            }
+
+                        ?>
+
+                        <!-- wp:image {"id":<?php echo $candidate_image['id']; ?>,"width":"200px","height":"200px","scale":"cover","align":"center","style":{"color":{}},"className":"is-resized"} -->
+                        <figure class="wp-block-image aligncenter is-resized <?php if ($candidate_elected) { echo "elected-candidate"; } ?>"><img src="<?php echo $candidate_image['url']; ?>" alt="" class="wp-image-<?php echo $candidate_image['id']; ?>" style="object-fit:cover;width:200px;height:200px;"/>
+                            <?php if ($candidate_elected): ?>
+                                <figcaption>ELECTED</figcaption>
+                            <?php endif; ?>
+                        </figure>
+                        <!-- /wp:image -->
+
+                        <!-- wp:heading {"textAlign":"center","className":"wp-block-heading has-text-align-center has-medium-font-size","style":{"spacing":{"margin":{"top":"1rem","bottom":"0.5rem"}}}} -->
+                        <h2 class="wp-block-heading has-text-align-center has-medium-font-size" style="margin-top:1rem;margin-bottom:0.5rem"><strong><?php echo htmlspecialchars($candidate['Candidate Name']); ?></strong></h2>
+                        <!-- /wp:heading -->
+
+                    <?php endif; ?>
+
+                </div>
+                <!-- /wp:column -->
+            <?php endfor; ?>
+
+        </div>
+        <!-- /wp:columns -->
+    <?php endforeach; ?>
+    
+<?php endforeach; ?>
+
+<?php if (isset($config['footer'])): ?>
+<!-- wp:paragraph -->
+<p><?php echo $config['footer']; ?></p>
+<!-- /wp:paragraph -->
+<?php endif; ?>

results/fetch.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+
+mkdir -p html
+wget https://www.vec.vic.gov.au/results/2024-council-election-results -O html/lga_list.html
+
+IFS=$'\n'
+
+lgas=$(grep 'href="/voting/.*/results"' html/lga_list.html)
+
+for lga in $lgas ; do
+    lga=$(sed 's|.*href="|https://www.vec.vic.gov.au|' <<< $lga)
+    lga=$(sed 's|">.*||' <<< $lga)
+    file=$(sed 's|.*elections/||' <<< $lga | sed s'|/results||')
+    wget $lga -O html/$file
+done

results/gen-elected.php

@@ -0,0 +1,142 @@
+<?php
+
+$options = getopt("", ["candidates-files:", "results-file:"]);
+
+if (isset($options['candidates-files'])) {
+    $candidates_files = $options['candidates-files'];
+} else {
+    error_log("Error: Missing required option '--candidates-files'.");
+    exit(1);
+}
+
+if (isset($options['results-file'])) {
+    $results_file = $options['results-file'];
+    $results_string = file_get_contents($results_file);
+
+    if ($results_string !== FALSE) {
+        $results = json_decode($results_string, true);
+    } else {
+        error_log("Error opening results.json.");
+        exit(1);
+    }
+} else {
+    error_log("Error: Missing required option '--results-file'.");
+    exit(1);
+}
+
+function trim_sluggify($input) {
+    return strtolower(str_replace(' ', '-', trim($input)));
+}
+
+function match_words($words, $list) {
+    /* Match database names to VEC names */
+    $max_score = 0;
+    $best_match = "no match";
+    foreach ($list as $possible_match) {
+        $aa = preg_split("/[^a-z]/", strtolower($words));
+        $bb = preg_split("/[^a-z]/", strtolower($possible_match));
+
+        $score_sum = 0;
+        foreach ($aa as $a) {
+            foreach ($bb as $b) {
+                similar_text($a, $b, $score);
+                if ($score > 70) $score_sum += $score;
+                else $score_sum -= 1;
+            }
+        }
+
+        if ($score_sum > $max_score) {
+            $max_score = $score_sum;
+            $best_match = $possible_match;
+        }
+    }
+    return array($max_score, $best_match);
+}
+
+$candidates_files = explode(" ", $candidates_files);
+
+/* Generate dictionary of candidates and LGAs */
+$candidate_data = [];
+foreach ($candidates_files as $file) {
+    $config_file = dirname($file)."/config.json";
+    $config_string = file_get_contents($config_file);
+
+    if ($config_string !== FALSE) {
+        $config = json_decode($config_string, true);
+    } else {
+        error_log("Error opening config.json.");
+        exit(1);
+    }
+
+    $candidate_data[$config['councilName']]['_filename'] = $file;
+
+    if (($handle = fopen($file, "r")) !== FALSE) {
+        $headers = fgetcsv($handle);
+        while (($data = fgetcsv($handle)) !== FALSE) {
+            $candidate = [];
+            foreach ($headers as $key => $value) {
+                $candidate[$value] = $data[$key];
+            }
+            $name_slug = trim_sluggify($candidate['Candidate Name']);
+            $candidate_data[$config['councilName']][$name_slug] = $candidate;
+        }
+    }
+}
+
+$vec_lga_names = [];
+foreach ($results as $lga => $data) {
+    $vec_lga_names[] = $lga;
+}
+
+function was_elected($candidate, $vec_wards) {
+    foreach ($vec_wards as $vec_candidates) {
+        list($score, $match) = match_words($candidate, $vec_candidates);
+        if ($score > 180) return true;
+    }
+    return false;
+}
+
+$header = ["Ward", "Candidate Name", "Elected"];
+
+foreach ($candidate_data as $lga => $db_candidates) {
+    /* Find LGA in results dict */
+    list($score, $vec_lga_name) = match_words($lga, $vec_lga_names);
+    $vec_wards = $results[$vec_lga_name];
+
+    $elected = [];
+    /* Go through database candidates and build list of elected candidates */
+    foreach ($db_candidates as $key => $value) {
+        if ($key === '_filename') {
+            $output_file = dirname($value)."/candidates-elected.csv";
+            continue;
+        }
+        if (was_elected($value['Candidate Name'], $vec_wards)) {
+            $elected[] = $value;
+        }
+    }
+
+    /* Don't create file if none were elected. */
+    if (count($elected) === 0) continue;
+
+    if (($handle = fopen($output_file, "w")) === FALSE) {
+        error_log('Error opening output file');
+        exit(1);
+    }
+
+    if (fputcsv($handle, $header) === FALSE) {
+        error_log('Error writing headers to output file');
+        exit(3);
+    }
+
+    foreach ($elected as $candidate) {
+        $line = array($candidate['Ward'], $candidate['Candidate Name'], "y");
+        if (fputcsv($handle, $line) === FALSE) {
+            error_log('Error writing candidate to output file');
+            exit(3);
+        }
+    }
+
+    fclose($handle);
+}
+
+exit(0);

results/parser.py

@@ -0,0 +1,53 @@
+from bs4 import BeautifulSoup, Tag as HTMLTag
+import json, re, argparse
+
+parser = argparse.ArgumentParser()
+parser.add_argument('filenames', nargs='*')
+args = parser.parse_args()
+
+def get_vacancies(ward):
+    text = ward.parent.parent.h2.text
+    ward_name = re.search("[^\(]*", text)[0].strip()
+    vacancies = int(re.search("\([0-9]+", text)[0].strip("("))
+    return (ward_name, vacancies, ward)
+
+def get_candidate_names(ward_desc):
+    names = []
+    for sibling in ward_desc[2].parent.next_siblings:
+        if not isinstance(sibling, HTMLTag):
+            continue
+        if not (blocks := sibling.find_all('td', class_="list-item-body")):
+            continue
+        for block in blocks:
+            names.append(re.sub('\n.*', '', block.text.strip()))
+        return names
+
+def parse_lga(filename):
+    with open(filename, 'r') as results_fp:
+        html_doc = results_fp.read()
+
+    soup = BeautifulSoup(html_doc, 'html.parser')
+    wards0 = soup.find_all(string="Successful candidates")
+    wards1 = soup.find_all(string="Elected candidates")
+
+    ward_info = []
+    for ward in wards0:
+        ward_info.append(get_vacancies(ward))
+    for ward in wards1:
+        ward_info.append(get_vacancies(ward))
+
+    results = {}
+    for ward in ward_info:
+        names = get_candidate_names(ward)
+        assert len(names) == ward[1]
+        results[ward[0]] = names
+    
+    return results
+
+all_results = {}
+for lga in args.filenames:
+    lga_name = re.sub('html/lgas/', '', lga)
+    results = parse_lga(lga)
+    all_results[lga_name] = results
+
+print(json.dumps(all_results, indent=4))

update-elected.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+
+# This script uses the jq, wp, and php commands, make sure they are installed before running this script.
+
+# The folder containing data for each council.
+# Includes the list of candidates and any media.
+DATA_PATH="../spl-data"
+
+# Iterate over folders in data path
+candidates_files=()
+for folder in "$DATA_PATH"/*; do
+    if test -f "$folder"/candidates-generic.csv; then
+        candidates_files+=("$folder"/candidates-generic.csv)
+    fi
+    # Community groups get priority
+    if test -f "$folder"/candidates.csv; then
+        candidates_files+=("$folder"/candidates.csv)
+    fi
+done
+
+php results/gen-elected.php --candidates-files "${candidates_files[*]}" \
+                            --results-file $DATA_PATH/results.json

update-pledges.sh

@@ -0,0 +1,31 @@
+#!/bin/bash
+
+# If this runs as a cron job - might want to limit the number of revisions
+# wordpress stores:
+#   wp-config.php:
+#     define( 'WP_POST_REVISIONS', 3 );
+
+#wp post list --post_type=page
+#wp post get 426 --field=content > current-homepage
+#wp post get 1409 --field=content > movie-homepage
+#wp post create --post_type=page --post_title="test_pledge" movie-homepage
+#wp post update 1803 ../spl-data/movie-homepage
+
+DATA_PATH="../spl-data"
+
+candidates_files=()
+for folder in "$DATA_PATH"/*; do
+    if test -f "$folder"/candidates-generic.csv; then
+        candidates_files+=("$folder"/candidates-generic.csv)
+    fi
+    # Community groups get priority
+    if test -f "$folder"/candidates.csv; then
+        candidates_files+=("$folder"/candidates.csv)
+    fi
+done
+
+pledge_sed=$(php pledge-update/homepage.php --candidates-files "${candidates_files[*]}")
+
+content=$(sed "$pledge_sed" ../spl-data/movie-homepage)
+
+echo "$content" | wp post update 1803 -

upload-media.sh

@@ -7,7 +7,7 @@
 # Additionally, make sure the wp-cli/restful package is installed in the wp command (via "wp package install wp-cli/restful")
 
 # Controls the flags that are passed to every usage of the wp command.
-WP_FLAGS="--allow-root --path=/var/www/html"
+#WP_FLAGS="--allow-root --path=/var/www/html"
 
 media_path="$1"