[Xtide-announce] data refresh

[David Flater]

harmonics-dwf-20151227

Data refresh available from http://www.flaterco.com/xtide/files.html#harmonicsfiles.

   Imported NOS data from snapshot 2015-12-15.

   Note:  All comparisons with NOAA predictions now are using dynamically
   generated "online" predictions rather than "tables" that used to be posted
   annually.  Differences of 5 minutes are now common.

   *Tides*

   Harmonic constants are now being retrieved from
   opendap.co-ops.nos.noaa.gov/axis.

   Datums are now being retrieved from tidesandcurrents.noaa.gov/api.
   Benchmark Data Sheets are no longer used.  In the 37 cases where the API
   failed to return a usable value, the Benchmark Data Sheet either didn't
   exist or didn't help.  The same was true of online predictions.  Missing
   datums are still being set to an estimated MLLW that most often turns out
   to be lower than the value eventually adopted by NOAA.

   Datums for stations on the Columbia River have been corrected.
   Previously, import mishandled stations that were using the Columbia River
   Datum.  Bug report credit:  Mark Hayden.

   The time zone of Savoonga has been harmonized with the rest of
   St. Lawrence Island (HAST).  Wikipedia sources are now claiming that
   St. Lawrence Island is on AKST (which is where NOAA puts it), but see
   http://alaska.gov/kids/learn/timezones.htm:  "The far reaches of the
   Aleutian Islands and St. Lawrence Island are in the next zone,
   Hawaii-Aleutian Standard Time."  The 2003 CFR do not clearly address
   St. Lawrence Island at all, though by default, it appears that it should
   be on Alaska time.

   The errata for Texaco Dock and Helmick Point are unchanged.

   *Currents*

   Harmonic constants for currents still are not available for download.  The
   following reference current stations no longer match NOS predictions and
   have been expired along with all dependent subordinate stations:
     Old Tampa Bay Entrance (Port Tampa), Florida Current (15d)
     Tampa Bay (Sunshine Skyway Bridge), Florida Current (15d)

   The import of subordinate current stations has been migrated from the old
   Table 2-based pages, which still have not been posted for 2016, to the new
   "NOAA Current Predictions *Beta*" site.  Names now include "Depth Xft" in
   the middle instead of "(Xd)" at the end to indicate depths.  Legacy
   current refs were not renamed.

   The footnotes that used to be in Table 2 have been dropped.

   The harmonic constants for Wrangell Narrows obtained in 2011 were labelled
   as depth 6 ft.  They appear to match with the online predictions for depth
   4 ft and are being used for all subordinate stations that request the 4 ft
   reference.

   *Coordinates*

   The coordinates for all imported stations were taken preferably from the
   regional station lists.  Imported reference stations not included in the
   regional lists got their coordinates from the metadata of the harmonic
   constants.  The coordinates of legacy current refs were left alone.

   Coordinates retrieved from different places usually disagree with one
   another.  There is no general pattern such as rounding to the nearest
   minute to show that one or the other is better.  However, for a small
   sample of reference stations where the disagreement was large (9455606,
   9457527, 9465831), the station list coordinates were more plausible.

   Coordinates for American Samoa are now in the correct hemisphere.

   *Constituent definitions*

   I received untraceable evidence this year that NOAA may be using
   definitions of 3KM5, RP1, and KJ2 that are phase-reversed from what is in
   the harmonics file.  At present the only impacted NOAA station is
   Anchorage, where 3KM5 has amplitude of 0.04 ft and the other two are null.
   Comparison of all high/low tide predictions for Anchorage for year 2015
   with NOAA tides online showed on average a small improvement after
   flipping the phase of 3KM2:

                     Time RMSE (s)    Tide RMSE (cm)
   Default              63.7              2.1
   Phase-reversed       44.1              1.7

   Congen formulates 3KM5 as a compound constituent, which does not offer the
   opportunity to accidentally reverse the phase with a data entry error.
   More investigation is needed.  For now I am holding off on upheaving the
   constituent definitions, which would be a breaking change for anyone using
   harmonic constants from non-NOAA sources, and instead am merely flipping
   the phase of 3KM5 for Anchorage as an erratum.

   *Other notes*

   The anomaly with St. Michael, Norton Sound, Alaska is unchanged.

   Some stations show dramatically different names in the regional station
   list than on their prediction pages.  It is hopeless to try to identify
   all of these and figure out where they really are.

   "D.C.", "Washington, D.C." and "District of Columbia" have been
   standardized to just "D.C."

   Inconsistent capitalization in the names of subordinate current stations
   is not worth fixing.  XTide's name matching is case-insensitive.

-------------------

harmbase2-20151227

    PostgreSQL 9.4.5.

    Significant changes to import.
    - Harmonic constants from opendap.co-ops.nos.noaa.gov/axis
    - Datums from tidesandcurrents.noaa.gov/api
    - Sub currents from "NOAA Current Predictions *Beta*" site
    - Coordinates preferentially taken from regional station lists
    - The relevant NOS formats are now called NOS (for tidal harmonic
      constants), NOSST (NOS subordinate tide), and NOSSC (NOS
      subordinate current).

    For this year, the names of all sub current stations were synthesized
    from scratch instead of matched from old data.  The few post-import
    fixups are in renamings-cursubs.txt.

    On the NOAA currents site, if a station is in an area (e.g., Potomac
    River) that crosses several states (e.g., MD, VA, DC), the station is
    duplicated under all three states.  Import uses the state from the
    station list if it is unambiguous and otherwise makes a guess based
    on the nearest station appearing in last year's data.

    Corrected handling of stations on the Columbia River Datum.  Bug
    report credit:  Mark Hayden.

    The rename utility once again applies to all stations but logging has
    been adjusted on the assumption that only a few stations are being
    renamed.  Unmatched renamings are reported; unmatched stations are
    not.

    Purged obsolete code.

    Fixed time zone detection for Savoonga.


-- DWF