Re-release of and other issues found in "5-km resolution ensemble climate data in Japan."

As announced on February 18, 2024, the "5-km resolution ensemble climate data in Japan" (d4PDF_5kmDDS_JP) has been temporarily unavailable. However, we found no defects in the netCDF file date information after investigation, so we are now resuming its release.

Please refer to "1. netCDF date and calendar issue" below for the handling of date information.

In addition to the date issue above, we have found another problem in several data. We plan to replace it in the future. Until then, please refrain from using these data. For details, please refer to "2. Other issue in some data."

1. netCDF date and calendar issue

A Python user asked us if the initial date in netCDF was incorrect. The data provider confirmed that the date was off by two days when read in Python or Matlab. Therefore, we temporarily stopped releasing the data, and the data provider and DIAS personnel investigated the netCDF data on DIAS.

First, we found that this symptom was only present in the 1-hour 2-dimensional variables (hourly surf and ph2m) and the 30-minute precipitation (30min).

The netCDF date information for these data is recorded as "standard" calendar type and the date as the total time since January 1, 1 AD." The "standard" calendar is a mixture of the Gregorian and Julian calendars, switching from Julian to Gregorian in October 1587 (see Gregorian and Julian calendars for more information). We have confirmed that the recorded dates are correct under this calendar.

The data provider and DIAS personnel have confirmed that GrADS, NCAR NCL, Python+Xarray, etc., can read netCDF's standard calendar correctly. In contrast, the Python "datetime" library and Matlab adopted the Proleptic Gregorian calendar, which applied the Gregorian calendar even before 1587, and we found that using these libraries resulted in a 2-day discrepancy.

Based on the above investigation, we have determined that there is no issue with the data on DIAS itself, and we are re-releasing the data in its original form without replacing it.

Please refer to the FAQ for possible measures for Python users. Since we have not found a Matlab library that can handle the data correctly, please create a program to calculate the date by yourself or use the data as if the initial time is July 24, 00Z.

2. Other issue in some data

We found an issue in the data in the following years and experiments. There are some negative values for precipitation.

Experiments and years in which issues were found are:

• HFB_4K_MP_m101: 2079, 2080, 2082, 2084, 2085, 2103
• HFB_4K_CC_m102: 2051
• HFB_2K_GF_m102: 2069
• HFB_2K_HA_m101: 2039
• HFB_2K_MP_m102: 2087
• HPB_m004: 1978, 1982, 1983, 1984, 1986

For these experiments and years, the directories are prefixed with "." and renamed ".YYYY" (YYYY is the year). Please do not use these data.

The data provider is currently undergoing recalculations, and these data will be replaced upon completion. We expect to replace these data in mid-April.

We apologize for any inconvenience.

Follow me!