.. _data-access:
Data Access
===========
CDIP data can be accessed using a variety of data services
from a number of organizations:
- CDIP THREDDS data server
- CDIP Python API
- CDIP CGI scripts
- CDIP web
- NDBC web and Dial-A-Buoy service
- NWS
CDIP THREDDS data server
------------------------
Our THREDDS data server is the preferred method of accessing the data.
Both observed and modeled data can be browsed using the default
`web interface`_ or can be automatically accessed
using the `OpenDap`_ url builder provided by THREDDS.
In addition, the data can be accessed programmatically by
`Python`_ using the netCDF library or by `Matlab`_.
To get started check out our `plotting examples`_.
CDIP netCDF data files
~~~~~~~~~~~~~~~~~~~~~~
A good way to see which data files we serve is
to browse them using the THREDDS `web interface`_.
- `Archived data`_ folder contains data of all past station deployments and should not change over time.
- `Realtime data`_ folder contains data of the current deployment and is updated every 30 minutes.
- `Modeled data`_ folder contains output from a variety of CDIP wave models.
Within the *Realtime* folder is the file `latest_3day.nc`_ which contains the latest 3 days
of data for all stations. In addition, for each station there are two files: an *rt* file which contains
wave spectra and wave parameters for the current deployment and an *xy* file which contains
buoy motion data in 3 directions (vertical, east-west and north-south) for the current deployment.
Within the *Archive* folder each station has two types of file: the *deployment* file indicated
with a deployment number (e.g `036p1_d42.nc`_ is the 42'nd deployment of station 036) and
an *historic* file containing wave spectra and parameters over all deployments (but contains no xy data).
Within the *Models* folder are three data sets containing model output for validation points, grid points
and alongshore points. Each such point can be thought of as a virtual buoy. The validation points
are specific points chosen to help validate the models, the grid points are used to produce
maps of wave statistics such as significant wave height (Hs) and the alongshore points are
nearshore points generally 10m - 20m in depth and 100m apart that extend over the entire California
coast.
.. _web interface : https://thredds.cdip.ucsd.edu
.. _Python : https://python.org
.. _Matlab : https://mathworks.com
.. _OpenDap : https://www.opendap.org/
.. _SOS : http://www.opengeospatial.org/standards/sos
.. _plotting examples : https://docs.google.com/document/d/1Uz_xIAVD2M6WeqQQ_x7ycoM3iKENO38S4Bmn6SasHtY/pub?embedded=true
.. _Archived data : https://thredds.cdip.ucsd.edu/thredds/catalog/cdip/archive/catalog.html
.. _Realtime data : https://thredds.cdip.ucsd.edu/thredds/catalog/cdip/realtime/catalog.html
.. _Modeled data : https://thredds.cdip.ucsd.edu/thredds/catalog/cdip/model/catalog.html
.. _latest_3day.nc : https://thredds.cdip.ucsd.edu/thredds/catalog/cdip/realtime/catalog.html?dataset=CDIP_Realtime/latest_3day.nc
.. _036p1_d42.nc : https://thredds.cdip.ucsd.edu/thredds/catalog/cdip/archive/036p1/catalog.html?dataset=CDIP_Archive/036p1/036p1_d42.nc
CDIP Python API
---------------
Currently our netCDF data files cannot be aggregated across deployments
using the THREDDS data server. For example to retrieve all of the xy data
for station 036 currently requires opening 44 deployment files. To handle this
problem and other aggregation problems with the wave models,
we have developed a Python API with methods that seamlessly access
both observed and modeled data.
Check out our Python API `plotting examples`_.
.. _Python API : https://bitbucket.org/cdip_admin/cdippy
CDIP CGI Scripts
----------------
In our effort to modernize data access we are recommending
using THREDDS because ... well, that's what it was designed for - serving data!
However, we do have a few CGI scripts that fulfill some needs
in a straightforward way.
- `ndar.cdip`_ To access buoy data and non-buoy wave data
- `sccoos.cdip`_ To get a summary of our active stations including location, wave parameters and depth
- `MEM_2dspectrum.cdip`_ To create a 2D (dir/freq) energy spectra using the Maximum Entropy Method (MEM)
.. _ndar.cdip : https://cdip.ucsd.edu/data_access/ndar.cdip
.. _sccoos.cdip : https://cdip.ucsd.edu/data_access/sccoos.cdip?h
.. _MEM_2dspectrum.cdip : https://cdip.ucsd.edu/data_access/MEM_2dspectra.cdip
.. _synopsis_9band.cdip : https://cdip.ucsd.edu/data_access/synopsis_9band.cdip
CDIP web
--------
The `CDIP website`_ also has a number of ways to interactively access our data as well as
a variety of statistical summaries and graphical representations of the data.
.. _CDIP website : https://cdip.ucsd.edu/
NDBC web and Dial-a-buoy
------------------------
The latest data from CDIP stations are also available through the
National Data Buoy Center's (`NDBC`_) web page and their
`Dial-A-Buoy service`_. Every thirty minutes the latest measurements from
CDIP's buoys are transmitted to the NDBC in FM-13 format. The NDBC then
distributes these data through their Advanced Weather Interactive Processing
System (AWIPS). NDBC identification numbers are displayed on CDIP
web site station pages.
.. _NDBC : https://www.ndbc.noaa.gov/
.. _Dial-a-buoy service : https://www.ndbc.noaa.gov/dial.shtml
NWS
---
Data are also transmitted every thirty minutes to the National Weather
Service (`NWS`_). These data are incorporated
into their weather reports, marine forecasts and weather radio
broadcasts.
.. _NWS : https://weather.gov
Data Use and Acknowledgements
-----------------------------
CDIP data and products are freely available for public use, provided
that they are not altered in any way. When used for web displays and
online resources, please provide a link to the CDIP homepage. For
instance, in standard html:
::
Data courtesy of CDIP
For offline references, please choose the appropriate form from the
recommended acknowledgements below.
- Short form (figure captions, etc.)
"... data from CDIP, Scripps Institution of Oceanography."
- Longer form (in text)
"...data were furnished by the Coastal Data Information Program,
Integrative Oceanography Division, operated by the Scripps
Institution of Oceanography."
- Full form (acknowledgements at conclusion of papers, etc.)
"...data were furnished by the Coastal Data Information Program
(CDIP), Integrative Oceanography Division, operated by the Scripps
Institution of Oceanography, under the sponsorship of the U.S. Army
Corps of Engineers and the California Department of Parks and
Recreation."
- `https://doi.org/10.18437/C7WC72 `_