Pre-Deployment Checklist

A printable checklist of absolute requirements and it-would-be-nice-if requirements is available at ${BUOY_ROOT}/etc/requirements.kwd. .

The following bits of information must be obtained before starting the buoy processing.

1. The GOES platform ID and the GOES binary file layout for the particular buoy. This may require writing a new goes2?????.m routine.

2. The accelerometer processing has no special requirements. The serial number is suggested, though.

3. RDI doppler processing has no special requirements other than the correct number of doppler bins be properly set in the NetCDF files. This is usually a concern only for the GOES doppler processing, as the campbell logger usually accounts for the full 30 bins.. This should be taken care of during pre-deployment testing.

4. The FSI processing has no special requirements.

5. The met processing has no special requirements.

6. The SBE16 plus usually has either an optional C Star transmissometer (buoy K) or an oxygen sensor (A and C), which require several calibration coefficients and extra parameters each. The manual for each instrument should provide the calibration coefficients.

   Table 1. SBE16plus Requirements

   Requirement	Description	NetCDF Construct	SED Script Tag
   Boc	oxygen current bias	dissolved_oxygen:Boc	SBE16_BOC
   Soc	oxygen current slope	dissolved_oxygen:Soc	SBE16_SOC
   tcor	temperature correction factor for membrane permeability	dissolved_oxygen:TCor	SBE16_TCOR
   pcor	pressure correction factor for membrane permeability	dissolved_oxygen:PCor	SBE16_PCOR
   offset	voltage produced for zero oxygen current	dissolved_oxygen:offset	SBE16_OXOFFSET
   Vd	analog output voltage of the instrument with the beam blocked.	transmissivity:Vd	V_DARK
   Vair	analog output voltage of the instrument with a cleared beam path	transmissivity:Vair	V_AIR
   Vref	analog output voltage of the instrument with clean water in the path	transmissivity:Vref	V_REF
   path length	Optical path length, usually 0.10 or 0.25 m. This is not set at the factory.	transmissivity:pathlength	PATHLENGTH
   Transmissometer Serial number	Serial number of the transmissometer	{transmissivity,c}:sensor_sn	TRANSMISSOMETER_SERIAL_NUMBER
   Instrument Number	Id of the instrument as identified by the campbell logger. For an SBE16 plus, this is usually 11 or 12. It is required in order to load the SBE16 NetCDF files into the database.	global attributes:instrument_NUMBER	SBE16_INSTRUMENT_NUMBER
   Instrument Depth	The SBE16 has gone in at 10, 15, 20, and 50 meters depth, so this is not hard coded.	Value of depth variable.	SBE16_DEPTH

7. All the optics instruments (OCR504, OCR507, AC9, VSF, and DFLS) come with calibration files and tables. These need to be tested and loaded into the proper locations. The m-files to test the calibration files are

   • ReadDFLScal

   • ReadOCRcal

   • ReadVSF3cal

   • ReadAC9cal

   The directories under which these should be stored is ${DATA_ROOT}/ancillary/optics/Calibrations

8. If an Aanderaa current meter is part of the deployment plan (usually it is for all buoys), then the following calibration parameters are required. These parameters must be put into the sed file used to create the CDL files from whence the NetCDF files are produced.

   Table 2. Aanderaa Current Meter Requirements

   Requirement	Description	NetCDF Attribute	SED Script Tag
   Aanderaa Wide Range Temperature Calibrations	Calibration coefficients to turn raw temperature counts into celsius. It is always the wide range set of calibrations.	temperature:calibration_coefs	AANDERAA_WT_CALIBRATION
   Aanderaa Serial number	Serial number of the Aanderaa current meter	global attributes:aanderaa_serial_number	AANDERAA_SERIAL_NUMBER

   There are also calibration coefficients for the current speed and direction, but these coefficients have thusfar remained the same for all deployments, so they are currently hardwired into the CDL files.

9. If an SBE16 or SBE1603 is part of the deployment plan, the instrument_number must be determined. This id is in the 2nd column in the Campbell data stream and seems to usually be 11 or 12.

10. If part of the deployment plan, the dissolved oxygen calibrations.

Any reference to ${BUOY_ROOT} below refers to the official GoMOOS disk layout. Right now, this is gyre:/data/gomoos/buoy

In order to successfully prepare for a buoy deployment, one must create the NetCDF files that describe the instruments and parameters being measured by the buoy. The files should be created from CDL files (see the man page for ncgen ( 1 ) and ncdump ( 1 ) ). Rather than create these from scratch, one should copy existing CDL templates from the appropriate ${BUOY_ROOT}/cdl directories or use ncdump to create a CDL file from an existing NetCDF file. Generally, a previous deployment's CDL files will serve as a good guide to what the pending deployment's CDL files should look like.

Due to the high overhead of getting the setup done correctly, it is a good idea to start the deployment while in the test phase. For example, assuming that buoy A0107 is to be deployed. The deployment plan called for the following instruments:

1. RM Young (met)

2. RDI Workhorse (doppler)

3. 1. SBE37 SM at 1m

   2. SBE37 IM at 20m

   3. SBE37 IM at 50m

4. Aanderaa Current Meter

5. Accelerometer

6. Diagnostics

7. Above and surface optics.

Make directory and copy all of the necessary CDL files into it. The CDL files can be found in ${BUOY_ROOT}/cdl. As of recent deployments, the CDL files are logically separated by processing stream and physically separated into subdirectories. This is done to make the NetCDF files easier to visually inspect. Since the A0107 deployment will have data telemetered back via both cell phone and GOES, there will be a sensor-raw and realtime datastream, as well as a goes-sensor-raw and goes-realtime datastream. For backwards compatibility with GOMOOS, the raw and goes-raw datastreams are provided as well. The raw datastream is an amalgamation of sensor-raw and realtime products (and is also END-OF-LIFEd). PHOG requirements also give rise to an hourly and filtered datastream deriving from the arising out of the sensor-raw datastream. This gives eight total logical datastreams (and a lot of NetCDF files), so separating them into subdirectories makes sense.

jevans@micmac: > ls /data/gomoos/buoy/archive/A0107
goes-realtime/  goes-sensor-raw/  realtime/  sensor-raw/

jevans@micmac: > ls /data/gomoos/buoy/archive/A0107/*/*.nc

/data/gomoos/buoy/archive/A0107/goes-realtime:
A0107.aanderaa.goes-realtime.nc
A0107.accelerometer.goes-realtime.nc
A0107.doppler.goes-realtime.nc
A0107.met.goes-realtime.nc
A0107.sbe37.goes-realtime.1m.nc
A0107.sbe37.goes-realtime.20m.nc
A0107.sbe37.goes-realtime.50m.nc

/data/gomoos/buoy/archive/A0107/goes-sensor-raw:
A0107.aanderaa.goes-sensor-raw.nc
A0107.accelerometer.goes-sensor-raw.nc
A0107.diagnostics.goes-sensor-raw.nc
A0107.doppler.goes-sensor-raw.nc
A0107.met.goes-sensor-raw.nc
A0107.optics_a.goes-sensor-raw.nc
A0107.optics_s.goes-sensor-raw.nc
A0107.sbe37.goes-sensor-raw.1m.nc
A0107.sbe37.goes-sensor-raw.20m.nc
A0107.sbe37.goes-sensor-raw.50m.nc

/data/gomoos/buoy/archive/A0107/realtime:
A0107.aanderaa.realtime.nc
A0107.accelerometer.realtime.nc
A0107.doppler.realtime.nc
A0107.met.realtime.nc
A0107.optics_a.realtime.nc
A0107.optics_as.realtime.nc
A0107.optics_s.realtime.nc
A0107.sbe37.realtime.1m.nc
A0107.sbe37.realtime.20m.nc
A0107.sbe37.realtime.50m.nc

/data/gomoos/buoy/archive/A0107/sensor-raw:
A0107.aanderaa.sensor-raw.nc
A0107.accelerometer_fft.sensor-raw.nc
A0107.accelerometer.sensor-raw.nc
A0107.diagnostics.sensor-raw.nc
A0107.doppler.sensor-raw.nc
A0107.goes_tx.sensor-raw.nc
A0107.met.sensor-raw.nc
A0107.monitor.sensor-raw.nc
A0107.optics_a.sensor-raw.nc
A0107.optics_s.sensor-raw.nc
A0107.sbe37.sensor-raw.1m.nc
A0107.sbe37.sensor-raw.20m.nc
A0107.sbe37.sensor-raw.50m.nc

      

The template files must be edited before turning them into actual NetCDF files. For the most part, the necessary edits are global attributes. The required global attributes are

• buffer_type: specifies the datastream for this NetCDF file. In this example, we have an accelerometer for wave data.

• instrument_number: a numeric ID that can discriminate one of possibly multiple identical instruments (such as Seacats at 1 meter, 10 meters, 20 meters, or 50 meters). Usually (but not always), if there is just a single instrument of a specific type, this value is zero.

• processing_type: for the realtime data coming in from the cell phone, this is [goes-]sensor-raw ( [goes-]raw for backwards compatibility ) for the data telemetered back, and [goes-]realtime for the processed products (also [goes-]raw in backwards compatibility case ).

• history: any non-routine operation on the NetCDF file should append to this attribute, detailing exactly what is being done.

• water_depth: required for certain doppler operations, general metadata

• mooring_site_id: in this example, it is A0107

• mooring_site_desc: general text description of where the buoy will be

• breakout_id: specifies a numeric id that identifies it in the Campbell input file

• magnetic_variation: specific to each buoy, used to modify the directions computed for wind and currents due to the offset of the magnetic north pole.

• last_mysql_update_index: currently used to keep track of the last record for which the MySQL gomoos_data database has been updated. May be deprecated.

$ runsed *.cdl 
      

When finished, create the NetCDF files:

$ /data/gomoos/buoy/bin/make_ncfiles.sh A0107
      

Install them in the proper directory. This should be something like ${BUOY_ROOT}/buoy/archive/A0107 or whatever, depending upon what the mooring ID is. Actually, you will probably want to install them to the subdirectories sensor-raw , goes-sensor-raw , realtime , goes-realtime , etc.

Make sure that each file is owned by the gomoos user and users group. This is crucial. Just about every problem you will encounter after setting the data processing in motion can be traced back to a permissions problem.

Invoke MATLAB and place them into the MySQL database:

>> options = get_options ( 'project=gomoos', 'mooring_id=A0107'  );
>> init_deployment ( options, 'relative/subdir/of/of/root_dir' );
      

The Campbell data file needs to be pointed in the proper directory. This is done in the Setup Connections window. Click on the proper buoy, then hit the Browse button. Point to the gomoos share. On gyre, the data file will show up as gyre:/data/gomoos/buoy/incoming/DATA/A0107.DAT. Change to the above directory and execute the command "ln -s DATA/A0107.DAT ./A0107.dat". Note the lowercase "dat".

A good way to test the deployment is to use matlab and try to read the input file as follows:

>> f = '/data/gomoos/buoy/incoming/A0107.dat'

f =

/data/gomoos/buoy/incoming/A0107.dat

>> [b,s] = read_campbell_buffer ( options, f );
Reading Campbell data...
Discriminating met data...
Discriminating doppler data...
Discriminating sbe37 data...
Discriminating aanderaa data...
Discriminating accelerometer data...
Discriminating diagnostics data...
Discriminating fft data...
Discriminating monitor data...
>> b

b =

  met           : {[1x1 struct]}
  doppler       : {[1x1 struct]}
  sbe37         : {[1x1 struct]  [1x1 struct]  [1x1 struct]}
  aanderaa      : {[1x1 struct]}
  accelerometer : {[1x1 struct]}
  diagnostics   : {[1x1 struct]}
  fft           : {[1x1 struct]}
  monitor       : {[1x1 struct]}


      

You can just run the matlab processing by starting up MATLAB from the command line and typing something like

>> process_realtime('A0107);
      

Assuming you are ok at this point, the last thing you need to do is to add an entry for A0107 in the crontab file. The following entry times the processing to start 10 minutes after the hour. You should make sure that whatever time you choose is a few minutes after the buoy is scheduled to call in.

10 * * * *     /data/gomoos/buoy/bin/process_buoys.sh A0107 
      

Now watch the setup as it processes the buoy data each hour. The system should catch any particularly egregious errors and report them via email, and the /data/gomoos/buoy/var/log/buoy.log file may prove helpful in further determining any problems.

When satisfied with the setup, soft-link the file correctly.

$> cd ${BUOY_ROOT}/buoy/incoming
$> ln -s DATA/A0107.DAT ./A0107.dat".  You are now setup to