1. What meteorological data exists within SITES?
2. What metadata is needed?
3. What meteorological data types exists?
4. What to publish?   
5. When to publish data?
6. Content and format of the data file
7. Before you upload
8. How to upload?
9. Documents

 
1. What meteorological data exists within SITES?  <back to top>

Long-term collection of meteorological data (sometimes referred to as weather data) within SITES is conducted at numerous locations. Each location is linked to a SITES station. A map of all locations is found here, and the list of all parameters here.

The station has the responsibility for the data collection, quality assessment, and publication of the data on the SITES portal conforming to the requirements regarding structure, format and metadata.  


 
2. What metadata is needed? <back to top>

Metadata is the information that describes and puts the dataset in a context in order to make it findable and usable.
The metadata elements for meteorological data is divided into four sections (tables):

1. Data type – the data type is in the SITES portal is a predefined set of mandatory and non-mandatory parameters, i.e. the set of columns of the data file
2. Location – is the geographical reference and the metadata describes several characteristics of this place.   
3. Station – is the creator and organization responsible for the data.

For continuous record as the meteorological data, metadata for each location and datatype will only need to be registered once, except the time period the data represents.    
All metadata for the three sections are available here.


 
3. What meteorological data types exists? <back to top>

Each data level and time resolution of the data defines a data type. Of technical reasons, there is for the meteorological data also a split into a separate datatypes based on what thematic programme the data belongs to (Station base programme or SITES Water).
The full list of datatypes is found here.  (see sheet datatype)


 
4. What to publish? <back to top>

 Two sets of data from each location within your station are to be published:

1. Data with the time resolution of the output (sub-hourly)
2. Daily mean data

From each location it is mandatory to publish level 2 (cleaned) meteorological data. Data levels refer to the SITES definitions.

It is also preferable that Level 0 data (logger output) is uploaded and stored on the portal. The metadata requirements for level 0 data are minimized as they are not intended for public access. 20191112: Portal is not yet ready for level 0, contact secretariat if you intend to upload such data


 
5. When to publish data? <back to top>
 

• The first batch of data can be uploaded at anytime (preferably a.s.a.p.). Daily mean data should form a time series as long as possible. Data at output (sub-hourly) time resolution should be uploaded in files containing individual years.
•  The output (sub-hourly) data should be uploaded four times a year, preferably corresponding to the quarters of the year. The dataset will be incremental within a calendar year, i.e. the file with the first quarter will be replaced with the file containing two quarters etc. See further in upload instructions.
• The daily mean data should be uploaded at least once a year – latest 31 March the following year. The dataset will be incremental building up a long time series, i.e. the file containing the time series daily means 1988-2018, will be replaced by a data file containing 1988-2019. See further upload instructions.  

 
6. Content and format of the data file <back to top>

Data columns
The data file should contain all the columns listed in the inventory for the specific location. If there are any discrepancies between the content of the file and the inventory contact SITES secretariat.
There is no general requirements regarding the order of the columns, expect that TIMESTAMP must be the first column.
 
Header
The file must carry a header with some basic metadata and explanation of the parameters included.
A template of the header is available here,
and an example of a full file (data and header) available here.

Once constructed a header for a specific data type and location it can be reused for subsequent uploads by only changing the time period the data represents.
Each new constructed header template should be sent to the secretariat for check and approval before upload.
 
Formats
The data file must:

• be comma separated using the extension .csv (make sure it is not semicolon separated)
• use point (not comma!) as decimal delimiter
• use NaN for missing data.
• use LOD (Limit of Detetion) for data under the detection limit
• be coded in UTF-8 (an excellent free text-editor tool that easily does the job is Notepad++ https://notepad-plus-plus.org/downloads/ )

 
The time format for TIMESTAMP must for:

• daily mean data be: (YYYY-MM-DD) (Year-Month-Day e.g. 2012-09-12)
• output time resolution (sub-hourly)  (YYYY-MM-DD hh:mm) (Year-Month-Day hours:minutes e.g. 2012-09-12 12:10)

File name format
The file name is constructed from:
SITES_[ABBREVIATION_DATATYPE]_[ABREV_STATION]_[ABREV_LOCATION]_[TIME_INTERVAL]_l[DATALEVEL]_[TIMERESOLUTION]

Abbreviations of datatype, station and location are found in the metadata document.
Time interval notation: YYYYMMDD-YYYYMMDD i.e. 20100101-20121231

Data level notation:
l0=level 0
l1=level 1
l2=level 2
etc

Time resolution notation:
Output resolution:
For all data on higher time resolution than daily mean the time interval is annotated, exampel:
10min
60min
2h

Daily mean is annotated with: daily


A data set with daily mean data level 3 from location Storglaciären, Tarfala:
SITES_MET_TRS_SGL_20121101-20141204_l2_daily.csv


 
7. Before you upload <back to top>

• Completed all metadata registration for the datatype, location, station. Document is here.
• Have a header template for the specific data type and location. If it is used for the first time it must be checked by SITES secretariat.
• Checked that the file complies with the requirements regarding format.

 

8. How to upload? <back to top>

Go to https://meta.fieldsites.se/uploadgui/
Please contact Jonathan Thiry at the Carbon Portal to setup your account before you can upload for the first time (jonathan.thiry@nateko.lu.se).

In the upload window:
FILE UPLOAD
Submitter ID: Select your station (if any selection)
Type: Data
Previous Version: If this file replaces an existing file – as when updating a yearly file with an additional quarter of  year or adding a year to a longer time series.
Copy paste the Hashsum of the file you are replacing
Pre-existing DOI – Not applicable

 
DATA
Data level: Select data level
Data type: Make sure to select the correct time resolution and thematic programme.
Number of rows: This is the number of rows of with real data – all lines below the header after #### in the data file.
 
ACQUISITION
Station: Select the station the location belongs to
Location/Ecosystem: Make sure to select the right combination. There can be several ecosystem for each location
From UTC/To UTC: Not needed, read from file
Sampling height: Not applicable – heights of instruments to be included in the header of the file
Instrument: Not applicable – info on instruments to be included in the header of the file
 
ADD PRODUCTION METADATA
(small field above Upload button)
Possibility to add information on the specific dataset – not mandatory.
 
Double check and thereafter
Click Upload!
 
The dataset will appear on the portal in few minutes - be patient.


 
9. Documents  <back to top>
 
- Inventory of all meteorological parameters within SITES
- Metadata of Data types, Locations, Stations (in separate sheets)
- Data file header template for meteorological data
- Example of meteorological data file
- Compiled Vocabulary
- Data level definitions