1. Do I need to register before downloading MyOcean products?
2. How to download MyOcean products ?
3. Can I use command lines to download MyOcean product(s)?
4. How to write and run the script to download MyOcean products?
5. Is a download limited in volume terms?
6. How to download a large amount of data?
7. Can you give a few examples of command lines to download?
8. Do I need to ask for access codes each time I want to download a new product?
9. How far is MyOcean Information System (MIS) interoperable?
10. I am having trouble downloading data using Internet Explorer. What'swrong?
11. When working with Mozilla Firefox, I enter my username and password but the page refreshes without allowing access to my data. What is wrong?
12. What tool or library is useful to visualize and analyze netCDF data ?
13. What tool is useful to manipulate netCDF data ?

1. Do I need to register before downloading MyOcean products?

YES

 Please find below the registration steps:

1. The registration form can be found in the section "USER CORNER" and in the "SERVICES" section. This registration form is a part of the MyOcean agreement named "Service Level Agreement" (SLA).
2. Read the MyOcean service commitments and licence
3. Fill in, sign and send the Service Level Agreement on-line form.
4. If you have any problems with the on-line SLA, please use this PDF form
5. If you have any problems with the "SEND" button of the SLA (PDF form), save the form and send it by e-mail to servicedesk@myocean.eu.org.
6. You will receive a Login & password from the Service Desk within a maximum 3 working days. This means that the SLA is accepted.
7. You can download your products (see FAQ2)
8. If you have sent the form and not had a response within 3 working days, please contact servicedesk@myocean.eu.org.

 For general questions and further assistance please contact servicedesk@myocean.eu.org.

You are now registered to use MyOcean. The service desk has given you all necessary login/password information. You can now download your products.

2. How to download MyOcean products?

You first need  to register (see FAQ 1)

To download data through the web portal, launch a web browser and connect the MyOcean web site

Click on “Direct access to catalogue” on the top-right of the page:

Select a product (with the search mode or by browsing the full catalogue)

Click on “Data Access”

Click on “Go”, then you have to login with a login/passwd provided by the MyOcean service desk.

You’re redirected on the data access web page corresponding to the product.

The interface will propose you all the datasets available. Select the required dataset.

 

The next page allows you to use selections in your download request, then download the data or generate a command line:

1.    Select your geographical area
2.    Select your time window
3.    Select the variable you want to download
4.    Click on “Download” button
 

Depending on the type of download service used, the web interface can propose the selection of the time window only. But the download is similar:

1.    Select the time range
2.    Click on “Download”
 

For some products, FTP is available on request. Please contact the Service Desk.



 

3. CAN I USE COMMAND LINES TO DOWNLOAD MyOcean PRODUCT(S)?

 YES and the use of command line can be particularly useful:

•    if you want to download MyOcean data regularly and automatically (without having to connect manually to the web portal each time),
•    if you want to download a large amount of data

Command lines use a python script and can be launched under different environments:  Windows (possibly through Cygwin), Linux and MacOS.

FAQ 4 will help you writing command lines, with the relevant parameters corresponding to your needs.
If you have to download a large amount of data, please also read FAQ6.

4 . HOW TO WRITE AND RUN THE SCRIPT TO DOWNLOAD MyOcean PRODUCTS?

A prerequisite to use command lines is that the target product is available through the MyOcean data access portal.
The best way to know if a product is available through the MyOcean data access portal is to click on “Go” from the Data access page on the catalogue. If you see the logging page (see below), it means it can be accessed through scripts.

Figure 1: logging for data access

Generating a template command line through the web portal
 

The MyOcean web portal allows you to generate a template command line using the python script for the download of the dataset you are interested in.

This command line integrates the extraction parameters you might want to specify.
 

To first create your command line, you have to navigate as if you wanted to download the data through the web portal.

After having selected the product specification in the catalogue, click on “Go” in the “Data access” and log in (as shown above in FAQ 2).

The interface will propose you all the datasets available. Select the required dataset.

Figure 2: select the dataset

 

The next page allows you to use selections in your download request, then download the data or generate a command line:

1. Select your geographical area
2. Select your time window
3. Select the variable you want to download

Click on “Script” button in order to generate a command line corresponding to your request :

Figure 3: select constraints, and generate a command line
 

Depending on the type of download service used, the web interface can propose the selection of the time window only.

But the generation of the command line is similar:
 

1. Select the time range
2. Click on “script”
    

Figure 4: select constraints, and generate a command line
 

The generated command line is opened in a new tab of your Web Browser.

Figure 5: command line generated through the MyOcean web portal
 

With this web page, you just have to copy and paste the command line (after downloading the motu client python and python itself if it’s the first time you’re using the script) and change the very local parameters: your password, the output file name, the output directory, and eventually the proxy parameters if you have to pass a proxy to access internet.
 

Note : if the dataset has a hourly temporal resolution, you may have to define the temporal parameter in the command line with quotes.

Example : python motu-client.py -u your_login -p your_password -m http://data.ncof.co.uk/mis-gateway-servlet/Motu -s http://purl.org/myocean/ontology/service/database#SST_GLO_SST_L4_NRT_OBSERVATIONS_010_001_a -d METOFFICE-GLO-SST-L4-NRT-OBS-SST -x -179.97500610351562 -X 20 -y 0 -Y 70 -t “2011-06-23 12:00:00” -T “2011-06-23 12:00:00” -v analysed_sst -o your_output_directory -f your_output_file_name 

How to run the command line
 

As it is explained in the above web page, if it is the first time you are using download scripts, you might need to download and install python, and the client script called “motu-client”.
 

You’ll find python here http://www.python.org/getit/. Please download python 2.7.2.

You’ll find the last version of the script “motu-client” here http://sourceforge.net/projects/cls-motu/files/client/motu-client-python/.

The last version is highly recommended.

Running command line on windows
 

The operating system Microsoft Windows is not the most convenient operating system to run command lines. But it’s of course feasible to do so, by using DOS command, or installing Cygwin if you are familiar with unix environment.
 

Click on “Start” => “Accessories” => “Command line”
 

Go in the directory where you saved the motu client script.
cd C:your_current_directory
path_to_your_python_binarypython motu-client-pythonmotu-client.py -u XXX -p XXX -m http://misgw-qo-externe.cls.fr/misgw-qo-servlet/Motu -s http://purl.org/myocean/ontology/service/database#misgw-qo-thr -d LR_MOD -x -180.0 -X 180 -y -89 -Y 90 -t 2011-01-02 -T 2011-01-02 -z 0 -Z 0 -v salin-ity temperature -o . -f test.nc

Running command line on Linux or MacOS
 

These operating systems are well adapted to run command line. Just open a terminal, go to the directory where you saved the motu client script and run the command.

 

5. IS A DOWNLOAD LIMITED IN VOLUME TERMS?

YES

At the moment, the maximum amount of data you can download is 1 Gigabyte. If your request is beyond this size, an error message will be immediately displayed (" The size of the data to download - ... Megabytes- exceeds the maximum allowed - 1024 Megabytes")and you will have to split your data accordingly. You can then encounter a "proxy error 502". This error message also appears in the event the request handling is lasting too long a time. In this case you will also have to split up your data requests.
 

6 . HOW TO DOWNLOAD A LARGE AMOUNT OF DATA?

Downloading a very large amount of data, especially for downloading large time coverage (like 10 days or 20 years) is possible through the MyOcean download service.

The way to do it is to split your download requests in some simple download requests.

One download request corresponds to one file, so downloading a very large amount of data (more than 1Gb) is not allowed.

Anyway it’s not recommended to manipulate files larger than 1Gb, for usability purpose.

The simplest way can be to perform a loop on dates.

For example, below you can find a loop coded in a simple shell script.

# Definition of the dates
DATES=(2011-01-01 2011-01-02 2011-01-03 2011-01-04 2011-01-05 2011-01-06)

# Loop on the dates
for (( i = 0 ; i < ${#DATES[@]} ; i++ ))
do
      #Download request (should be adapted)
        python motu-client-python/motu-client.py -u XXX -p XXX -m http://misgw-qo-externe.cls.fr/misgw-qo-servlet/Motu -s http://purl.org/myocean/ontology/service/database#misgw-qo-thr -d LR_MOD -x -180.0 -X 180 -y -89 -Y 90 -t ${DATES[$i]} -T ${DATES[$i]} -z 0 -Z 0 -v salinity temperature -o ./ -f test.nc
done
 

The array DATES shall be adapted to each situation (time coverage of the dataset and your own need).

Of course, depending on the way you want to code the loop (python, perl, java…), you’ll have to adapt the code.
 

The line in blue is a command line to download one time step of data, you can find explanation on how to write it in the beginning of the document.

 

7 . CAN YOU GIVE A FEW EXAMPLES OF COMMAND LINES TO DOWNLOAD?

Ocean Colour product

python motu-client-python/motu-client.py -u XXX -p XXX -m http://myocean.artov.isac.cnr.it/mis-gateway-servlet/Motu -s http://purl.org/myocean/ontology/service/database/OCEANCOLOUR_MED_CHL_SEAWIFS_L3_RAN_OBSERVATIONS_009_025_a-TDS -d OCEANCOLOUR_MED_CHL_SEAWIFS_L3_RAN_OBSERVATIONS_009_025_a -o . -t 2009-12-04 -T 2009-12-04 --out-name test_color.nc

python motu-client-python/motu-client.py -u XXX -p XXX -m http://myocean.acri.fr/mis-gateway-servlet/Motu? -s http://purl.org/myocean/ontology/service/database/OCEANCOLOUR_ARC_KD490_MERIS_MODIS_L _KD490_MERIS_MODIS_L3_L4_NRT_OBSERVATIONS_009_010_m -d dataset-oc-acri-arctic-1km-kd490-daily-nrt -o ./ -t 2010-09-03 -T 2010-09-03 -y 0 -Y 68 -x 0 -X 100 -z 0 -Z 50 -v volume_attenuation_coefficient_of_downwelling_radiative_flux_in_sea_water --proxy-server http://proxyURL:port --out-name test_color.nc
 

Global analysis and forecast product
 

python motu-client-python/motu-client.py -u XXX -p XXX -m http://atoll.mercator-ocean.fr/mfcglo-mercator-gateway-servlet/Motu? -s http://purl.org/myocean/ontology/service/database#GLOBAL_ANALYSIS_FORECAST_PHYS_001_001_b-TDS -d dataset-psy2v4-pgs-nat-myocean-bestestimate -o ./ -t 2010-01-21 -T 2010-01-22 -y 0 -Y 68 -x 0 -X 100 -z 0 -Z 50 -v sea_water_salinity --proxy-server http://proxyURL:port --out-name test_mercator.nc

python motu-client-python/motu-client.py -u XXX -p XXX -m http://atoll.cls.fr/mfcglo-armor-gateway-servlet/Motu? -s http://purl.org/myocean/ontology/service/database#GLOBAL_REANALYSIS_PHYS_001_003_b-TDS -d dataset-armor-3d-ran-v1-myocean -o ./ -t 2009-11-20 -T 2009-11-22 -y -80 -Y 80 -x -180 -X 180 -z 0 -Z 0 -v sea_water_salinity --proxy-server http://proxyURL:port --out-name test_armor.nc
 

Mediterranean Sea product
 

An example of shell script with a loop

DATES=(2005-01-01 2005-01-02) #etc...
# Loop on the dates
for (( i = 0 ; i < ${#DATES[@]} ; i++ ))
 

do
 

                  #Download request
       

python motu-client-python/motu-client.py -u XXX -p XXX -m

http://gnoodap.bo.ingv.it:4080/mis-gateway-servlet/Motu? -s

http://purl.org/myocean/ontology/service/database#MEDSEA_ANALYSIS_FORECAST_PHYS_006_001_a-TDS -d myov01-med-ingv-sal-analysis -o ./ -z 1.47 -Z 7.95 -t ${DATES[$i]} -T ${DATES[$i]} --out-name ingv_analysis_sal_${DATES[$i]}.nc

done
 

Sea Level product
 

Please note that concerning the Sea Level products, you receive a zip file containing some netCDF files, and not the netCDF file directly.

python motu-client-python/motu-client.py -u XXX –p XXX -m http://atoll.cls.fr/sltac-gateway-servlet/Motu? -s http://purl.org/myocean/ontology/service/database#cls-toulouse-fr-sltac-motu-rest -d http://purl.org/myocean/ontology/product/database#dataset-duacs-nrt-global-j1-sla-l3 -o ./ -t 2010-11-02 -T 2011-10-03 --proxy-server http://proxyURL:port --out-name test_sltac.zip

 

8. Do I need to ask for access codes each time I want to download a new product?

NO

With your login/password delivered by the Service Desk when you first registered you can access most of the product of the catalogue.

However, still some products are not available for download via a centralized access. When choosing one of them, you will be redirected to the service desk (by email) to ask the access codes.


 

9. How far is MyOcean Information System (MIS) interoperable?

MyOcean Information System (MIS) was built with respect state-of-the-art recommendations in the field of geospatial interoperability.

- ISO 19115 and ISO 19119 are used for products and service general description

- ISO 19139 is then applied as official XML implementation for these metadata. One MyOcean-ISO 19139 profile was defined; how ISO rules were applied on MyOcean metadata is fully described here (lien vers MYO-MIS-TN-INT-META-V1-1.odt)

- CSW-ISO is the OGC protocol used to request the metadata catalogue. If one wants to request MyOcean Information System using “direct” CSW formalism into a web browser and harvest metadata, the complete procedure is detailed here .

- As ISO do not encompass all operational oceanography features, several vocabularies were needed (such as CF variables) or defined. RDF, and especially the W3C standard SKOS was used to capture those XML metadata extensions.

- Each element within MyOcean Information System is uniquely identified via one URI. To play the role of URIs, a persistent uniform resource locator (PURL) was allocated to product specifications, description of product specifications (ISO 19139 XML), dataset, , services, description of services (ISO 19139 XML files), and even vocabulary terms. Especially, all MyOcean distributed components (dataset access installed among Dissemination Units) refer to such URIs to ensure the dialogue between real data and central information system.

- Combining RDF with URIs allows one user to request MyOcean Information System using SPARQL language. This is an additional functionality, that focuses on relationships between MyOcean Information System elements, and the vocabulary terms used to identify them. More information and example of SPARQL requests are detailed in the "MIS / Product and access Services Metadata" document available here: MIS/ Product and Access services METADATA .

Note also that regarding  INSPIRE, MyOcean metadata profile is fully compliant with INSPIRE metadata profile . It is actually based on INSPIRE skeleton with additional ISO and SKOS elements. Due to the project timeline, no other INSPIRE Implementing Rules (e.g. for network services) was implemented yet, because they were not available at the conception phase of our project.

10. I am having trouble downloading data using Internet Explorer. What's wrong?

Unfortunately the default security settings on Internet Explorer do not allow downloading from third party cookies, which is needed to access MyOcean data through the web portal. 

In this situation a small icon appears on the status bar to inform the user that a cookie has been rejected.

Decreasing the security settings will prevent this issue, to do this :

1. Select ‘Tools’ from the top menu bar

2. Select ‘Internet Options’

3. Select the ‘Security’ tab

4. Move the slider all the way down to ‘accept all cookies’.

In some cases this may not be possible, for example if internet security settings are centrally controlled.  If this is the case it is possible to access data through other browsers (such as Mozilla Firefox, or Google Chrome, both free to download from the internet) usually without having to change the security settings.  We recommend trying to use a different web browser if you experience issues with Internet Explorer.

Please be aware that the MyOcean team are working on better solutions for this issue.

11. When working with Mozilla Firefox, I enter my username and password but the page refreshes without allowing access to my data. What is wrong?

This is due to your computers security settings : the direct download service requires you to accept cookies from a third party website (http://coremis-cas-int.cls.fr/cas/). 

The easiest way to enable this is to access your browsers security settings and select ‘accept third party cookies’.

12. WHAT TOOL or LIBRARY IS USEFUL TO VISUALIZE and ANALYSE NetCDF DATA ?

(non exhaustive list)

Quick visualization (easy to install)

-          Ncview (http://meteora.ucsd.edu/~pierce/ncview_home_page.html)

Simple visualization and analysis (easy to install on unix/linux platforms, possible on windows)

-          Ferret (http://ferret.wrc.noaa.gov/Ferret/)

Advanced visualization and analysis (you need to develop your own program)

-          Matplotlib, python library (http://matplotlib.sourceforge.net/)

13. WHAT TOOL IS USEFUL TO MANIPULATE NetCDF DATA ?

(non exhaustive list)

-          Details about the NetCDF data format can be found at: http://www.unidata.ucar.edu/software/netcdf/.

-          From this website, the NetCDF software package, including libraries for programming languages such as Fortran and C++, can be freely downloaded, please see: http://www.unidata.ucar.edu/software/netcdf/docs/faq.html#howtoget. This package is the best way for performing detailed processing of NetCDF files, including conversion to ASCII or other file formats.

-          A powerful set of tools for working with NetCDF files are the NetCDF operators (NCO), which can be freely obtained from http://nco.sourceforge.net/