SOS Server

Author:

Jeff McKenna

Contact:

jmckenna at gatewaygeomatics.com

Last Updated:

2021-03-27

Introduction

SOS (Sensor Observation Service), currently an OGC discussion paper, is part of of the OGC’s SensorWeb Enablement (SWE) group of specifications. These specifications describe how applications and services will be able to access sensors of all types over the Web. Specifically, SOS provides an API for managing deployed sensors and retrieving sensor data.

SOS support is available in MapServer 4.10.0 or more recent. Note that no client tools currently exist in MapServer for SOS; you can try QGIS desktop and its plugins, or try using a JavaScript library such as OpenLayers as a client.

SOS support was implemented in MapServer to the guidelines of MapServer MS RFC 13: Support of Sensor Observation Service in MapServer.

This document assumes that you are already familiar with certain aspects of MapServer:

  • MapServer application development and setting up .map files.

Relevant Definitions

The following is taken from the SOS 1.0.0 specification:

Observation

An observation is an event with a result which has a value describing some phenomenon.

Observation Offering

An observation offering is a logical grouping of observations offered by a service that are related in some way.

Observed Value

A value describing a natural phenomenon, which may use one of a variety of scales including nominal, ordinal, ratio and interval.

Sensor

An entity capable of observing a phenomenon and returning an observed value. A sensor can be an instrument or a living organism (e.g. a person).

Setting Up an SOS Server Using MapServer

Install the Required Software

SOS requests are handled by the «mapserv» CGI program. The first step is to check that your mapserv executable includes SOS support. One way to verify this is to use the «-v» command-line switch and look for «SUPPORTS=SOS_SERVER».

Example 1. On Unix:

$ ./mapserv -v
MapServer version 4.9 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG OUTPUT=WBMP
OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=FREETYPE SUPPORTS=WMS_SERVER
SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT
SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER SUPPORTS=THREADS INPUT=JPEG
INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE DEBUG=MSDEBUG

Example 2. On Windows:

C:\MS4W\Apache\cgi-bin> mapserv -v
MapServer version 4.9 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG OUTPUT=WBMP
OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=FREETYPE SUPPORTS=WMS_SERVER
SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT
SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER SUPPORTS=THREADS INPUT=JPEG
INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE DEBUG=MSDEBUG

Nota

Windows users can install MS4W which supports SOS server.

If you don’t have SOS support in your MapServer build, then you must compile MapServer with the following in mind:

  • flag -DUSE_SOS_SVR is required

  • requires either -DUSE_WMS_SVR or -DUSE_WFS_SVR flags to be enabled

  • requires libxml2 and proj libraries

  • requires ICONV support (-DUSE_ICONV) on Windows

For more help with MapServer compilation see the appropriate HowTo: Unix / Windows

Configure a Mapfile For SOS

Each instance of SOS server that you setup needs to have its own mapfile. It is just a regular MapServer mapfile in which some parameters and some metadata entries are mandatory. Most of the metadata is required in order to produce a valid GetCapabilities output.

Here is the list of parameters and metadata items that usually optional with MapServer, but are required (or strongly recommended) for a SOS configuration:

MAP level:

  • Map NAME

  • Map PROJECTION

  • Map Metadata (in the WEB Object):

    • sos_title

    • sos_onlineresource

    • sos_srs

    • sos_enable_request

    • see the Reference Section of this document for a full list of metadata and descriptions

LAYER level:

  • Layer NAME

  • Layer PROJECTION

  • Layer METADATA

    • sos_offering_id

    • sos_observedproperty_id

    • sos_observedproperty_id

    • sos_describesensor_url

    • see the Reference Section of this document for a full list of metadata and descriptions

Onlineresource URL

The sos_onlineresource metadata is set in the map’s web object metadata and specifies the URL that should be used to access your server. This is required for the GetCapabilities output. If sos_onlineresource is not provided then MapServer will try to provide a default one using the script name and hostname, but you shouldn’t count on that too much. It is strongly recommended that you provide the sos_onlineresource metadata.

Here is a valid online resource URL:

http://my.host.com/cgi-bin/mapserv?map=mysos.map&

By creating a wrapper script on the server it is possible to hide the «map=» parameter from the URL and then your server’s online resource URL could be something like:

http://my.host.com/cgi-bin/mapserv?

This is covered in more detail in the «More About the Online Resource URL» section of the WMS Server document.

Nota

It is strongly recommended to review the security steps for the MAP= call to the MapServer executable, by setting MS_MAP_PATTERN or MS_MAP_NO_PATH or hiding the MAP= parameter on public servers, as recommended in the document Limit Mapfile Access. All possible environment variables to secure your server are listed in Environment Variables.

Example SOS Server Mapfile

The following is an example of a bare minimum SOS Server mapfile. Note the comments for the required parameters.

MAP
  NAME "SOS_DEMO"
  STATUS ON
  SIZE 300 300
  EXTENT -66 44 -62 45
  UNITS METERS
  SHAPEPATH "./data/"
  IMAGECOLOR 255 255 0
  SYMBOLSET "./etc/symbols.sym"

  IMAGETYPE png

  WEB
    IMAGEPATH "/ms4w/tmp/ms_tmp/"
    IMAGEURL "/ms_tmp/"

    METADATA
      "sos_onlineresource" "http://127.0.0.1/mapserv?map=/sos/sos_test.map" ## REQUIRED
      "sos_title"          "My SOS Demo Server" ## Recommended
      "sos_srs"            "EPSG:4326" ## REQUIRED
      "sos_enable_request" "*" ## REQUIRED
    END
  END

  PROJECTION
    "init=epsg:4326"
  END

  LAYER
    NAME "test_sos_layer"
    METADATA
      "sos_procedure"  "NS01EE0014" ## REQUIRED
      "sos_offering_id" "WQ1289" ## REQUIRED
      "sos_observedproperty_id" "Water Quality" ## REQUIRED
      "sos_describesensor_url" "http://some/url/NS01EE0014.xml" ## REQUIRED
    END
    TYPE POINT
    STATUS ON
    DATA "sos_test"

    PROJECTION
      "init=epsg:4326"
    END

    CLASS
      NAME "water quality"
      STYLE
        COLOR 255 0 0
        SYMBOL "circle"
        SIZE 8
      END
    END
  END

END #map

Test Your SOS Server

GetCapabilities Request

The GetCapabilities request allows the clients to retrieve service metadata about a specific service instance. For an SOS service, it allows to identify such things as offerings and observed property available, as well as information on sensors that are used.

Using a web browser, access your server’s online resource URL to which you add the parameters «SERVICE=SOS&REQUEST=GetCapabilities» to the end, e.g.

http://my.host.com/cgi-bin/mapserv?MAP=mysos.map&SERVICE=SOS&REQUEST=GetCapabilities

If everything went well, you should have a complete XML capabilities document. Search it for the word «WARNING»… MapServer inserts XML comments starting with «<!–WARNING: « in the XML output if it detects missing mapfile parameters or metadata items. If you notice any warning in your XML output then you have to fix all of them before you can try your server with an SOS client, otherwise things are likely not going to work.

Nota

The SERVICE parameter is required for all SOS requests.

GetObservation Request

The GetObservation request is designed to query sensor systems to retrieve observation data in the form defined in the Observation and Measurement specification (O&M), and more information on this O&M spec can be found at https://www.ogc.org/standards/om. Upon receiving a GetObservation request, a SOS shall either satisfy the request or return an exception report.

The following is a list of the possible parameters for a GetObservation request:

request: (Required) value must be «GetObservation».

service: (Required) value must be «SOS».

version: (Required) value must be «1.0.0».

offering: (Required) The Offering identified in the capabilities document.

observedProperty: (Required) The property identified in the

capabilities document.

responseFormat: (Required) The format / encoding to be returned

by the response.

eventTime (Optional) Specifies the time period for which

observations are requested.

procedure: (Optional) The procedure specifies the sensor system

used. In this implementation, the procedure is equivalent to be the sensor id that will be used when doing a DescribeSensor request.

featureOfInterest: (Optional) In this implementation, this will

be represented by a gml envelope defining the lower and upper corners.

Result: (Optional) The Result parameter provides a place to put

OGC filter expressions based on property values.

resultModel: (Optional) Identifier of the result model to be

used for the requested data. The resultModel values supported by a SOS server are listed in the contents section of the service metadata (GetCapabilities). MapServer currently supports om:Observation and om:Measurement. om:Measurement provides a flat model of the geometry and attributes, similar to WFS GetFeature output. om:Observations provides a more compact definition which includes an XML header of the field names and definitions, followed by a «DataBlock» of delimited records (default is CSV delimited output). The default output is om:Measurement.

srsName: (Optional) srs (EPSG code) of the output response.

Here are some valid examples:

Example 1:

http://127.0.0.1/cgi-bin/mapserv.exe?map=D:/ms4w/apps/sos/sos_test.map&
Request=GetObservation&service=SOS&Offering=WQ1289&
observedproperty=Water Quality&version=1.0.0&
responseFormat=text/xml; subtype="om/1.0.0"

Example 2:

http://127.0.0.1/cgi-bin/mapserv.exe?map=D:/ms4w/apps/sos/sos_test.map&
Request=GetObservation&service=SOS&Offering=WQ1289&
observedproperty=Water Quality&eventtime=<ogc:TM_Equals><gml:TimePeriod>
<gml:beginPosition>1991-05-01</gml:beginPosition><gml:endPosition>
1993-02-02</gml:endPosition></gml:TimePeriod></ogc:TM_Equals>
&result=<Filter><Or><PropertyIsEqualTo><PropertyName>COLOUR
</PropertyName><Literal>180</Literal></PropertyIsEqualTo>
<PropertyIsEqualTo><PropertyName>COLOUR</PropertyName><Literal>200
</Literal></PropertyIsEqualTo></or></Filter>&version=1.0.0
&responseFormat=text/xml; subtype="om/1.0.0"

Example 3:

http://127.0.0.1/cgi-bin/mapserv.exe?map=D:/ms4w/apps/sos/sos_test.map&
Request=GetObservation&service=SOS&Offering=WQ1289&
observedproperty=Water Quality&featureofinterest=<gml:Envelope>
<gml:lowerCorner srsName='EPSG:4326'>-66 43</gml:lowerCorner>
<gml:upperCorner srsName='EPSG:4326'>-64 45</gml:upperCorner>
</gml:Envelope>&version=1.0.0&
responseFormat=text/xml; subtype="om/1.0.0"

Example 4:

http://127.0.0.1/cgi-bin/mapserv.exe?map=D:/ms4w/apps/sos/sos_test.map&
Request=GetObservation&service=SOS&Offering=WQ1289&
observedproperty=Water Quality&version=1.0.0&
responseFormat=text/xml; subtype="om/1.0.0"&resultModel=om:Observation

DescribeSensor Request

The DescribeSensor request gives the client the ability to retrieve the characteristics of a particular sensor and return the information in a SensorML xml document. In this implementation, MapServer does not generate the SensorML document but only redirect the request to an existing SensorML document.

The following is a list of the possible parameters for a DescribeSensor request:

request: (Required) value must be «DescribeSensor»

service: (Required) value must be «SOS».

version: (Required) value must be «1.0.0».

procedure: (Required) This is the sensor id, which was specified

in the «sos_procedure» metadata.

outputFormat: (Required) The format encoding to be returned by

the response.

Here is a valid example:

http://127.0.0.1/cgi-bin/mapserv.exe?map=D:/ms4w/apps/sos/sos_test.map&
Request=DescribeSensor&procedure=urn:ogc:def:procedure:NS01EE0014&
service=SOS&version=1.0.0&outputFormat=text/xml; subtype="sensorML/1.0.0"

Limitations / TODO

1. Have MapServer generate the SensorML document, instead of redirecting the request to an existing SensorML document.

Reference Section

The following metadata are available in the setup of the SOS Server mapfile:

Nota

Each of the metadata below can also be referred to as “ows_*” instead of “sos_*”. MapServer tries the “sos_*” metadata first, and if not found it tries the corresponding “ows_*” name. Using this reduces the amount of duplication in mapfiles that support multiple OGC interfaces since «ows_*» metadata can be used almost everywhere for common metadata items shared by multiple OGC interfaces.

Web Object Metadata

ows_allowed_ip_list (or sos_allowed_ip_list)

  • Description: (Optional) A list of IP addresses that will be allowed access to the service.

    Example:

    METADATA
      "ows_allowed_ip_list" "123.45.67.89 11.22.33.44"
    END
    

ows_denied_ip_list (or sos_denied_ip_list)

  • Description: (Optional) A list of IP addresses that will be denied access to the service.

    Example:

    METADATA
      "ows_denied_ip_list" "123.45.67.89 11.22.33.44"
    END
    

ows_language

  • Description: (Optional) Descriptive narrative for more information about the server. Identifier of the language used by all included exception text values. These language identifiers shall be as specified in IETF RFC 1766. When this attribute is omitted, the language used is not identified. Examples: «en-CA», «fr-CA», «en-US». Default is «en-US».

ows_schemas_location

  • Description: (Optional) (Note the name ows_schemas_location and not sos/_… this is because all OGC Web Services (OWS) use the same metadata) Root of the web tree where the family of OGC SOS XMLSchema files are located. This must be a valid URL where the actual .xsd files are located if you want your SOS output to validate in a validating XML parser. Default is http://schemas.opengis.net/sos/.

ows_updatesequence

  • Description: (Optional) The updateSequence parameter can be used for maintaining the consistency of a client cache of the contents of a service metadata document. The parameter value can be an integer, a timestamp in [ISO 8601:2000] format, or any other number or string.

sos_abstract

  • Description: (Optional) Descriptive narrative for more information about the server.

sos_accessconstraints

  • Description: (Optional) Text describing any access constraints imposed by the service provider on the SOS or data retrieved from this service.

sos_addresstype, sos_address, sos_city, sos_country, sos_postcode, sos_stateorprovince

  • Description: Optional contact address information. If provided then all six metadata items are required.

sos_contactelectronicmailaddress

  • Description: Optional contact Email address.

sos_contactfacsimiletelephone

  • Description: Optional contact facsimile telephone number.

sos_contactinstructions

  • Description: (Optional) Supplemental instructions on how or when to contact the individual or organization.

sos_contactorganization, sos_contactperson, sos_contactposition

  • Description: Optional contact information. If provided then all three metadata items are required.

sos_contactvoicetelephone

  • Description: Optional contact voice telephone number.

sos_enable_request (or ows_enable_request)

  • Description: Space separated list of requests to enable. The default is none. The following requests can be enabled: GetCapabilities, GetObservation and DescribeSensor. A «!» in front of a request will disable the request. «*» enables all requests.

  • Examples:

    To enable only GetCapabilities and GetObservation:

    "sos_enable_request" "GetCapabilities GetObservation"
    

    To enable all requests except GetCapabilities

    "sos_enable_request" "* !GetCapabilities"
    

sos_encoding_blockSeparator

  • Description: (Optional) For GetObservation requests using resultModel=om:Observation (SWE DataBlock encoding). Record separator to be used. Default is “\n”

sos_encoding_tokenSeparator

  • Description: (Optional) For GetObservation requests using resultModel=om:Observation (SWE DataBlock encoding). Token (field) separator to be used. Default is “,”

sos_fees

  • Description: (Optional) Fees information. Use the reserved word «none» if there are no fees.

sos_hoursofservice

  • Description: (Optional) Time period (including time zone) when individuals can contact the organization or individual.

sos_keywordlist

  • Description: (Optional) A comma-separated list of keywords or keyword phrases to help catalog searching.

sos_maxfeatures

  • Description: (Optional) The number of elements to be returned by the SOS server. If the not set all observations are returned

sos_onlineresource

  • Description: (Required) The URL that will be used to access this OGC server. This value is used in the GetCapabilities response.

  • See the section «Onlineresource URL» above for more information.

sos_role

  • Description: (Optional) Function performed by the responsible party. Possible values of this Role shall include the values and the meanings listed in Subclause B.5.5 of ISO 19115:2003.

sos_service_onlineresource

  • Description: (Optional) Top-level onlineresource URL.

sos_srs

  • Description: (Required) Contains a list of EPSG projection codes that should be advertized as being available for all layers in this server. The value can contain one or more EPSG:<code> pairs separated by spaces (e.g. «EPSG:4269 EPSG:4326») This value should be upper case (EPSG:3978…..not epsg:3978) to avoid problems with case sensitive platforms.

sos_title

  • Description: (Recommended) A human-readable name for this Layer.

Layer Object Metadata

ows_allowed_ip_list

Same as ows_allowed_ip_list in the Web Object.

ows_denied_ip_list

Same as ows_denied_ip_list in the Web Object.

sos_describesensor_url

  • Description: (Required) This metadata item is only a temporary measure until the describe sensor is generated from MapServer. Right now when a DescribeSensor request is sent with a procedure (sensorid), it will redirect it to the url defined by this metadata item.

  • In MapServer 5.0, it is possible to use variable substitution on the url. For example «sos_describesensor_url» "http://foo/foo?mysensor=%procedure%"will substitute the %procedure% in the metadata with the procedure value coming from the request.

    "sos_describesensor_url" "http://some/url/NS01EE0014.xml"
    

sos_enable_request (or ows_enable_request)

  • Description: Space separated list of requests to enable. The default is none. The following requests can be enabled: GetCapabilities, GetObservation and DescribeSensor. A «!» in front of a request will disable the request. «*» enables all requests.

  • Examples:

    To enable only GetCapabilities and GetObservation:

    "sos_enable_request" "GetCapabilities GetObservation"
    

    To enable all requests except GetCapabilities

    "sos_enable_request" "* !GetCapabilities"
    

sos_[item name]_alias

  • Description: (Optional) An alias for an attribute’s name that will be returned when executing a GetObservation request.

sos_[item name]_definition

  • Description: (Optional) An associated definition (usually a URN) for a component, that will be returned when executing a GetObservation request. Default is "urn:ogc:object:definition"

sos_[item name]_uom

  • Description: (Optional) An associated unit of measure URN) for a component, that will be returned when executing a GetObservation request. Default is "urn:ogc:object:uom"

sos_observedproperty_authority

  • Description: (Optional) An associated authority for a given component of an observed property

sos_observedproperty_id

  • Description: (Required) ID of observed property, possibly in number format.

sos_observedproperty_name

  • Description: (Optional) Name of observed property, possibly in string format.

sos_observedproperty_version

  • Description: (Optional) An associated version for a given component of an observed property

sos_offering_description

  • Description: (Optional) Description of offering.

sos_offering_extent

  • Description: (Optional) Spatial extents of offering, in minx, miny, maxx, maxy format:

    "sos_offering_extent" "-66, 43, -62, 45"
    

    The logic for the bounding box returned as part of the offering is the following:

    • note that it is a mandatory element that needs an espg code and lower/upper corner coordinates

    • looks for the espg parameter in the first layer of the offering (this could be an ows/sos_srs or a projection object with the epsg code (mandatory)

    • looks for sos_offering_extent. If the metadata is not available, the extents of all layers in the offering will be used to compute it.

    Here is an example result from a GetCapabilities request:

    <gml:boundedBy>
      <gml:Envelope>
        <gml:lowerCorner srsName="EPSG:4326">-66 43</gml:lowerCorner>
        <gml:upperCorner srsName="EPSG:4326">-62 45</gml:upperCorner>
      </gml:Envelope>
    </gml:boundedBy>
    

sos_offering_id

  • Description: (Required) ID of offering, possibly in number format.

sos_offering_intendedapplication

  • Description: (Optional) The intended category of use for this offering.

sos_offering_name

  • Description: (Optional) Name of offering, possibly in string format.

sos_offering_timeextent

  • Description: (Optional) Time extent of offering, in the format of «begin/end». Here is an example:

    "sos_offering_timeextent" "1990/2006"
    

    If end is not specified it will be set to now. Here is an example result from a GetCapabilities request:

    <sos:eventTime>
      <gml:TimePeriod>
         <gml:beginPosition>1990</gml:beginPosition>
         <gml:endPosition>2006</gml:endPosition>
      </gml:TimePeriod>
    </sos:eventTime>
    

sos_procedure

  • Description: (Required) Normally a sensor unique id. One per layer:

    "sos_procedure"  "NS01EE0014"
    

    Nota

    sos_procedure can also be a list, separated by spaces, i.e.:

    "sos_procedure" "35 2147 604"
    

    All sos_procedure links from layers in the offerings will be outputed together, such as the following taken from a GetCapabilities response:

    <procedure xlink:href="urn:ogc:object:feature:Sensor:3eTI:csi-sensor-1"/>
    <procedure xlink:href="urn:ogc:object:feature:Sensor:3eTI:csi-sensor-2"/>
    

sos_procedure_item

  • Description: (Required if sos_procedure is not present): See section 5 for more details

    "sos_procedure_item"  "attribute_field_name"
    

sos_timeitem

  • Description: (Optional) Name of the time field. It will be used for queries when a GetObservation request is called with an EVENTTIME parameter. It is layer specific and should be set on all layers.

    "sos_timeitem" "TIME"
    

Layer Metadata API

If sos_metadataurl_href is not defined, MapServer will provide a link to the Layer Metadata API for the given layer. See the Layer Metadata API documentation for more information.

Use of sos_procedure and sos_procedure_item

In MapServer 5.0 SOS support has been upgraded to use a new metadata called sos_procedure_item. The value for sos_procedure_item is the field/attribute name containing the procedure values. The use of this metadata as well as the sos_procedure is described here per type of request (refer to https://github.com/MapServer/MapServer/issues/2050 for more description):

It should be noted that, for very large datasets defined only with sos_procedure_item, this may result in costly processing, because MapServer has to process attribute data. It is advised to setup and manage datasets accordingly if dealing with large observation collections.

GetCapabilities

  • if sos_procedure is defined, use it

  • if not look for sos_procedure_item : procedure values are extracted from the layer’s attribute specified by this metadata. Not that this can be time consuming for layers with a large number of features.

  • if none is defined return an exception

DescribeSensor

  • if sos_procedure is defined, use it

  • if not look for sos_procedure_item : procedure values are extracted from the layer’s attribute specified by this metadata

  • if none is defined return an exception

GetObservation

Both sos_procedure and sos_procedure_item can be define. Here are the cases:

  • case 1only sos_procedure is defined.
    • Use this metadata to match the layer with the procedure value sent in the request

    • When outputting the <member/procedure> output the value of the metadata

Nota

If more than one procedure is defined per LAYER object, output observations will have incorrect sos:procedure values, because there is no way to map procedures to observations. This is where sos_procedure_item should be used (i.e. when more than one procedure makes up a LAYER object).

  • case 2: only procedure_item is defined.
    • Use the sos_procedure_item and do a query on the layer to match the procedure with the layer.

    • When outputting the <member/procedure> use the procedure_item as a way to only output the attribute value corresponding to the feature.

  • case 3: both are defined.
    • check in sos_procedure to match the procedure with the layer.

    • When outputting the <member/procedure> use the procedure_item as a way to only output the attribute value corresponding to the feature.