MS RFC 36: Simplified template support for query output¶
- Date:
2007/10/23
- Author:
Steve Lime
- Contact:
steve.lime at dnr.state.mn.us
- Last Edited:
2007/10/23
- Status:
Development
Overview¶
Problem:
Currently a driver like GML isn’t available to the CGI as a means of presenting query results
The templating scheme (HEADER/TEMPLATE/FOOTER) for queries isn’t user friendly nor is it ammenable to multiple presentation formats. That is, one layer => one template set
Solution:
Use output format objects to define formats that can be used to output query results in addition to drawing images. For example:
OUTPUTFORMAT
NAME 'gml3'
DRIVER GML3
MIMETYPE 'text/xml; subtype=gml/3.2.1'
END
Might need to extend that object to discriminate between map rendering and query formatters but that can happen in mapdraw.c and mapserv.c too. That is, drivers are explicitly referenced in those places so if someone tries to draw a map with a GML3 driver it would throw an error.
Use the webObj QUERYFORMAT property to reference formats: «QUERYFORMAT gml3». Right now that property carries a mime-type but it could be used to reference a format too.
Also allow applicable modes (i.e. WFS, WMS, SOS), to utilize DRIVER/TEMPLATE type formats (i.e. advertise in GetCapabilities responses, support through API [e.g. request=GetFeature&outputFormat=text/xml; subtype=gml/3.2.1]), mapped from OUTPUTFORMAT/MIMETYPE. Presently the WCS driver requires the developer to explicitly define supported output formats, other services could do the same and could reference templated output.
Define a TEMPLATE driver. Basically this would just invoke the normal query templating scheme. For example:
OUTPUTFORMAT
NAME 'kml'
DRIVER TEMPLATE
MIMETYPE 'application/vnd.google-earth.kml+xml'
TEMPLATE 'myTemplate.kml'
END
OUTPUTFORMAT
NAME 'geojson'
DRIVER TEMPLATE
MIMETYPE 'application/json; subtype=geojson'
TEMPLATE 'myTemplate.js'
END
Note that in the above examples we reference a file, so I’m thinking of supporting a single template system for queries in addition to the current mechanism. To do this I’d propose 4 new template tags: [resultset], [feature], [join] (for one-to-many joins), and [include] (to support code sharing between templates). All but the include tag would be blocks. An example might be:
[include src="templates/header.html"]
[resultset name=lakes]
... old layer HEADER stuff goes here, if a layer has no results this block disappears...
[feature]
...repeat this block for each feature in the result set...
[join name=join1]
...repeat this block for each joined row...
[/join]
[/feature]
...old layer FOOTER stuff goes here...
[/resultset]
[resultset name=streams]
... old layer HEADER stuff goes here, if a layer has no results this block disappears...
[feature]
...repeat this block for each feature in the result set...
[/feature]
...old layer FOOTER stuff goes here...
[/resultset]
[include src="templates/footer.html"]
A specific GML3 example might be:
<?xml version="1.0" encoding="ISO-8859-1"?>
[resultset layer=mums]
<MapServerUserMeetings xmlns="http://localhost/ms_ogc_workshop" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:gml="http://www.opengis.net/gml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://localhost/ms_ogc_workshop ./mums.xsd">
<gml:description>This is a GML document which provides locations of all MapServer User Meeting that have taken place</gml:description>
<gml:name>MapServer User Meetings</gml:name>
<gml:boundedBy>
<gml:Envelope>
<gml:coordinates>-93.093055556,44.944444444 -75.7,45.4166667</gml:coordinates>
</gml:Envelope>
</gml:boundedBy>
[feature]
<gml:featureMember>
<Meeting>
<gml:description>[desc]</gml:description>
<gml:name>[name]</gml:name>
<gml:location>
<gml:Point srsName="http://www.opengis.net/gml/srs/epsg.xml#4326">
<gml:pos>[x] [y]</gml:pos>
</gml:Point>
</gml:location>
<year>[year]</year>
<venue>[venue]</venue>
<website>[url]</website>
</Meeting>
</gml:featureMember>
[/feature]
<dateCreated>2007-08-13T17:17:32Z</dateCreated>
</MapServerUserMeetings>
[resultset]
A GeoJSON example might be:
[resultset layer=foo] {
"type": "FeatureCollection",
"features": [
[feature trim=',']
{
"type": "Feature",
"id": "[id]",
"geometry": {
"type": "PointLineString",
"coordinates": [
{
"type": "Point",
"coordinates": [[x], [y]]
}
]
},
"properties": {
"description": "[description]",
"venue": "[venue]",
"year": "[year]"
}
},
[/feature]
]
}
[/resultset]
This would allow for relatively complex text files of any sort to be built from multiple layers. All the normal template tags would still be supported but those normally available for query results would only be valid inside a [feature]…[/feature]. These tags would work with existing system too but just wouldn’t be as useful as with the 1 template idea.
Informacja
It is often a problem to have trailing record separator characters after the final record. For example, in the JSON template above the trailing comma in the [feature] block causes problems with Internet Explorer. So I propose supporting a „trim” attribute that tells the template processor to remove that string from the end of the output for the last feature processed.
Informacja
A resultset could be applied to multiple layers so the name attribute will take a comma delimited list of layers. The order listed is the order they results will be presented. It’s possible that groups could be used as well but at this point that seems like a fairly rare use case.
Informacja
A resultset will also take a maxresults attribute so that the number of features processed can be limited.
Additional Mapfile Changes¶
By moving templates out of a layer we lose the ability mark layers as queryable. Dan proposed adding a QUERYABLE TRUE/FALSE option to layerObj’s. That could be put in place as part of this RFC, although it is not required. We could continue to leverage dummy template values. Adding it would require the normal changes to support a new keyword, and a small change to function in mapquery.c that tests to see if a layer is queryable. Basically a layer would be queryable if: 1) it has a template or 2) QUERYABLE is TRUE (default would be FALSE).
Documentation¶
Documentation detailing the new templated output capabilities will be added to the mapfile reference guide (OUTPUTFORMAT and WEB objects) and to the template reference guide (new [resultset], [feature], [join] and [include] tags).
Implementation¶
mapoutput.c: No changes necessary (I think), no need to define a default format, nor do I think we need to extend the outputFormatObj structure.
mapfile.c/maplexer.l: Allow changing webObj QUERYFORMAT from a URL. (todo: add support for setting a layer as queryable)
maptemplate.c: Add processor functions for the new tags. Update process line to recognize the [resultset] and [join] tags (the [feature] tag would only be valid within a [resultset] block. Write a new single template processing function similar to msReturnQuery() in that same source file, something like msReturnSingleTemplateQuery().
mapserv.c: Add code at the end of the query processing switch statement to look at the value of web->queryformat. If it references an existing output format by name then use the file the format points to with msReturnSingleTemplateQuery(), otherwise process as currently done.
Caveats: to simplify tag parsing (at least initially) I propose requiring that start and end tags exist on their own lines in the template file (is this a requirement for legend templates?). Depending on the legend template block parsing this requirement could be removed once some implementation work is done.
MapScript¶
No changes are anticipated in MapScript at this time although we may choose to expose templated output as an option at a later date.
Backwards Compatibility Issues¶
No other compatibility issues are anticipated. The current templating mechanism would continue to function. In the event the QUERYFORMAT does not reference an outputFormatObj the current system would kick in. In fact, the current system can use several of the new proposed tags, specifically [join] and [include] tags.
Bug ID¶
None assigned.
Voting History¶
None