Revise OGC service handling

[roomthily]

Merging issues into one comprehensible thing.

Decision made: fork OWSLib, add WMS 1.3.0 (at a minimum), revise parsers to for OGC to use this lib (ignore remainders in that I don't actually think there will be any). We will still need to handle the endpoint definitions! and I will put together a parser for WMS 1.1.1 and one of the WFS/WCS versions (supported and we have an example for) to start to scope that out.

Tasks:

• Revise to use the output of the identifier class (service, version) to initialize OGC structure(s). (see Revise WxS parsers for OWS metadata and some standardization #12, Add some handling for the standard/non-standard WCS response #21)
• Update OWSLib for WMS 1.3.0
• Update OWSLib for WMS 1.0.0 (legacy/dark data needs, we should find some).
• Update parser code for new OWSLib version.
• Complete the config and code for the endpoint definitions (see Revisit Endpoints and method definitions for OGC parsing #7).
• Check OWSLib for supported services and versions to see if it can be forked or partially reused here.
• Related, Check with OWSLib about why versions not supported? and if they should be supported? Pull request?

For OWSLib reference: these are the high priority (as of the last harvest, so grain of salt):

[roomthily]

OWSLib Version Support Notes

WMS GetCapabilities 1.3.0 & 1.0.0 💥
(that's too bad - it can't handle most of the current arcgis services - 1.3.0)
(see geopython/OWSLib#25 for feature request from 2012)

WMS GetCapabilities 1.1.1 👍
(just to demonstrate that it installed correctly, etc)

In [7]: wms = WebMapService('http://disc1.sci.gsfc.nasa.gov/daac-bin/wms_trmm?service=wms&version=1.1.1&request=getcapabilities', version='1.1.1')

In [8]: wms.identification.title
Out[8]: 'Tropical Rainfall Measuring Mission (TRMM) Data from NASA Goddard Earth Sciences Data and Information Services Center (GES DISC)'

[roomthily]

Contributing back to owslib would mean handling any writes/posts as well as the reads we're most interested in. And they seem not to be merging pull requests for the larger needs (swe/sos implementation, etc). (they didn't merge the pull request but added sos 2.0 anyway so not sure what's going on with that.)

[roomthily]

Somewhat related (but more about crawling than parsing) a python THREDDS service crawler - https://github.com/asascience-open/thredds_crawler and https://svn.oss.deltares.nl/repos/openearthtools/trunk/python/OpenEarthTools/openearthtools/io/opendap/opendapcrawler.py

[roomthily]

Handling Version Support

Let's talk about the changes related to namespacing and default paths. I need to scope this out better (apologies).

*Update: * the root element name changes are an easy patch (and that's been made, see 8f14e4e).

WMS

1.0.0:

1.1.1:

Root node identification issues here:

<!DOCTYPE WMT_MS_Capabilities SYSTEM "http://schemas.opengis.net/wms/1.1.1/WMS_MS_Capabilities.dtd"
 [
 <!ELEMENT VendorSpecificCapabilities EMPTY>
 ]>  <!-- end of DOCTYPE declaration -->

<WMT_MS_Capabilities version="1.1.1">

1.3.0: not using the OWS schema (retains a very similar structure to earlier versions). The format lists are, finally, well structured and not relying on element names.

WFS

1.0.0: Uses the element tag as format (or other parameter) value. Mix of "standard" values and controlled vocab-like terms.

1.1.0: Switches to the OWS structure with normalized Format (or other parameter) value lists.

WCS

1.0.0: uses the "older" standard of WxS_Capabilities as the root node.

1.1.2: Oddly, does not use WxS_Capabilities as the root node (just Capabilities) and is using the OWS namespace after root. This is an outlier across the services

CSW

SOS

Format and other Enumerated Values

(note: these are clearly not the complete XML structure)

# from the WFS 1.0.0
<GetFeature>
    <ResultFormat>
        <GML2/>
    </ResultFormat>

# from the WFS 1.1.0
<ows:Value>XMLSCHEMA</ows:Value>
<ows:Value>text/xml; subtype=gml/2.1.2</ows:Value>
<ows:Value>text/xml; subtype=gml/3.1.1</ows:Value>

[roomthily]

OWSLib Method/Operation Handling

for wms 1.1.1
    method returns:
        .name: 
            GetCapabilities
        .methods:
            [
                {'url': 'http://disc1.sci.gsfc.nasa.gov/daac-bin/wms_trmm?', 'type': 'Get'}, 
                {'url': 'http://disc1.sci.gsfc.nasa.gov/daac-bin/wms_trmm?', 'type': 'Post'}
            ]
        .formatOptions:
            ['application/vnd.ogc.wms_xml']

for wfs 1.0.0
    method returns:
        .name: 
            GetFeature
        .methods:
            [
                {'url': 'http://mrdata.usgs.gov/cgi-bin/mapserv?map=/mnt/mrt/map-files/podchrome.map&', 'type': 'Get'}, 
                {'url': 'http://mrdata.usgs.gov/cgi-bin/mapserv?map=/mnt/mrt/map-files/podchrome.map&', 'type': 'Post'}
            ]
        .formatOptions:
            ['{http://www.opengis.net/wfs}GML2']

for wfs 1.1.0
    method returns:
        .name: 
            GetFeature
        .methods:
            [
                {'url': 'http://mrdata.usgs.gov/services/nmra?', 'type': 'Get', 'constraints': []}, 
                {'url': 'http://mrdata.usgs.gov/services/nmra?', 'type': 'Post', 'constraints': []}
            ]
        .formatOptions:
            ['text/xml']
        .parameters:
            {
                'resultType': {'values': ['results', 'hits']}, 
                'outputFormat': {'values': ['text/xml; subtype=gml/3.1.1']}
            }

for csw 2.0.2
    method returns:
        .name: 
            DescribeRecord
        .methods:
            [{'url': 'http://cwic.wgiss.ceos.org/discovery', 'type': 'Get', 'constraints': []}]
        .formatOptions:
            ['text/xml']
        .parameters:
            {
                'typeName': {'values': ['csw:Record', 'gmd:MD_Metadata']}, 
                'outputFormat': {'values': ['application/xml']}, 
                'schemaLanguage': {'values': ['XMLSCHEMA']}
            }

for wcs 1.0.0
    method returns:
        .name: 
            GetCoverage
        .methods:
            [
                {'url': 'http://www.ncddc.noaa.gov/arcgis/services/DataAtlas/CMECS_Salinity_Winter/MapServer/WCSServer?', 'type': 'Get'}, 
                {'url': 'http://www.ncddc.noaa.gov/arcgis/services/DataAtlas/CMECS_Salinity_Winter/MapServer/WCSServer?', 'type': 'Post'}
            ]
        .formatOptions:
            ((NONE))