JDIL

Data Integration with JSON

version 0.91 - 7/30/07

JSON (JavaScript Object Notation) is a serialization format for objects that originated with JavaScript. JSON is an order of magnitude less complex than XML. Still, it works nicely as a data interchange format for many applications - hence its rapidly growing popularity. Support for parsing and emitting JSON is now present in all major web programming environments.

However, unlike XML, JSON provides no direct support for namespaces - and thus no standard way for avoiding name collisions when mixing data from diverse sources. Something like a namespace mechanism is required to lift JSON to the level of a data integration platform, as opposed to a data exchange format only. Also lacking are standard ways of naming objects so that they can be referenced from elsewhere, and for representing properties with multiple values.

If these concerns are addressed, JSON's reach will extend over more of the domain currently occupied by XML, while bettering XML in the cardinal virtue of simplicity.

This is not hard to do. First, let us adopt XML's formulation of namespaces: a namespace is specified by a URI, and endowed with prefix which denotes that URI within a particular context. Qualified names of the form prefix:localpart are then used as short hand for the URI arrived at by appending localpart to the URI that prefix represents. Within JSON, this idea is implemented by giving properties qualified names: for example:

{
  "@namespaces": {
     "dc":"http://purl.org/dc/elements/1.1/",
     "rss":"http://purl.org/rss/1.0/",
     "georss":"http://www.georss.org/georss/"
  },
  "@type":"rss:channel",
  "rss:items": [
     { "@type":"rss:item",
       "rss:title":"A visit to Astoria",
       "rss:description":"sample description",
       "dc:coverage":{
          "dc:title":"Astoria, Oregon, US",
          "georss:point":"46.18806  -123.83"
        }
      }
   ]
 }

(note: dc:coverage means "where")

Those familiar with RSS will recognize this as a JSONized version of an RSS feed with one post.

JDIL built-in propertes start with "@" - the full list is @namespaces, @type, @id, and @values - plus @value, @alt and @bag when full RDF compatability is needed. The fragment

  "@namespaces": {
     "dc":"http://purl.org/dc/elements/1.1/",
     "rss":"http://purl.org/rss/1.0/",
     "georss":"http://www.georss.org/georss/"
  }

asserts, eg, that dc is the namespace prefix for http://purl.org/dc/elements/1.1/ in the scope of the current object.

For example:

{
  "@namespaces": {
     "":"http://purl.org/rss/1.0/"
  },
  "@type":"channel",
  "items": [
     { "@type":"item",
       ":title":"Example"
     }
   ]
 }

-->

Identifiers may be attached to objects within a JDIL file with "@id"

{
  "@namespaces": {
     "dc":"http://purl.org/dc/elements/1.1/",
     "rss":"http://purl.org/rss/1.0/",
     "georss":"http://www.georss.org/georss/"
  },
  "@type":"rss:channel",
  "rss:items": [
     { "@type":"rss:item",
       "rss:title":"A visit to Astoria",
       "rss:description":"sample description",
       "dc:coverage":{
          "@id":"a0",
          "dc:title":"Astoria, Oregon, US",
          "georss:point":"46.18806  -123.83"
        }
      },
      { "@type":"rss:item",
        "rss:title":"A second visit to Astoria",
        "rss:description":"sample description",
        "dc:coverage":{"@id":"a0"}
      }
   ]
 }

All JSON objects with the same value for an @id property represent the same JDIL object. Formulated in mathematical language, the object/proprerty graph denoted by a JDIL file is constructed by identifying JSON object nodes with the same @id property values. Thus, the above example represents an RSS feed with two posts whose coverage properties point to a common value - the object with @id value "a0" representing the town of Astoria, Oregon.

A property name may be prefixed with "*" to reference to an object by its identifier. For example,

"*dc:coverage":"a0"

is an abbreviation for

"dc:coverage":{"@id":"a0"}

Qualified names (which denote URIs) can also serve as identifiers for objects by appearing as values of the @id property. Unlike ids, URIs are intended to carry a common identity across files. Example:


{
  "@namespaces": {
    "dc":"http://purl.org/dc/elements/1.1/",
    "geofeature":"http://sws.geonames.org/",
    "rss":"http://purl.org/rss/1.0/",
    "thisFeed":"http://jdil.org/examples/feed0/"
  },
  "@type":"rss:channel",
  "rss:items": [
     { "@type":"rss:item",
       "@id":"thisFeed:item0",
       "rss:title":"A visit to Astoria",
       "rss:description":"sample description",
       "*dc:coverage":"geofeature:5711847/"
       }
     }
  ]
}

In the above, "geofeature:5711847/" is a name whose scope is the whole web: it is the name assigned to the town of Astoria Oregon by Geonames.

The property "@id":"thisFeed:item0" assigns the URI http://jdil.org/examples/feed0/item0 as the name of the the current item. This allows other JDIL files to refer to the item via URI, just as this example refers to the town of Astoria via the URI http://sws.geonames.org/5711847/.

The value of "@id" (or *'ed properties) may be supplied as a full URIs instead of a qualified name. In the example above, the line

    "@id":"thisFeed:item0",

may be replaced by

    "@id":"http://jdil.org/examples/feed0/item0",

Sometimes it is useful to assign more than one value to a property. JSON does not support this directly, so an encoding is needed. An expression of the form

{
  "nso:prop0":{"@values":["a","b"]}
}

represents the assertion that the property ns0:prop0 has two values, "a", and "b". This is distinct from asserting that the value of the property is an array with two values.

Unqualified Property Names

In the examples above, properties have been given qualified names, but JDIL also permits the use of unqualified names - names in which no colon appears. Formally expressed, JDIL specifies a JSON encoding of object-property graphs. Properties and nodes are labeled either by URIs or by unqualified names; properties are always labeled by one or the other, but it is not required that nodes be labeled. In applications, the meaning of unqualified names for properties is normally disambiguated by the @type of the object to which they apply. The GeoJson below provides an example.

GeoJSON

GeoJSON is a JSON format for geography. Since GeoJSON is specified without namespaces (at least for its own primitives), a minor transformation is needed for embedding it into JDIL: lines of the form

"type":"<GeoJSON Type>

need to be replaced by

"@type:":"geo:<GeoJSON Type>";

where "geo" is the assigned prefix of the GeoJSON namespace.

Here is an example of using GeoJSON instead of GeoRSS in the context of an RSS feed:


{
  "@namespaces": {
     "dc":"http://purl.org/dc/elements/1.1/",
     "rss":"http://purl.org/rss/1.0/",
     "geo":http://geojson.org/ns#"
  },
  "@type":"rss:channel",
  "rss:items": [
     { "@type":"rss:item",
       "rss:title":"A visit to Astoria",
       "rss:description":"sample description",
       "dc:coverage":{
           "@type":"geo:Feature",
           "@id":"a0",
           "dc:title":"Astoria, Oregon, US",
           "dc:coverage": {
           
              "@type": "geo:Polygon",
              "coordinates": [
                 [ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ]
               ]
             
           }
         }
       },
       { "@type":"rss:item",
        "rss:title":"A second visit to Astoria",
        "rss:description":"sample description",
        "*dc:coverage":"a0"
      }
   ]
 }

Note that the "coordinates" property of the polygon has an unqualified name. There is no need for qualification in this context: the type of the node disambiguates the meaning of the property.

External Definition of Namespaces

If the value of the @namespace property is a URL string rather than an object, then the URL is expected to hold the JDIL namespace definition. Here's an example:


{
  "@namespaces":"http://jdil.org/examples/namespaces.jdil",   
  "@type": "geo:Point",
  "coordinates": [100,20]
}

If the namespace URL is served with an Expires header far in the future, applications where the namespace definitions remain stable over many transmissions can reduce boilerplate burden.

RDF

The conventional XML syntax for RDF is an encoding of a data model which is very close to the one we have just described. If we posit that JDIL arrays represent RDF sequences (rdf:Seq), and that JDIL's @type is a synonym for rdf:type, then the JDIL data model is in fact a subset of the RDF model. A few minor structural extensions extend JDIL to full coverage of the RDF model: data types, rdf:Bag, and rdf:Alt. The remainder of the RDF specification consists of assignments of meaning to particular URIs. (For example, "rdf:List" names the class of lists, and "rdf:first" the property that extracts the first element of a list.)

Here are the details. The construction

{"@bag":["a","b"]}

represents an rdf:Bag with elements "a" and "b", while

{"@alt":["a","b"]}

represents an rdf:Alt with alternate values "a" and "b". Finally, consider

{
"@namespaces": {
  "xsd":"http://www.w3.org/2000/10/XMLSchema#"
  "ns0":"http://example.org/sampleNamespace"
 }
{"ns:prop0":
  {"@type":"xsd:decimal","@value":33}
}
}

The construction:

{"@type":"xsd:decimal","@value":33}

represents the assignment of the datatype "xsd:decimal" to the numeral 33.

With these extensions, JDIL provides an alternative syntax for RDF - one that is much simpler than the standard XML RDF syntax. The complexity of the latter has motivated several proposed alternatives, such as Turtle, and RPV. JDIL provides comparable simplicity within a JSON framework.

The RDF-specific extensions are placed in a separate section to clarify the distinction between the data model and the use of the model by RDF to represent statements. JDIL, as a syntax for the former, can be used in any application requiring transport of URI-labeled object-property graphs, without restriction to those which happen to encode RDF.

Due to the rules governing the RDF data model, all property names will be qualified in JDIL that carries RDF content.

A Sample JDIL Feed

Platial's maps are available as JDIL feeds - they are JDIL encoding of Platial's GeoRSS feeds. Here's an example: http://platial.com/jdil/map/1419 which is the JDIL feed for this Platial map. The corresponding feed in XML syntax is http://platial.com/rss/map/1419.

Related Work

Whenever RDF is processed in a JavaScript environment, as in Jim Ley's JavaScript RDF parser, or Masahide Kanzaki's Turtle parser, a JSON encoding has in effect has taken place. Kanzaki's parser has an explicit option for emitting JSON. These parsers employ representations which differ in detail from JDIL. In particular, the category of values (eg URI reference vs string literal) are coded by string format rather by object structure as in JDIL - neccessitating peeking inside of strings to determine triple content in the former case.

A method has been specified for serializing SPARQL query results in JSON, but this method is specialized to the case of representing the variable bindings (resp boolean values) returned by SELECT (resp ASK) queries. This is not the same task as representing general RDF structures in JSON.

Contact:Chris Goad at chris@platial.com