json-static: JSON validator and converter for OCaml
![[difficulty = 2 camels]](/mjambon2016/2camels.png "difficulty = 2 camels (requires a knowledge of OCaml's basics)")
Introduction
json-static is a tool for converting parsed JSON data with an unchecked structure into specialized OCaml types and vice-versa. It is a complement to the json-wheel library which provides a parser and a (pretty-) printer.
By reading a type definition, the preprocessor inserts code that converts between OCaml types (lists, arrays, tuples, objects, polymorphic variants, ...) and untyped JSON data. Those type definitions are written in a syntax which is very close to regular OCaml type definitions.
Why? OCaml is statically typed (unlike Perl, Python, Ruby, PHP or JavaScript).
It means that no type checking
occurs at runtime, because it's all done at compile-time. It allows the
programs to be faster and less memory consuming, as well as safer since
simple bugs such as if some_rare_event then 2 + "2"
will be detected at the time of developing the program rather than when
some_rare_event happens, which may well crash
a space shuttle. Now, this seems to be a problem for reading JSON data since
an OCaml record or object has field labels which must be known at compile-time.
Using only the core OCaml language,
you can load JSON data as a single type, but not specialized types
such as records, lists, etc. That is inconvenient since most of the time
you would manually write code that does the conversion,
and returns an error if the
format of the data isn't right. Well, this is precisely what
the json-static syntax extension
provides: when specifying the expected type of a JSON document,
converter functions from/to that type are created by the preprocessor.
The function that converts from generic JSON to the specific type
is also a validator, since the conversion will fail if the structure
of the JSON data doesn't comply to the expected format.
The big difference with dynamically-typed languages such as those mentioned above is that once the JSON data is converted (and validated), it is guaranteed that every bit of the program which uses data of that type will read it safely, e.g. it cannot attempt to read a misspelled field name or a string instead of an int because that would trigger a compilation error. In other words, you cannot forget to use the validator.
This syntax extension can be loaded like any other package and is compatible with other syntax extensions and libraries.
Documentation
Installation instructions, documentation and examples are given in the README file.
A realistic example: Yahoo! Image Search
View the colorized source code
or save the script yahoo.ml.
It's all explained inside.
Another cool example: del.icio.us popularity query
This is an ocamlscript script that queries del.icio.us and shows how many times a given URL was bookmarked regardless of the tags as well as for each tag.
How it works:
The special type definition is what defines the structure of the JSON data as returned by del.icio.us. For example, see what a query to del.icio.us for http://www.google.com/ returns.
type json results = < hash: string; top_tags: (string * int) assoc; url: string; total_posts: int > list
For those who don't speak OCaml: this is a list of objects with
four methods/fields (hash, top_tags, url, total_posts).
(string * int) assoc maps to the OCaml type
(string * int) list, but using assoc
specifies that it derives from a JSON object rather than a JSON array of arrays
of two elements.
The following functions are automatically generated and can be used in the rest of the program:
val json_of_results : results -> Json_type.t val results_of_json : Json_type.t -> results
These functions convert between generic JSON data and the
results type. json_of_results
always succeeds, but a call to
results_of_json would fail by raising
an exception if the structure of the JSON data is not compatible
with the expected type. This is why this converter is also a validator.
Important note: if any JSON object has more fields than expected or some expected fields are given more than once, those are silently ignored. It makes it easy to ignore fields that are not needed or unexpectedly added to the format.
Download
This software was written by Martin Jambon. It is distributed under a BSD license. The current version is 0.9.8; see Changes.
Disclaimer: this is a beta release.
You can install this library from GODI or manually:
The first version that works with OCaml 3.10.1 is 0.9.6.
Access to the development version of json-static is offered by OCamlForge.
Related projects
All of the following translate some kind of type definition into code that can validate the structure of a document and convert it to that type:- Kwalify: a YAML (and JSON) validator written in Ruby and ported to Java.
- Sexplib: same ideas as json-static, but for S-expressions instead of JSON.
- Col: same ideas as json-static, but for CSV files.
- IoXML: mapping from OCaml type definitions to an XML format.
- CDuce: an OCaml-friendly language for the manipulation of XML documents.
- Pa_arg: record-like type declarations for the automatic conversion of command-line options.