Easily lint JSON files in CI

In a few projects, we sometimes had the problem, that invalid JSON files were provided by the customer and not checked for errors. To prevent this, we searched for the smallest possible solution.

At first, I thought about using JSON Schema but this would require the generation or creation of a correct schema and a special linter, which applies the schema to the JSON files.

This would be more effort than really needed since the provided invalid files didn't pass any simple JSON validation. In most cases, we had superfluous commas and quotes or they were missing in some parts.

After thinking for a while about an optimal approach, that can cover this with the smallest effort, I remembered the command line tool jq for working with JSON data.

So I tried to go with this and validate or lint the JSON files.

First I had to install jq in our CI setup:

apt-get update
apt-get install jq -y

For systems which use apk I have used the following commands:

apk update
apk add jq

Now we just had to find the right command to implement the validation logic. jq works with filters and for our cases we needed the simplest filter, which is a dot (.). This normally outputs the provided JSON string in pretty-printed form.

From man jq:

BASIC FILTERS
   Identity: .
       The absolute simplest filter is . . This is a filter that takes its input and produces it unchanged as output. That is, this is the identity operator.

       Since jq by default pretty-prints all output, this trivial program can be a useful way of formatting JSON output from, say, curl.

           jq '.'
              "Hello, world!"
           => "Hello, world!"

With this, we can create the needed commands to parse one or more JSON files.

Parse single JSON file:

jq . app/public/assets/data/data.json

Parse multiple JSON files (via find command):

find src/i18n/*.json -exec jq . {} +

The results will look like this:

$ find src/i18n/*.json -exec jq . {} +
parse error: Expected value before ',' at line 2, column 14
Cleaning up project directory and file based variables 00:00
ERROR: Job failed: exit code 1

But the output does not show, which exact files are invalid. For this, you can print the filename and run the jq command with a bash script.

for f in src/i18n/*.json
do
  echo "\n$f" && jq . "$f"
done

To hide the output to stdout from jq, we can pipe it to /dev/null:

for f in src/i18n/*.json
do
  echo "\n$f" && jq . "$f" 1>/dev/null
done

As oneliner, the final code looks like this:

for f in src/i18n/*.json; do echo "\n$f" && jq . "$f" 1>/dev/null; done

There is an open issue about providing the filename and more details in the error messages of jq. Until this is implemented, we use this simple approach to check individual files and catch them as early as possible in our CI pipelines.