Wednesday, October 14, 2026
HomeProgrammingThink about using JSON arrays as a substitute of objects to switch...

Think about using JSON arrays as a substitute of objects to switch knowledge


When implementing the superior MULTISET operator in jOOQ, its implementation principally relied on SQL/JSON help of assorted RDBMS. Briefly, whereas customary SQL helps nested collections by way of ARRAY or MULTISET operators like this:

SELECT
  f.title,
  MULTISET(
    SELECT a.first_name, a.last_name
    FROM actor AS a
    JOIN film_actor AS fa ON a.actor_id = fa.actor_id
    WHERE fa.film_id = f.film_id
  )
FROM movie AS f;

That is poorly supported in most RDBMS, so jOOQ emulates it utilizing SQL/JSON as follows (or comparable):

SELECT
  f.title,
  (
    SELECT json_arrayagg(json_array(a.first_name, a.last_name))
    FROM actor AS a
    JOIN film_actor AS fa ON a.actor_id = fa.actor_id
    WHERE fa.film_id = f.film_id
  )
FROM movie AS f;

Wait a second. A JSON array of arrays??

You’d anticipate a nested assortment in JSON to be of this way, usually, for debuggability and good measure, and so forth. and so forth.:

[
{
"first_name": "John",
"last_name": "Doe"
}, {
"first_name": "Jane",
"last_name": "Smith"
}, ...
]

However jOOQ is serialising this, as a substitute??

[
[ "John", "Doe" ],
[ "Jane", "Smith" ],
...
]

Why, sure in fact! jOOQ is in full management of your SQL assertion and is aware of precisely what column (and knowledge kind) is at which place, since you helped jOOQ assemble not solely the question object mannequin, but additionally the consequence construction. So, a a lot sooner index entry is feasible, in comparison with the a lot slower column title entry.

The identical is true for odd consequence units, by the best way, the place jOOQ at all times calls JDBC’s ResultSet.getString(int), for instance, over ResultSet.getString(String). Not solely is it sooner, but additionally extra dependable. Take into consideration duplicate column names, e.g. when becoming a member of two tables that each include an ID column. Whereas JSON is just not opinionated about duplicate object keys, not all JSON parsers help this, not to mention Java Map sorts.

Enjoyable truth, whereas the Map API specification forbids duplicate keys, an implementation might in precept ignore this specification and outline non-standard behaviour that enables for accessing duplicate keys by way of the iteration strategies, corresponding to Map::entrySet.

See some experiments on this difficulty right here: https://github.com/jOOQ/jOOQ/points/11889

In any case, as soon as access-by-name is changed by positional entry, duplicate column names are not a problem.

Not associated to SQL

Whereas jOOQ does this in a SQL context, word that you could possibly additionally do that between any consumer and server software with any programming languages on all sides. That is on no account associated to SQL itself!

When to use this recommendation

Efficiency recommendation is at all times a tough factor. There’s no such factor as “at all times” (use JSON arrays) or “by no means” (use JSON objects). It’s a tradeoff. Within the case of jOOQ, JSON arrays of arrays are getting used to serialise knowledge that customers don’t see in any respect. Customers don’t even suppose when it comes to JSON, when utilizing the MULTISET operator, and since it’s mature sufficient now, they hardly should debug the serialisation. If we might, we’d use a binary serialisation format between the server and the consumer in jOOQ for even sooner outcomes. Regrettably, there’s no such factor in SQL as binary knowledge aggregation.

If you design an API (particularly one that’s consumed by others), then readability could also be way more necessary than rising pace by some percentages. JSON objects are way more clear when modelling an information kind, in comparison with JSON arrays that solely help positional subject entry.

I believe that this recommendation applies to many technical JSON serialisation use-cases, the place JSON needs to be used, but when it have been doable, binary codecs could be a most well-liked choice. But when doubtful, at all times do measure first!

Compression

Folks could also be lured into pondering that that is about knowledge switch, in case of which compression might mitigate a few of the repetitive object key overheads. That is definitely true, particularly in an HTTP context.

However the price of compression and decompression continues to be current when it comes to CPU overhead at each ends of the switch, so whereas it reduces the transferred knowledge measurement, it may be a lot easier to only do that manually with a unique knowledge format.

Benchmark

What would a declare about issues being sooner be with no reproducible benchmark? The next benchmark makes use of:

  • H2 in-memory, however you could possibly definitely reproduce it anyplace else, too, e.g. on Oracle or PostgreSQL
  • JMH, regardless of this not being a micro benchmark. JMH provides nice reproducibility and statistics instruments (common, customary deviation, and so forth.)

Through the use of these two instruments, the benchmark can measure all of:

  • The server aspect implications of serialising knowledge
  • The consumer aspect implications of parsing knowledge
  • The community switch overhead (not measured within the case of H2, so compression wouldn’t assist right here anyway)

It makes use of JDBC instantly, not jOOQ, so mapping overheads are omitted. It does use jOOQ’s built-in JSON parser, although, however you’ll get comparable outcomes with some other JSON parser, e.g. Jackson.

The outcomes on my machine are (greater is healthier)

For H2 (in reminiscence):

Benchmark                                   Mode  Cnt        Rating       Error  Models
JSONObjectVsArrayBenchmark.testJsonArray thrpt 7 1005894.475 ┬▒ 57611.223 ops/s
JSONObjectVsArrayBenchmark.testJsonObject thrpt 7 809580.238 ┬▒ 19848.664 ops/s

For PostgreSQL:

Benchmark                                   Mode  Cnt     Rating      Error  Models
JSONObjectVsArrayBenchmark.testJsonArray thrpt 7 1588.895 ┬▒ 433.826 ops/s
JSONObjectVsArrayBenchmark.testJsonObject thrpt 7 1387.053 ┬▒ 1132.281 ops/s

In every case, there’s roughly a 15% – 25% enchancment for a small knowledge set (10 rows with 2 columns). I’d anticipate it to be extra important for bigger outcomes, e.g. right here’s 10000 rows once more:

For H2:

Benchmark                                   Mode  Cnt     Rating    Error  Models
JSONObjectVsArrayBenchmark.testJsonArray thrpt 7 2932.684 ┬▒ 41.095 ops/s
JSONObjectVsArrayBenchmark.testJsonObject thrpt 7 1643.838 ┬▒ 31.943 ops/s

For PostgreSQL:

Benchmark                                   Mode  Cnt    Rating   Error  Models
JSONObjectVsArrayBenchmark.testJsonArray thrpt 7 122.875 ┬▒ 7.133 ops/s
JSONObjectVsArrayBenchmark.testJsonObject thrpt 7 71.916 ┬▒ 3.232 ops/s

These outcomes are important sufficient to justify the lowered readability, provided that each jOOQ question utilizing MULTISET will revenue from the speed-up.

The benchmark code is beneath. It makes use of a quite simple JSON parser handler, which doesn’t actually maintain monitor of object keys or array indexes, simply assumes that there’s precisely 1 quantity column and 1 string column. This shouldn’t actually matter for extra complicated outcomes, although.

Benchmark code beneath:

package deal org.jooq.impl;

import java.io.InputStream;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Assertion;
import java.util.ArrayList;
import java.util.Listing;
import java.util.Properties;

import org.jooq.DSLContext;
import org.jooq.check.benchmarks.native.PGQueryBenchmark.BenchmarkState;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Stage;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.TearDown;
import org.openjdk.jmh.annotations.Warmup;

@Fork(worth = 1, jvmArgsAppend = "-Dorg.jooq.no-logo=true")
@Warmup(iterations = 3, time = 3)
@Measurement(iterations = 7, time = 3)
public class JSONObjectVsArrayBenchmark {

    @State(Scope.Benchmark)
    public static class BenchmarkState {

        DSLContext ctx;
        Connection connection;
        Assertion  assertion;

        @Setup(Stage.Trial)
        public void setup() throws Exception {
            // H2 setup
            connection = DriverManager.getConnection("jdbc:h2:mem:json-benchmark", "sa", "");
            ctx = DSL.utilizing(connection);

            assertion = connection.createStatement();
            assertion.executeUpdate("drop desk if exists t");
            assertion.executeUpdate("create desk t (number_column int, string_column varchar(100))");
            assertion.executeUpdate("insert into t choose i, i from system_range(1, 10) as t(i)");

            // PostgreSQL setup
//            strive (InputStream is = BenchmarkState.class.getResourceAsStream("/config.postgres.properties")) {
//                Properties p = new Properties();
//                p.load(is);
//                Class.forName(p.getProperty("db.postgres.driver"));
//                connection = DriverManager.getConnection(
//                    p.getProperty("db.postgres.url"),
//                    p.getProperty("db.postgres.username"),
//                    p.getProperty("db.postgres.password")
//                );
//                ctx = DSL.utilizing(connection);
//
//                assertion = connection.createStatement();
//                assertion.executeUpdate("drop desk if exists t");
//                assertion.executeUpdate("create desk t (number_column int, string_column varchar(100))");
//                assertion.executeUpdate("insert into t choose i, i from generate_series(1, 10) as t(i)");
//            }
        }

        @TearDown(Stage.Trial)
        public void teardown() throws Exception {
            assertion.executeUpdate("drop desk if exists t");
            assertion.shut();
            connection.shut();
        }
    }

    // Simply present that each strategies actually do the identical factor
    public static void major(String[] args) throws Exception {
        JSONObjectVsArrayBenchmark f = new JSONObjectVsArrayBenchmark();
        BenchmarkState state = new BenchmarkState();
        state.setup();

        System.out.println(f.testJsonObject(state));
        System.out.println(f.testJsonArray(state));
    }

    report R(int n, String s) {}

    @Benchmark
    public Listing<R> testJsonObject(BenchmarkState state) throws SQLException {
        Listing<R> consequence = new ArrayList<>();

        strive (ResultSet rs = state.assertion.executeQuery(

            // H2 syntax (if solely there was a solution to summary over syntax ;)
            """
            choose
              json_arrayagg(json_object(
                key number_column worth number_column,
                key string_column worth string_column
              ))
            from t
            """

            // PostgreSQL syntax
//            """
//            choose
//              json_agg(json_build_object(
//                number_column, number_column,
//                string_column, string_column
//              ))
//            from t
//            """
        )) {
            whereas (rs.subsequent()) {
                new JSONParser(state.ctx, rs.getString(1), ch(consequence)).parse();
            }
        }

        return consequence;
    }

    @Benchmark
    public Listing<R> testJsonArray(BenchmarkState state) throws SQLException {
        Listing<R> consequence = new ArrayList<>();

        strive (ResultSet rs = state.assertion.executeQuery(

            // H2 syntax
            """
            choose
              json_arrayagg(json_array(number_column, string_column))
            from t
            """

            // PostgreSQL syntax
//            """
//            choose
//              json_agg(json_build_array(number_column, string_column))
//            from t
//            """
        )) {
            whereas (rs.subsequent()) {
                new JSONParser(state.ctx, rs.getString(1), ch(consequence)).parse();
            }
        }

        return consequence;
    }

    non-public JSONContentHandler ch(Listing<R> consequence) {
        return new JSONContentHandler() {

            int degree;
            int n;
            String s;

            @Override
            public void valueTrue() {
            }

            @Override
            public void valueString(String string) {
                s = string;
            }

            @Override
            public void valueNumber(String string) {
                n = Integer.parseInt(string);
            }

            @Override
            public void valueNull() {
            }

            @Override
            public void valueFalse() {
            }

            @Override
            public void startObject() {
                if (degree >= 1) {
                    n = 0;
                    s = null;
                }
            }

            @Override
            public void startArray() {
                if (degree++ >= 1) {
                    n = 0;
                    s = null;
                }
            }

            @Override
            public void property(String key) {
            }

            @Override
            public void endObject() {
                consequence.add(new R(n, s));
            }

            @Override
            public void endArray() {
                if (level-- >= 1) {
                    consequence.add(new R(n, s));
                }
            }
        };
    }
}
RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular

Recent Comments