Bigtable Pagination in Java – Java Code Geeks

 Think about a set of rows saved in Bigtable desk known as “folks”:

My goal is to have the ability to paginate a number of data at a time, say with every web page containing 4 data:

Web page 1:

Web page 2:

Web page 3:

Excessive-Degree Method

A excessive stage method to doing that is to introduce two parameters:

• Offset — the purpose from which to retrieve the data.
• Restrict — the variety of data to retrieve per web page

Restrict in all circumstances is 4 in my instance. Offset supplies some option to point out the place to retrieve the following set of data from. Bigtable orders the document lexicographically utilizing the important thing of every row, so one option to point out offset is by utilizing the important thing of the final document on a web page. Given this, and utilizing a marker offset of empty string for the primary web page, offset and document for every web page seems like this:

Web page 1 — offset: “”, restrict: 4

Web page 2 — offset: “individual#id-004”, restrict: 4

Web page 3 — offset: “individual#id-008”, restrict: 4

The problem now could be in determining tips on how to retrieve a set of data given a prefix, an offset, and a restrict.

Retrieving data given a prefix, offset, restrict

Bigtable java consumer supplies a“readRows” api, that takes in a Question and returns a listing of rows.

import com.google.cloud.bigtable.information.v2.BigtableDataClient
import com.google.cloud.bigtable.information.v2.fashions.Question
import com.google.cloud.bigtable.information.v2.fashions.Row

val rows: Checklist<Row> = bigtableDataClient.readRows(question).toList()

Now, Question has a variant that takes in a prefix and returns rows matching the prefix:

import com.google.cloud.bigtable.information.v2.BigtableDataClient
import com.google.cloud.bigtable.information.v2.fashions.Question
import com.google.cloud.bigtable.information.v2.fashions.Row

val question: Question = Question.create("folks").restrict(restrict).prefix(keyPrefix)
val rows: Checklist<Row> = bigtableDataClient.readRows(question).toList()

This works for the primary web page, nevertheless, for subsequent pages, the offset must be accounted for.

A option to get this to work is to make use of a Question that takes in a variety:

import com.google.cloud.bigtable.information.v2.BigtableDataClient
import com.google.cloud.bigtable.information.v2.fashions.Question
import com.google.cloud.bigtable.information.v2.fashions.Row
import com.google.cloud.bigtable.information.v2.fashions.Vary

val vary: Vary.ByteStringRange = 
    Vary.ByteStringRange
        .unbounded()
        .startOpen(offset)
        .endOpen(finish)

val question: Question = Question.create("folks")
                    .restrict(restrict)
                    .vary(vary)

The issue with that is to determine what the tip of the vary ought to be. That is the place a neat utility that the Bigtable Java library supplies is available in. This utility given a prefix of “abc”, calculates the tip of the vary to be “abd”

import com.google.cloud.bigtable.information.v2.fashions.Vary

val vary = Vary.ByteStringRange.prefix("abc")

Placing this all collectively, a question that fetches paginated rows at an offset seems like this:

val question: Question =
    Question.create("folks")
        .restrict(restrict)
        .vary(Vary.ByteStringRange
            .prefix(keyPrefix)
            .startOpen(offset))

val rows: Checklist<Row> = bigtableDataClient.readRows(question).toList()

When returning the consequence, the ultimate key must be returned in order that it may be used because the offset for the following web page, this may be finished in Kotlin by having the next sort:

information class Web page<T>(val information; Checklist<T>, val nextOffset: String)

Conclusion

I’ve a full instance accessible right here — this pulls in the correct library dependencies and has all of the mechanics of pagination wrapped right into a working pattern.