As a Software program Engineer in Microsoft’s Java Engineering Group, a part of my job is to research and enhance the efficiency of Java’s rubbish collectors. As a Java utility runs, the rubbish collector is answerable for allocating objects on the heap and releasing up heap area when these objects are not dwell. On this weblog submit, I’ll focus particularly on how the GC handles totally different volumes of dwell knowledge and objects of varied sizes.
In case you’re operating JDK 11 or later (and you have to be), the default rubbish collector is the Rubbish First Rubbish Collector (G1 GC). G1 is a regionalized collector, that means that the Java heap is partitioned into quite a lot of equal-sized areas. Areas sizes are an influence of two and may vary from 1 MB to 32 MB, sized such that there are at most 2048 areas. These areas are logically grouped into Eden, Survivor, and Tenured. Empty areas are thought of Free and may turn out to be any of the opposite three. As G1 is a generational collector, new objects are allotted into Eden areas. Objects which have survived not less than one GC cycle get copied into Survivor areas, and as soon as these objects survive sufficient GC cycles, they’re promoted to Tenured areas. One notable exception is humongous objects, that are allotted into contiguous Free areas as an alternative of Eden. These areas turn out to be Humongous areas and are thought of a part of Tenured area.
What are humongous objects?
G1 considers all objects greater than one-half of a reminiscence area to be humongous objects. From JEP 278, humongous objects at all times take up a complete variety of areas. If a humongous object is smaller than one area, then it takes up the entire area. If a humongous object is bigger than N areas and smaller than (N+1) areas, then it takes up (N+1) areas. No allocations are allowed within the free area, if any, of the final area. Humongous objects can by no means be moved from one area to a different.
What does this imply for the Java heap?
If humongous objects at all times take up a complete variety of areas, sure sizes of humongous objects are more room environment friendly than others. Let’s check out an instance heap with 32 MB areas. If areas are 32 MB, all objects over 16 MB are thought of humongous.
Humongous objects which are barely above the humongous threshold (on this instance, simply over 16 MB) or barely better than the dimensions of N areas (simply over 32 MB, 64 MB, and many others.) end in vital area left over in humongous areas.
Quite the opposite, humongous objects which are barely smaller than the dimensions of N areas (just below 32 MB, 64 MB, and many others.) occupy most of these areas with much less empty area left over.
What does this imply for my utility?
Based mostly on the sizes of the allotted objects, the identical quantity of dwell knowledge can really take up totally different quantities of heap area. Let’s study a number of totally different situations and see how the GC responds.
I initially ran these benchmarks as a part of a GC investigation with @Ana Marsh that didn’t focus particularly on humongous objects. Due to this, the next examples present the impact humongous objects can have on the heap in a real-world (albeit benchmarking) situation.
Within the first simulated system, long-lived knowledge occupies 50% of the heap. Within the second, long-lived knowledge occupies 50% of the heap and medium-lived knowledge occupies a further 25% of the heap. For every system sort, we examined how the GC performs each with and with out humongous objects.
Working the Benchmark
We used the HyperAlloc benchmark developed by the Amazon Corretto crew. HyperAlloc repeatedly allocates objects to attain a goal allocation fee and heap occupancy. Every newly allotted object is a random measurement between the required minimal and most object sizes. I used Microsoft’s construct of OpenJDK 17.0.2 with an 80 GB heap. For an 80 GB heap, heap areas are 32 MB by default, so all objects bigger than 16 MB are handled as humongous. I examined the next combos utilizing CBL-Mariner Linux (model 2.0.20220226) on an Intel Xeon Silver 4210 Processor with 40 vCPUs (2 sockets, 10 cores per socket, 2 threads per core) and 376 GB of RAM:
- 50% of the heap is dwell with objects from 128 bytes – 16 MB
- 50% of the heap is dwell with objects from 128 bytes – 20 MB (humongous objects)
- 75% of the heap is dwell with objects from 128 bytes – 16 MB
- 75% of the heap is dwell with objects from 128 bytes – 20 MB (humongous objects)
In case you’d wish to comply with alongside at house, right here’s an instance command line:
jdk-17.0.2+8/bin/java -Xms80G -Xmx80G -Xlog:gc*,gc+ref=debug,gc+phases=debug,gc+age=hint,safepoint:file=<GC log file> -XX:+AlwaysPreTouch -XX:+UseLargePages -XX:+UseTransparentHugePages -XX:ParallelGCThreads=40 -XX:+UseG1GC -javaagent:<jHiccup listing>/jHiccup.jar=-a -d 0 -i 1000 -l <jHiccup log file> -jar <HyperAlloc listing>/HyperAlloc-1.0.jar -a 4096 -h 81920 -s 40960 -d 600 -c false -t 40 -n 128 -x 655360 -l <CSV output file>
Let’s break this down a bit:
| jdk-17.0.2+8/bin/java | path to the java executable we’re testing |
| -Xms80G -Xmx80G | set the preliminary and most heap measurement to 80 GB |
| -a 4096 | set the goal allocation fee to 4 GB/s |
| -h 81920 | inform the benchmark the heap measurement is 80 GB |
| -s 40960 | set the goal long-lived heap occupancy to 40 GB (50% of the heap) |
| -m 20480 | set the goal mid-aged heap occupancy to twenty GB (25% of the heap) |
| -d 600 | set the benchmark length to 600 seconds |
| -c false | set compressedOops assist to false, wanted for a 32 GB or bigger heap |
| -t 40 | set the variety of employee threads to 40 |
| -n 128 | set the minimal object measurement to 128 bytes |
| -x 16777216 | set the utmost object measurement to 16 MB |
I used 40 employee threads as a result of my take a look at machine has 40 vCPUs. For this investigation, a benchmark length of 600 seconds is lengthy sufficient to present us the required data. I used a goal allocation fee of 4 GB/s, however in these 4 benchmark runs, my noticed allocation charges ranged from 19 GB/s to 50 GB/s. The precise allocation fee is dependent upon the variety of employee threads and the dimensions of allotted objects, so it might differ considerably from the goal handed to HyperAlloc. This variation within the precise allocation fee isn’t a problem for us, however it’s one thing to concentrate on in case you’re making an attempt to copy this experiment.
I additionally used a number of different JVM flags for these benchmarking runs. -XX:+AlwaysPreTouch tells the JVM to the touch all bytes of the max heap measurement with a ‘0’, inflicting the heap to be allotted in bodily reminiscence as an alternative of solely in digital reminiscence. This reduces web page entry time as the appliance runs, as pages are already loaded into reminiscence, however comes at the price of elevated JVM startup time. It’s usually a good suggestion to allow pre-touch on giant heaps or to simulate a long-running system. In Linux, we will enhance the dimensions of those pages utilizing -XX:+UseLargePages and -XX:+UseTransparentHugePages. Aleksey Shipilëv’s weblog submit offers extra details about the use instances and tradeoffs of utilizing these flags.
The info within the following sections was collected utilizing the next JVM logging choices: -Xlog:gc*,gc+ref=debug,gc+phases=debug,gc+age=hint,safepoint. This weblog submit offers intro on Unified JVM Logging (JEP 158).
Extra data on the way to arrange and run the benchmark is on the market from the supply at HyperAlloc’s README. To parse the generated GC logs, I used the open-source Microsoft GCToolkit.
Outcomes
Now that we’ve run the benchmark, let’s have a look at the heap earlier than and after GC. 50% of the heap is dwell and objects vary from 128 bytes – 16 MB, so there are not any humongous objects.
Every level on the graph on the left represents the heap occupancy on the immediate earlier than a GC occasion. The corresponding level on the identical timestamp on the fitting exhibits the heap occupancy straight after that very same GC occasion. Factors are coloured by the kind of GC occasion (G1GC younger preliminary mark, G1GC cleanup, combined GC, and younger GC). A very powerful factor to deal with right here is the younger GCs (proven as blue triangles).
The heap occupancy will get as much as roughly 72 GB earlier than a younger GC is triggered. After a younger GC, the heap occupancy is diminished to about 44 GB. We might anticipate this discount as a result of 50% of the heap (40 GB) is dwell for our simulated workload. The heap occupancy is 44 GB as an alternative of 40 GB post-GC resulting from floating rubbish after younger GC (there are some objects which are not dwell that haven’t but been collected). It’s price noting that though this workload with out humongous objects is our “least-stressed” situation, we’re nonetheless allocating plenty of knowledge and placing loads of strain on the GC.
Now let’s see what the heap appears like once we add humongous objects. Right here, 50% of the heap is dwell and objects vary from 128 bytes – 20 MB:
On this case, the GC is a little more confused – the heap occupancy typically will get as much as 78 GB earlier than younger GC! One other notable distinction between this situation and the earlier one is the heap occupancy AFTER younger GC. Now that we’ve added humongous objects, we see a post-GC heap occupancy of about 54 GB in comparison with the earlier 44 GB. This time, the distinction between the noticed heap occupancy and the precise quantity of dwell knowledge is because of humongous objects.
On this situation, 20% of the allotted objects are humongous objects, and these humongous objects make up 36% of the info quantity. Which means of the 40 GB of dwell knowledge, 25.6 GB lives in common objects and 14.4 GB lives in humongous objects. The typical humongous object on this workload is eighteen MB. As a result of our area measurement is 32 MB, humongous areas are solely 56.25% full on common. The 14.4 GB of knowledge in humongous objects due to this fact occupies 25.6 GB of heap area, with 11.2 GB of unusable area trapped in humongous areas. We must always anticipate to watch a heap occupancy of 51.2 GB (25.6 GB taken up by common objects + 25.6 GB taken up by humongous objects). The extra distinction between the projected 51.2 GB and the noticed 54 GB is due floating rubbish left after younger GC.
The heap strain resulting from humongous objects turns into much more outstanding as the amount of dwell knowledge will increase. Within the subsequent instance, 75% of the heap is dwell and objects vary from 128 bytes – 16 MB (no humongous objects).
We’re actually pushing the bounds of G1 right here. Now that 75% (60 GB) of the heap is dwell, we have now much less free area to accommodate transient knowledge. Younger GCs are first triggered when the heap occupancy reaches about 64 GB, however they can’t liberate any area, as proven by almost an identical traces of factors at 64 GB earlier than and after younger GC. Younger GCs get triggered once more when the heap occupancy rises to roughly 68 GB, 72 GB, after which 75 GB earlier than going right into a full GC. All rubbish is collected throughout a full GC, lowering the heap occupancy to the anticipated 60 GB.
That is what the heap appears like once we add humongous objects. 75% of the heap is dwell and objects vary from 128 bytes – 20 MB:
Wow. G1 is confused and it SHOWS. Like within the earlier case, younger GCs don’t appear to be efficient in lowering heap occupancy. Nevertheless, not like the earlier case, the heap occupancy hovers at round 77 GB even after full GC.
Like within the second situation, humongous objects make up 36% of the info quantity. Which means of the 60 GB of dwell knowledge, 38.4 GB lives in common objects and 21.6 GB lives in humongous objects. As a result of the typical humongous object is eighteen MB, humongous areas are solely 56.25% full on common. The 21.6 GB of knowledge in humongous objects due to this fact occupies 38.4 GB of heap area, with 16.8 GB of unusable area trapped in humongous areas. We must always anticipate to watch a heap occupancy of 76.8 GB (38.4 GB taken up by common objects + 38.4 GB taken up by humongous objects). As a reminder, solely 60 GB of our heap is meant to be dwell – however due to humongous objects, we want roughly 77 GB to accommodate all of the dwell knowledge. This leaves little or no area for transient knowledge, leading to near-constant GCs and considerably diminished throughput.
Testing the extremes
We got here dangerously near operating out of heap area within the earlier instance – so what occurs once we push G1 even additional? To check this, I ran HyperAlloc once more utilizing workloads completely made up of humongous objects. I used an 80 GB heap with 32 MB areas, so all objects bigger than 16 MB are handled as humongous:
- 75% of the heap is dwell with objects from 30 MB – 32 MB
- 75% of the heap is dwell with objects from 16 MB – 18 MB
Within the first workload, humongous areas are 96.88% full on common. The 60 GB of knowledge in humongous objects due to this fact occupies 61.9 GB of heap area, with only one.9 GB of unusable area trapped in humongous areas. Although all objects are humongous, there’s sufficient heap area for the appliance to run easily.
Within the second workload, humongous areas are solely 53.13% full on common. The 60 GB of knowledge in humongous objects due to this fact requires 112.94 GB of heap area – way over our 80 GB heap! After I examined this, the benchmark crashed with an OutOfMemoryError in about 5 seconds.
What can I do?
In case you discover that your utility has higher-than-expected heap utilization, you could have the identical drawback. If attainable, it might be useful to vary the sizes of allotted objects to keep away from humongous objects that end in wasted heap area. It might even be useful to regulate the dimensions of heap areas utilizing -XX:G1HeapRegionSize= such that the problematic objects are not humongous, or experiment with utilizing a special GC.
