Summary written by Altanbagana Ankhbayar & Siddhant Gadamsetti

In this paper we will talk about tail latencies and why they must be taken seriously when we are building cloud based software services. Also we will talk about addressing tail latencies in database systems engineering. And to address them, we have to faithfully capture and measure them. That could be tricky because the benchmark harness itself can impose latencies that are not actually observed in the system under observation. And we will tell about eradicating the latencies that are caused by systemic or neighbourly noise to make sure that what we are looking at are really latencies generated inside of the database system layer.

There are highly popular books and there are many books that are not as popular. Amazon found a way to actually make money on this “long tail”. They didn’t have to rent out the expensive floorspace in the city center or hire shop assistants. They invested in the great website, a great user experience and the books they stocked up in the warehouse, where rent is cheap. With that they can reduce the price that they need to invest in order to deliver a book to the customer and that allows them the creatidute to this long tail and actually make money on it. That is the notion of the “long tail”.

Something very similar applies to you when you are a vendor for database as a service. And if we again look at all the customers out there and we rank them in the descending order by their revenue, the cost that we have by providing DB-Software per customer determines the market that we can address. If you can lower the price of the cost of providing DB-Software per customer you can address a large market. And what we’ve been seen for the last 15 years is that, cloud providers such as Google, Telecom, IBM, Amazon, Microsoft, they have managed to really bring those costs down and to increase the market that they can address. Imagine you are a provider in this market. You have a narrow profit margins which means you want to utilize your ressources to the fullest, with available hardware you want to cater as many customers/tenants as possible. When I am such a provider, I have to make sure I am providing a good service. And the key metric here is the latencies.

So let’s capture the tail-latencies in DB systems benchmarking. And to do that we want to have trust in our results. And here we show that if we use a Java-based benchmark harness, the JVM/garbage collector can in fact make a noise and distort measured tail-latencies. Inside a JVM there’s this concept of Stop-the-World Pauses, for instant garbage collections. If the Stop-the-World event happens before or after you’ve taken your measurement, that’s fine, we are not going to count that towards the latencies that’ve been measured, but if this happens in between, Stop-the-World pause latencies will add this to our measured value.

Experimental Setup: A very popular benchmark harness is OLTP Bench, it is written in Java and it supports a range of DB systems such as Maria DB or PostgreSQL. It has also support from various Benchmarks. We chose TPC-C(analytical warehouse querys), YCSB(is more light-weighted compared to TPC-C) and NoOp(empty SQL statements, unrealistic benchamrk). Results that we are showing to you were on the Java Version 16 and we tried out 2 JVMs. And then we hand pick garbage collectors that are all geared towards producing low latencies in garbage collections.

We conduct an experiment for 60 seconds. We are measuring the latency in a milliseconds and each dot is 1 measurement and we focus on maximum latencies. Here you can see the results of NoOp benchmark on Hotspot JVM with G1 garbage collector. We compare logs from OLTP Bench and from JVM. The tail latencies look pretty similar, because it is an artifact of the garbage collection mechanism on the side of benchmark harness.

Let’s look at here the YSCB Benchmark. Here we can see the same picture with standard garbage collector. We also see the different garbage collectors have a different behaviour, they implement different strategies. The Z garbage collector actually does a good job at being low latency when collecting a garbage.