Prof. Dr. Tilmann Rabl

Tail Latencies in the DB/OS Stack

Stefanie Scherzinger, Uni Passau

Summary written by Altanbagana Ankhbayar & Siddhant Gadamsetti

In this paper we will talk about tail latencies and why they must be taken seriously when we are building cloud based software services. Also we will talk about addressing tail latencies in database systems engineering. And to address them, we have to faithfully capture and measure them. That could be tricky because the benchmark harness itself can impose latencies that are not actually observed in the system under observation. And we will tell about eradicating the latencies that are caused by systemic or neighbourly noise to make sure that what we are looking at are really latencies generated inside of the database system layer.

There are highly popular books and there are many books that are not as popular. Amazon found a way to actually make money on this “long tail”. They didn’t have to rent out the expensive floorspace in the city center or hire shop assistants. They invested in the great website, a great user experience and the books they stocked up in the warehouse, where rent is cheap. With that they can reduce the price that they need to invest in order to deliver a book to the customer and that allows them the creatidute to this long tail and actually make money on it. That is the notion of the “long tail”.

Something very similar applies to you when you are a vendor for database as a service. And if we again look at all the customers out there and we rank them in the descending order by their revenue, the cost that we have by providing DB-Software per customer determines the market that we can address. If you can lower the price of the cost of providing DB-Software per customer you can address a large market. And what we’ve been seen for the last 15 years is that, cloud providers such as Google, Telecom, IBM, Amazon, Microsoft, they have managed to really bring those costs down and to increase the market that they can address. Imagine you are a provider in this market. You have a narrow profit margins which means you want to utilize your ressources to the fullest, with available hardware you want to cater as many customers/tenants as possible. When I am such a provider, I have to make sure I am providing a good service. And the key metric here is the latencies.

So let’s capture the tail-latencies in DB systems benchmarking. And to do that we want to have trust in our results. And here we show that if we use a Java-based benchmark harness, the JVM/garbage collector can in fact make a noise and distort measured tail-latencies. Inside a JVM there’s this concept of Stop-the-World Pauses, for instant garbage collections. If the Stop-the-World event happens before or after you’ve taken your measurement, that’s fine, we are not going to count that towards the latencies that’ve been measured, but if this happens in between, Stop-the-World pause latencies will add this to our measured value.

Experimental Setup: A very popular benchmark harness is OLTP Bench, it is written in Java and it supports a range of DB systems such as Maria DB or PostgreSQL. It has also support from various Benchmarks. We chose TPC-C(analytical warehouse querys), YCSB(is more light-weighted compared to TPC-C) and NoOp(empty SQL statements, unrealistic benchamrk). Results that we are showing to you were on the Java Version 16 and we tried out 2 JVMs. And then we hand pick garbage collectors that are all geared towards producing low latencies in garbage collections.

Throughput for MariaDB for 3 benchmarks and all garbage collectors.

We conduct an experiment for 60 seconds. We are measuring the latency in a milliseconds and each dot is 1 measurement and we focus on maximum latencies. Here you can see the results of NoOp benchmark on Hotspot JVM with G1 garbage collector. We compare logs from OLTP Bench and from JVM. The tail latencies look pretty similar, because it is an artifact of the garbage collection mechanism on the side of benchmark harness.

Let’s look at here the YSCB Benchmark. Here we can see the same picture with standard garbage collector. We also see the different garbage collectors have a different behaviour, they implement different strategies. The Z garbage collector actually does a good job at being low latency when collecting a garbage.

Here is another chart showing the distributions with boxplots.

Summary: A Java-based benchmark harness serves well for measuring throughput or 95th or 99th latency percentile. if you want to look carefully what is going on in the tail latencies you have to take care, because the JVM/GC can introduce latencies in the range of milliseconds and depending on your database workload it can distort your measurements. The more light-weighted your workload is, the more vulnerable it will be to that effects.

Second experiment

The idea of our second experiment is to customize database and operating systems stack in that way to eradicate the sources of systemic noise and shield the database engine from neighbourly noise, when multiple tennants are computing for the same resources. We take the database management system, run a benchmark on it, take standard operating system such as Linux, but tailor it carefully using modern features of the system such that we got latencies under control. We have 2 execution platforms: realistic one (Dell PowerEdge T440) with a list of noise sources and almost bare metal(Beaglebone Black) without any additional hardware. Our benchmark scenario: we use main memory database called DBToaster. Our benchmark continiously evaluates queries on data stream.

We define different scenarios for our experiment.

Summary: We combine databases with real-time embedded systems, we build several different tailored standard operating sytems. We aggressively eradicated noise to the point where measuring time actually becomes itself a major source of noise and we have to re-think how we measure time. We have been able to isolate the tenants without harming them and their productivity and we have managed to run our load scenarios that tail latencies are as well-behaved as they are in non-load scenarios.