Do you need a database gateway?

What is a database gateway?

A database gateway is an appliance (usually a virtual one) that sits between your database server(s) and your database client(s). By acting as a proxy between them, at the wire protocol level, the gateway can monitor, redirect, inject, modify and reject requests and responses between database clients and database servers.

In other words, a database gateway can change the way your databases and your applications work, without actually modifying your database or your client.

This is no different than other types of gateways. For instance, when you look at a web page in your browser, it's entirely possible that it's not exactly the page that was sent by the site you're browsing. Many enterprises and ISPs have gateways that introspect HTML pages and add or remove ads, scripts, and other content. This happens transparently, and most people are completely unaware that this is happening.

A database gateway works similarly: it receives requests from database clients, examines them, applies various bits of logic to them, potentially rejects or changes some requests, and forwards these requests to the database server. It then does the same with the responses from the database server: it examines them and can modify them before sending them back to the database client.

Why would you want to do that?

For the same reason you use any type of gateway: control.

Possible applications include:

- Logic enforcement

Because database gateways have a deep access to all traffic to and from the database(s), they can enforce some logic in a way that applies to literally all clients, regardless of how they were written. Want to make sure that federal contracts can never be deleted? Or that bids can only be seen in their respective countries for certain people? That type of fine-grained logic is easily defined and enforced, for all clients, by database gateways. Because the gateway acts as a proxy, the clients and servers are completely unaware that this is going on.

- Legacy applications

Every large organization has a portfolio of applications, many of which are barely, or no longer, maintained. A database gateway can change how these applications work, without actually changing the applications or the databases. It's not a fix-all -- the gateway can only change the interactions of the applications with the databases -- but when it hits the spot, it can be a life-saver, and extend the lifetime of an application with minimal cost.

- Security

Sometimes you don't entirely trust your database clients. A database gateway can keep an eye on who's doing what to what data, and change, restrict or deny access to certain data based on any number of factors: access behavior, time of day, point of origin, the nature of the data,  etc... 

Security is a crowded field: there are many off-the-shelf products addressing various aspects of security, and if these products do what you need, they're probably the right solution. Database gateways are more generalized tools, which can step in when you need to do something that's not possible with off-the-shelf solutions.

- Monitoring

There are many solutions for monitoring databases, but they tend to be relatively inflexible. A database gateway gives you the ultimate access to everything going into and out of your database, without restrictions. Again, there are many excellent products in this area, but when you need to do something that's not addressed by off-the-shelf solutions, a gateway can be a quick and easy way to do what you need. Maybe you want to bill someone based on how much data they access, or maybe you want to record certain interactions for auditing purposes. This jack-of-all-trades aspect of database gateways makes them attractive for ad-hoc solutions.

- Externalizing logic

New applications can also benefit from a database gateway. Once you start thinking about your data as a potentially dynamic entity, you may realize that some application logic maybe does not belong in the application, but can be externalized to a gateway. This makes it easier to change this logic, to customize it at deployment time, and sometimes even at runtime.

And the list goes on. A full-blown database gateway can be a remarkably versatile tool. There are too many possible uses to go over here: merging data from multiple systems, masking or encrypting data, simulating failures or delays, deriving synthetic data from existing data on the fly, the list is endless.

All these uses can be summarized in one word: indirection. Crack open the pipe between database clients and database servers, and you open up a world of possibilities that are unthinkable otherwise. A database gateway is an elegant solution to some problems that would be very difficult to solve in any other way.

What about performance?

A database gateway is an extra hop between your database clients and your database servers, therefore it stands to reason that a gateway will reduce apparent database performance. In most cases, that's not an issue: most applications can take a 10% or 20% increase in database response times without a noticeable difference.

But if you have to extract every ounce of performance from your database at all cost, a database gateway is probably not a good option. A gateway can provide wonderful value, but that value comes at a (usually reasonable) performance cost. Note that you would incur a comparable cost using any other solution.

But that's not the real data!

When accessing the data in your database, you expect a straight answer. Databases are often sources of record: if the database says so, then it is so.

That is why the concept of a database gateway may be unsettling to some. Suddenly, when accessing a database through a database gateway, you no longer know whether the data you're getting is in fact what's on disk, or whether it's been changed or restricted (or even created) somehow by the gateway.

That's kind of the whole point: the database gateway becomes part of the total system, and the database clients should not (typically) worry about whether the data they're seeing has or has not been modified by the gateway. But if needed, the gateway may sometimes add some sort of hint or stamp to the data as a way of saying "this is not exactly the data that is stored in the database".

And, of course, you can always (if authorized) bypass the gateway and access the database directly.

Isn't this just a proxy?

A database gateway is different than a database proxy like ProxySQL, Heimdall Data or Tungsten. These products focus on routing, sharding, connection pooling and the like. These are useful, but they don't usually affect the apparent behavior of the database or of the clients.

A proper gateway will give you full access to the database traffic, down to the lowest level, and allow you to modify it to your heart's content, easily and transparently.

Conclusion

Database gateways are not applicable to all databases, obviously: they have a performance cost, and add complexity, but they provide a crucial control point, and make it possible to affect many systems and applications on the fly, without changing them.

At the end of the day, it's just another way to make your systems behave the way you want them to behave.