Monitoring databases in large,  distributed or hybrid environments

Monitoring database or SQL servers can in itself be a complex task - as we discuss in SQL Monitoring - 5 steps to full control. But what if you have a really complex environment, with multiple locations, restricted networks behind firewalls, high number of servers or a hybrid solution with some on-premise and some in a cloud? What are the issues, and how can you resolve them?

dbWatch Enterprise Manager 12

This year Bob’s job as a DBA has some challenging issues. His company is considering moving to the cloud, which has Bob worried about job security. The developers are interested in adopting Devops, to streamline their applications. Additionally, his boss wants more database security and decided that the system needs 24 x 7 maintenance. As if that’s not enough, there seems to be never ending demands for growth in services, solutions, data and the need for speed.

No matter what the service, no matter what the purpose, no matter what the product, the cloud is being touted as an answer to scalability by pretty much every major vendor. It started with data-storage, and rapidly moved to databases. In the past we were limited by the actual hardware that we owned. If we needed to expand on it, we needed to purchase more memory, and in many cases we needed to purchase new hardware as the old wore out, or was incapable of handling what we needed it to do.

It's an old problem. The more results you deliver, the more that are expected. The faster you provide them, the faster they are expected. Next thing you know, old methods don't work as well as they used to.

New demands require new workflows. On top of it new technologies are appearing making it seem like your old tools are no longer needed. Next thing you know the environment in which you work has changed. It is barely recognisable, and you might be afraid that you are no longer relevant.

As most DBAs know, security of data is one of the most difficult yet important tasks of maintaining a large estate of databases. It has kept more than one administrator up at night worrying about potential threats and pitfalls. With the growth of the information economy, not only is most information stored in databases, the value of that information has grown. As with anything with
value, the threats to security increases at a direct correlation to its worth.

Complexity is often a natural condition of most successful businesses. We build databases to handle complex data, to
maintain a layer of structure for important business information.

However, when building a database, or cluster of databases, typically the needs or requirements change over time. New
divisions or projects spring up. This is generally not a bad thing for a business or organisation. In most cases growth is good. However order to do this, without incurring huge amount of expense, often you a add these modules into existing databases rather than create new ones for different purposes.

There are many advantages to this approach; it makes accessing data easier if needed. However, in some (read: many) cases, you need to create new databases to handle different functions. One part of a business, such as vendor contract information, may have literally nothing to do with another, such as customer service records. So new databases are created. Maybe even the needs in one function, such as sales records increase beyond the capacity of the original
database they receive higher workloads than others. 

One of the primary features of a successful company is growth. In the earlier stages, you started working in a relatively simple environment. Maybe you had a few separate databases handling a few functions. Typically, these handled data for maintaining and managing products, employees, and sales.

Dealing with a wide variety of databases, platforms and versions is a reality for many companies. While it would be nice to be able to have one platform to handle all tasks necessary within an organisation, the reality is that different needs require different solutions. As a result, we create (or at least are forced to work with) diverse systems.

A variety of systems can be great for providing solutions which might not be easily available within one database platform. Unfortunately, this diversity also can bring unwanted complexity to the database management process. When a system becomes too complex, it can sometimes feel like you are losing track of the needs and goals of the organisation. While solving one problem, another problem occurs in a different database, which behaves entirely differently than the one you are working on.