This year Bob’s job as a DBA has some challenging issues. His company is considering moving to the cloud, which has Bob worried about job security. The developers are interested in adopting Devops, to streamline their applications. Additionally, his boss wants more database security and decided that the system needs 24 x 7 maintenance. As if that’s not enough, there seems to be never ending demands for growth in services, solutions, data and the need for speed.

No matter what the service, no matter what the purpose, no matter what the product, the cloud is being touted as an answer to scalability by pretty much every major vendor. It started with data-storage, and rapidly moved to databases. In the past we were limited by the actual hardware that we owned. If we needed to expand on it, we needed to purchase more memory, and in many cases we needed to purchase new hardware as the old wore out, or was incapable of handling what we needed it to do.

It's an old problem. The more results you deliver, the more that are expected. The faster you provide them, the faster they are expected. Next thing you know, old methods don't work as well as they used to.

New demands require new workflows. On top of it new technologies are appearing making it seem like your old tools are no longer needed. Next thing you know the environment in which you work has changed. It is barely recognisable, and you might be afraid that you are no longer relevant.

As most DBAs know, security of data is one of the most difficult yet important tasks of maintaining a large estate of databases. It has kept more than one administrator up at night worrying about potential threats and pitfalls. With the growth of the information economy, not only is most information stored in databases, the value of that information has grown. As with anything with
value, the threats to security increases at a direct correlation to its worth.

Complexity is often a natural condition of most successful businesses. We build databases to handle complex data, to
maintain a layer of structure for important business information.

However, when building a database, or cluster of databases, typically the needs or requirements change over time. New
divisions or projects spring up. This is generally not a bad thing for a business or organisation. In most cases growth is good. However order to do this, without incurring huge amount of expense, often you a add these modules into existing databases rather than create new ones for different purposes.

There are many advantages to this approach; it makes accessing data easier if needed. However, in some (read: many) cases, you need to create new databases to handle different functions. One part of a business, such as vendor contract information, may have literally nothing to do with another, such as customer service records. So new databases are created. Maybe even the needs in one function, such as sales records increase beyond the capacity of the original
database they receive higher workloads than others. 

One of the primary features of a successful company is growth. In the earlier stages, you started working in a relatively simple environment. Maybe you had a few separate databases handling a few functions. Typically, these handled data for maintaining and managing products, employees, and sales.

Dealing with a wide variety of databases, platforms and versions is a reality for many companies. While it would be nice to be able to have one platform to handle all tasks necessary within an organisation, the reality is that different needs require different solutions. As a result, we create (or at least are forced to work with) diverse systems.

A variety of systems can be great for providing solutions which might not be easily available within one database platform. Unfortunately, this diversity also can bring unwanted complexity to the database management process. When a system becomes too complex, it can sometimes feel like you are losing track of the needs and goals of the organisation. While solving one problem, another problem occurs in a different database, which behaves entirely differently than the one you are working on.

The goal of any organisation, business or otherwise, is to grow.

In the past, the size of a company was determined by the number of physical products that one either created or sold. Say you built and sold left-handed toolboxes. If you sold enough of these left-handed toolboxes, and if they were of good enough quality, more people wanted your toolboxes, so you created more.

With this growth, was a need to keep a record of these products, customers, sales, etc. Companies simply kept records in pen and paper, in stacks of ledgers. Soon these started to fill filing cabinets and these cabinets started to fill rooms and even buildings.

In theory, managing a database should be a relatively easy process. Once you've designed it, normalised your data model, and loaded the data, it should run pretty well on its own. However, theory rarely matches reality. There are, of course, many day-to-day tasks, such as tuning performance and rebuilding indexes when they get lost. These add layers of complexity, which only grow as the database gets larger. Add to this the likelihood that the number of instances you need keeps growing, and you see your time for other tasks rapidly becoming scarce.

You have a solid database architecture. You spent all the requisite time needed making sure your models are normalised in a way to provide the cleanest structures. However, reality being what it is, there are occasional problems.
Rarely is any database complete and perfect the first time it is deployed.

Sure, you can handle any problems that arise at first; every now and then something doesn't run the way it is supposed to. You can see most of what you need through a quick examination of the logs; there are few slow-downs, but for the most part, due to your diligence, you can fix it with a few tweaks. But now the business needs have grown and you're needing to create more and more instances.