Automation and management; that is what VMware is concentrating on at the moment. The reasons are clear, as you scale up and move more application load to the “cloud” you have issues with managing the scale of change and you lose visibility to what is happening with both the applications and especially the infrastructure that the applications are running over.
Whether you move your application load to a virtual machine, an Colo outsourcer, a VM container provider like Amazon EC2, an application container provider like Heroku or to a pure SaaS solution like salesforce.com you lose visibility and control . Of course you gain efficiently, scale and the ability to be agile.
Traditional monitoring solutions concentrated on infrastructure, not applications. More and more people are realising that the applications and their dependencies are when they should be worrying about. Agent based monitoring is the only way to get real data when you move your application to a container totally outside of your infrastructure, but what do you do with that data. It soon gets overwhelming trying to make sense of dynamically changing load and containers that get created automatically.
Of course I am going to say that visualisation is the key, I helped create a way to visualise applications and their dependencies. For me, you have to get a handle on what is dependant in order to make sure your users aren’t affected when the inevitable happens and something fails. Also finding outliers and looking for trends is a great way to predict what might cause issues in the future and what can be removed to save resources.
Pretty much all cloud providers provide an API in some shape or form. Software providers are slowly reasling that they no longer can wall off the data they collect because it is only a part of the whole. Integration, correlation and visualisation, all three enable insight.
Humans see patterns and recognise correlations in milliseconds, why not utilise that powerful ability?