First Look SAS Model Manager 3.1

SAS 9.3 shipped a couple of months ago and it included an update of SAS Model Manager 3.1. Model Manager (reviewed here) is a product for managing the analytic model lifecycle once a model is built – validating, deploying, monitoring and retraining predictive analytic models. Model Manager is designed to help with some of the common problems in analytic modeling – time to get them into production, a lack of process for model deployment, information for regulatory compliance, documentation of models and streamlining the process of deploying models once developed. The new release upgrades workflow, model retraining, R support, dynamic metadata and some dashboard usability changes.

The new release has a major upgrade to the workflow – it embeds the SAS workflow engine used in various SAS products like the Customer Intelligence suite and Fraud Framework offerings. Model Manager now ships with a workflow console that allows the definition of a model management workflow. Model Manager ships with a standard set of processes but full editing capabilities allow companies to change these workflow templates, change assignments and so on. There is also a web page for managing instances of the processes and to allow participants to see what their next activity is. This is important as many of the participants in the process may not have any reason to use Model Manager or Enterprise Miner – someone doing a data extract for instance or a DBA may have activities to complete outside of Model Manager and these can be managed too. In theory this would allow you to, for instance, prompt a risk manager to review the cut off rules because a model has been updated in a champion/challenger set up. Tasks support time outs, links to other systems, email remainders etc as you would expect. As well as assigning activities to people, activities in the flow can also invoke SAS jobs to automate steps in the model management process.

In addition the tool has new capabilities in model retraining. Model Manager can now retrain multiple Enterprise Miner models simultaneously and a wizard interface is provided to set up the batch job to retrain the model with updated data. Once the model is retrained, Model Manager provides reporting to compare the performance of the two models. All of this is integrated with the workflow allowing review and deployment steps to be coordinated.

R model support has been improved to allow registration of R models, comparison and reporting against them, scoring and monitoring. You can take the R files you have and register them with Model Manager. Once they are registered they can be retrained, managed with the workflow etc. PMML models and Base SAS models can be registered with Model Manager. Base SAS models can be scored too and work is underway to score PMML and retrain both these kinds of models.

The new release contains significantly more flexibility for defining projects and managing metadata about the projects. This dynamic metadata allows companies to keep track of a richer set of information related to a model or a set of models. In addition there is some more flexibility in the management of dashboards with more selective updates of dashboards. As before all the performance data is available as a SAS dataset, allowing it to be merged with other data sources in, for instance, a performance management environment.