A newly developed securitization app is the canary in the coal mine for a DevOps implementation at one major U.S. enterprise financial services company.
The Federal National Mortgage Association, known as Fannie Mae, is jointly developing securitization products with the Federal Home Loan Mortage Corporation ("Freddie Mac")*, and the applications behind this effort are leading the charge in Fannie Mae's DevOps implementation.
It's not mandatory for Fannie Mae to adopt DevOps capabilities, as it has a complicated infrastructure comprised of 400 applications, a mix of new apps and monolithic ones that have been around for a decade or more. Over 100 app development teams are responsible for the portfolio, and each app may undergo three big modification projects at a time.
"We drive adoption as it makes sense on an app-by-app basis," said Jason Anders, IT leadership for the securitization portfolio at Fannie Mae in Washington D.C.
Selling a DevOps implementation with results
Fannie Mae used the greenfield development opportunity of the new securitization app to learn lessons on how to build out DevOps capabilities, including automated infrastructure deployment, continuous delivery and continuous integration (CI/CD), test data management, code quality analysis and test automation.
Creating these capabilities for a greenfield application from scratch taught Anders' DevOps team how to prioritize rollout mechanisms and DevOps implementation processes to help things go smoothly and to act as a model to show other teams what DevOps can do.
"It's been going on for about a year and a half and we're reaching a critical mass point where people are really lining up," Anders said.
Jason AndersIT leadership for the securitization portfolio at Fannie Mae
In that lighthouse project, it normally would've taken two to three days to deploy the application's code to the relevant test environments and production. The DevOps implementation has shortened that to about 45 minutes using a CI/CD process.
Using CI/CD based on Jenkins, the greenfield application team builds and deploys code seven or eight times a day, Anders said. Each time the team deploys that code, they run it through a suite of automated test cases simultaneously and get a good look at code quality every time.
"That's something that almost no other application can touch within our enterprise," Anders said. That also has helped his team advertise DevOps to the rest of the company. "When you show people this type of result, they're quick to want it," he said.
The DevOps organization's selling methodology is to go application team by application team, and show its results with the greenfield app. Next, it works with the application team to plan what layers of the DevOps tool stack make sense for that team's application to adopt first.
These decisions are based on a questionnaire, which asks:
- What initiatives the app team has coming up,
- How many changes and releases are planned in the next 18 months,
- What the current test automation suite looks like and how well it's documented,
- What test data capabilities the team already has, and
- How much pain it sees in the test data space.
"Depending on the answers to these questions, maybe the CI/CD tools are the best ones to start with," Anders said. "Or we start with automating test cases, then work on some of the other prerequisites for the other new capabilities."
Test data management tools ease storage infrastructure sprawl
The test data management process and the sprawling storage capacity that's been built up over the last decade are currently undergoing a major overhaul at Fannie Mae as it automates tests.
The DevOps organization uses a combination of tools to accommodate multiple major ways to deliver data for a test program. If the app team tests against a large database, tools from Informatica create a small subset of that data. Masked full copies of data and synthetic data are handled by Delphix.
Anders estimated that two-thirds of the data that companies pay for is in test environments, and Fannie Mae is no exception. By using data virtualization tools, Fannie Mae expects substantial annual savings on storage costs, beginning with compression savings of 40%. Once more application teams sign on to use virtualized test data, that savings could increase to as much as 75%.
Ultimately, "we're putting ourselves in a position to be much easier to work with" for loan originator and servicer partners as well as customers, Anders said.
"We can be much more flexible, much more dynamic, and provide them with the tools that they need to interact with us far more easily," he said. However, "like anybody else, we've got to get ideas to production a lot faster than we're getting them there today."
*the original wording of the sentence was as follows: "... is developing joint securitization products with its sister corporation Freddie Mac ..."
A request from production IT teams
Shared tools make for DevOps in reality
How to monitor DevOps apps in production