Modern Infrastructure

Staring down the security issues in cloud computing


Storage performance analysis reveals IT's ongoing bottleneck battle

Flash storage improves performance for some users, but a flash upgrade won't help if storage isn't your bottleneck.

Throwing the latest technology at storage problems won't cure every ill.

Flash storage has been the sexy technology of the last few years, with unquestioned abilities to reduce I/O latency. As its cost has come down, adoption has skyrocketed. But some organizations that have invested in flash encounter unexpected issues that have made some ask a difficult question: Were the storage performance gains enough to justify the cost?

The focus is now shifting to more holistic storage performance analysis, leading to adoption of various software, software-defined storage and other approaches that hold hope for helping flash live up to its potential. Experts see growing sophistication of available products and practices, leading to improved efficiency and perhaps enhanced scalability as demands grow.

Tabush Group, a managed service provider located in New York City, has wrestled with that challenge, said CTO Mahmut Sarigedik. In 2006, when the cloud concept gained steam, his company decided to add a private cloud to its data center offerings. That turned out to be a good move, and it brought the company considerable success. Fast forward to 2013: With the services still growing in popularity, his company needed a way to make its storage faster. Moving to flash wouldn't necessarily solve all its problems, and, like many others, the company saw a need to navigate both the promise and the cost of flash while avoiding missteps.

Looking beyond flash

Flash hasn't solved everyone's problems, nor is it reasonable to expect the storage medium to cure all that ails a modern data center. 

"Sometimes it takes more than an aspirin to cure a headache," said Greg Schulz, senior advisory analyst at StorageIO, based in Stillwater, Minn.

The problem, as he sees it, is partly that customers have been disappointed by flash. But he also says vendors are making things seem even worse because they are interested in moving to yet another product cycle or simply hopping on a media-analyst train that's pointing toward newer technologies. It's good for business, in other words.

Fundamentally though, Schulz said, too many IT shops moved to flash storage without really getting answers through storage performance analysis. Sure, flash is almost always going to be faster than disk. But it won't improve overall performance if there are other problems.

"There may be a bottleneck somewhere else -- that could be in hardware, software, storage system architecture or something else," he said.

The solution, in many cases, should be going back to square one to determine why storage performance is problematic, he said.

Scott Sinclair, an analyst at Enterprise Strategy Group in Milford, Mass., agreed, but stressed that flash adoption has been mostly a positive development. "From our research, both quantitative surveys and anecdotal conversations, the No. 1 benefit people are seeing from flash is performance," he said. "I would venture to say the majority of organizations that have employed flash are seeing benefits. I refer to this as experiencing flash nirvana."

For others, though, whether based on existing infrastructure issues or not having adequately modernized a data center, their experience of flash storage performance is different.

Uses for solid-state and flash storage products

One fundamental issue goes back to principles of system design. Every system or process has a bottleneck -- and always will, Sinclair said. "For years, everyone assumed spinning disk was the bottleneck because it's the only thing that isn't 100% electrical," he noted.

In fact, some organizations deployed flash and saw improvements in only some instances, or they deployed it and simply didn't get good results against chosen storage performance metrics. The reason is generally that either the bottleneck wasn't actually spinning disk or that another constraint was exposed, he said. "Where the bottleneck went is often difficult to determine. And even for the flash nirvana people, the bottleneck moved, but performance is still faster than it was, so it doesn't yet matter," he added.

Data center storage evolution

Even where flash meets storage performance expectations, organizations can look forward to more sophisticated vendor offerings on the way. Gartner analyst Julia Palmer said new flash arrays even guarantee performance.

"In the past, deployment was often in some kind of hybrid arrangement. And, in some cases, the fast tier was fast, but the lower tiers would slow way down. When everything was tuned perfectly, you would be able to provide fast performance to the end users," she said. But it could be problematic. With emerging systems, performance is being optimized more across all tiers, she said. And that means fewer storage administrators are hearing complaints about slow storage.

In part, she said, the improvements arise from more and better application and infrastructure performance analytics.

"Modern vendors have cloud-based analytics tools to look for slowdowns elsewhere, beyond the storage; that capability has emerged along with flash," she said. "[Those tools are able to] look down the stack and down to the network interface to make sure everything is configured properly." Overall, it represents a growth in point-to-point analytics over the servers, the storage and everything in between, she explained.

Palmer also notes the emergence of a new style of architecture -- "currently in the hype cycle" -- that she calls shared accelerated storage. Nonvolatile memory express (NVMe) is a specification that helps a solid-state drive make effective use of a high-speed Peripheral Component Interconnect Express bus in a computer. NVMe commands are also now being tapped to move data between a host computer and solid-state storage over Ethernet, Fibre Channel and InfiniBand. That evolution still has elements of hype, Palmer said, but it is likely to trigger the emergence of more converged technology, in which solid-state fits effectively.

"The next-generation integrated systems or hyper-converged systems will emerge to provide this kind of end-to-end solution, and they will be finely tuned to eliminate bottlenecks,” said Palmer. And, she noted, you can't continually improve storage without ultimately bringing it closer to the compute layer.

That's just what's occurring, said Sinclair, who sees new storage startups beginning to design with an architecture oriented toward speed and processor proximity.

Other companies are also addressing latency and the challenges of flash by other means. Coho Data, for instance, has introduced more advanced networking capabilities into storage environments to reduce or eliminate bottlenecks. It turns out that's the direction that Sarigedik at Tabush chose to go. Determined to deliver almost everything Tabush offered customers through a cloud model, the company tackled storage performance analysis, hunting for a storage setup that was fast.

"We were talking about all-flash with our SAN [storage area network], but we wanted to be top-notch, with response in the 1 to 10 millisecond range and preferably around 5 milliseconds," he said.

All-flash options were "pretty expensive," Sarigedik explained. And there were questions regarding its ability to deliver. He eventually selected Coho, which provides converged infrastructure combining software-defined networking, software-defined storage and commodity hardware, including flash storage.

The balanced and optimized approach to storage performance enabled by Coho reduced the deployment's latency by 80%, Sarigedik said.

Not everyone is ready or able to make a big investment in flash-related performance analytics. Schulz said there are plenty of ways to try to cure those problems without big initial outlays.

"You can get a lot of insights on bottlenecks and other performance issues from tools built into VMware, Microsoft and Linux without spending anything," Schulz said.

However, he noted, many organizations may still face a skills gap. "It depends how deep you want to go; many tools can provide good, basic insights without demanding a lot of expertise," he said. For further depth, "you may need to pick up the phone and call on an expert."

Article 2 of 3

Next Steps

Flash changed enterprise storage. Now what?

Differentiating between all-flash array options

Future flash technologies: NVMe and more

Dig Deeper on Real-Time Performance Monitoring and Management

Get More Modern Infrastructure

Access to all of our back issues View All