News Stay informed about the latest enterprise technology news and product updates.

TPC-C benchmark gives way to OLTP successor

The Transaction Processing Performance Council has a new transaction performance benchmark that it says is more real-world.

The Transaction Processing Performance Council (TPC) is touting a new online transaction performance (OLTP) benchmark that it says relates more to today's real world.

The TPC-C benchmark has been around since 1992. But servers and databases perform different workloads than they did 15 years ago. To remedy that, TPC is today introducing the successor, TPC-E.

More on server benchmarks
SPEC forms server efficiency committee

SPEC server virtualization benchmark on the horizon

Server energy efficiency standard finalized

The new OLTP benchmark has been in development for five years and takes as its model a brokerage house. As such, it benchmarks a workload using more than three times the number of database tables than TPC-C, representing more transactions happening simultaneously.

"The size of the schema and its complexity has increased and improved," said Michael Molloy, a senior manager for enterprise server products at Dell Inc. who serves as TPC's chairman. "With TPC-C, databases were not as complex as they are now."

The new TPC-E benchmark is also now populated with data from the U.S. Census Bureau, so when vendors run queries on it, real data comes out the other side. With TPC-C, queries resulted in nonsensical, gobbledygook characters.

"You're giving me more metrics and basing it on complex transactions," said Joe Clabby, president of analyst firm Clabby Analytics. "It's designed to use a mult-faceted application to go after various types of data, and assemble it and give me a better transaction performance number."

"The way it was written before was essentially for manufacturing," Clabby added. "It was essentially a database read and database input/output. If they're going with something that is more reflective of what today is, that would be a good thing."

TPC-E certification

TPC is now working to certify auditors on the new benchmark, which should be completed in the next month, according to Michael Majdalany, the group's administrator. For a benchmark result to be published, it has to be approved by an auditor certified by the TPC. Vendors and companies can still use the old TPC-C if they want, as that is likely a metric they've gotten comfortable with over the years, but TPC expects published TPC-E results to start coming out this summer and gain in popularity thereafter.

Major server vendors often tout their performance or price-performance results from benchmarks as a selling point for customers. Aside from the TPC, vendors also rely on benchmarks from the Standard Performance Evaluation Corporation (SPEC), another nonprofit server benchmark firm.

The TPC is also a nonprofit and includes representatives from major IT vendors, such as Dell, Hewlett-Packard Co. (HP) and Intel Corp. The representatives -- there are 18 of them -- sit on a full council that decides the direction of the organization and its benchmarks. For anything major to happen, such as passing a new benchmark, 12 members must vote for it.

Let us know what you think about the story; e-mail: Mark Fontecchio, News Writer.

Dig Deeper on Real-Time Performance Monitoring and Management

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.