<!-- begin content -->
Submitted by Suresh Nageswaran on Mon, 28/02/2005 - 13:04. jMeter | Mercury LoadRunner | OpenSTA | performance testing
[textile]
In _*The Republic*_, _Plato_ conjectured on the idea of the dual level reality. One of these is known as the divided line:
!http://oregonstate.edu/instruct/phl201/images/philosophers/plato/divided_line.gif!
Above the line are the attributes of objective reality; below the line are the attributes of relative reality. This is not very different from user experienced times and response times measured from engineered tests. The problem, then, is to know whether a tool, even a favourite one, tells an objective truth or a relative truth! :)
So here's the real *_masala_* in our performance testing tool shootout. How did the numbers stack up? We ran tests for *1* virtual user (vUser) and multiple iterations, then *8* virtual users and multiple iterations. We stopped at *8* because our server couldn't handle more requests without timing out - apparently *9* is a magic figure. Still, these figures are "all things being equal" i.e. same application, same hardware.
**1 vUser/20 Iterations**
|_. Transaction|_. LR|_. OpenSTA|_. JMeter|
(dark). |MITOURS_01_HOME|1.152|1.16|0.466|
(dark). |MITOURS_02_LOGIN|1.239|0.7985|0.447|
(dark). |MITOURS_03_CLICK_FLIGHTS|1.598|0.6375|0.669|
(dark). |MITOURS_04_SEARCH_FLIGHT|0.4|0.172|0.223|
(dark). |MITOURS_05_SELECT_FLIGHT|0.222|0.2015|0.22|
(dark). |MITOURS_06_PURCHASE_FLIGHT|0.221|0.21|0.222|
(dark). |MITOURS_07_LOGOUT|0.982|0.4|0.449|
**8 VUser, 10 Minutes**
|*Transaction*|*LR*|*OpenSTA*|*JMeter*|
(dark). |MITOURS_01_HOME|1.343|1.247213115|0.561|
(dark). |MITOURS_02_LOGIN|1.384|0.835081967|0.537|
(dark). |MITOURS_03_CLICK_FLIGHTS|1.778|0.720901639|0.736|
(dark). |MITOURS_04_SEARCH_FLIGHT|0.488|0.071065574|0.266|
(dark). |MITOURS_05_SELECT_FLIGHT|0.354|0.258688525|0.235|
(dark). |MITOURS_06_PURCHASE_FLIGHT|0.349|0.305|0.265|
(dark). |MITOURS_07_LOGOUT|1.114|0.444672131|0.48|
**8 VUsers, 20 Iterations**
|*Transaction*|*LR*|*OpenSTA*|*JMeter*|
(dark). |MITOURS_01_HOME|1.323|1.266125|0.5|
(dark). |MITOURS_02_LOGIN|1.427|0.827|0.484|
(dark). |MITOURS_03_CLICK_FLIGHTS|1.761|0.7168125|0.743|
(dark). |MITOURS_04_SEARCH_FLIGHT|0.48|0.0801875|0.248|
(dark). |MITOURS_05_SELECT_FLIGHT|0.312|0.2435625|0.253|
(dark). |MITOURS_06_PURCHASE_FLIGHT|0.364|0.3183125|0.255|
(dark). |MITOURS_07_LOGOUT|1.16|0.4451875|0.481|
Just to find out how the figures stack up, I got MS Excel to spit out a correlation matrix of all the three tool response times for the **8 vuser, 20 iterations** test. Here's what came out.
||*LR*|*OpenSTA*|*Jmeter*|
(dark). |*LR*|1|-|-|
(dark). |*OpenSTA*|0.746747501|1|-|
(dark). |*Jmeter*|0.961989279|0.650244097|1|
*Simple observation, mine*:
Looks like JMeter and LoadRunner tests are very closely interrelated, but OpenSTA figures are off the mark by about 30%. Could it be because of the type of replay engine used by LoadRunner/JMeter? The reference here is to the native socket engine, of course. Might OpenSTA be using a WinInet engine - a look into OpenSTA source should answer that question.
分享到:
相关推荐
java hotspot源码比较JVM实现的性能 这是我的文章的源代码。 怎么跑 Java 8基准run-benchmark-java-8.sh Java 11基准run-benchmark-java-11.sh
Framework - Comparison of DataBinding in Web and Windows Forms Framework - Creating an Enterprise Services Component Framework - How-To Configuration Settings Framework - How-To Environment Settings ...
安卓下拉刷新开源库对比 目前仅比对github上star数>1500的下拉刷新开源库,在比较完成之后可能会加入其它有代表性的库. 本文的demo可以在github上找到。 Repo Repo Owner Star(2015.12.5) ...|:--:|:--:|:--:|:---:|...
In this paper, the performance of voltage-source converter-based shunt and series compensators used for load voltage control in electrical power distribution systems has been analyzed and compared, ...
Security Comparison: BluetoothTM Communications vs. 802.11
Algorithm-sorting-algorithms-performance-comparison.zip,一组排序算法的性能比较它计算每个算法用于排序列表的时间:已排序的数字和未排序的数字,算法是为计算机程序高效、彻底地完成任务而创建的一组详细的准则...
Jax-RS-性能比较Jax-RS实现与嵌入式容器的性能比较 写了一篇文章: 。 他列出了8种最佳的轻量级Java RESTful框架。 就我个人而言,我将jax-rs实现和微框架用于将应用程序部署为微服务,并且这些微服务可以部署在...
Chapter 2:AngularJS and Angular:a Detailed Comparison Chapter 3:An Introduction to TypeScript:Static Typing for the Web Chapter 4:Getting Past Hello World in Angular Chapter 5:Angular Components:...
Ad hoc networks are characterized by multihop wireless connectivity, frequently ... Based on the observations, we make recommendations about how the performance of either protocol can be improved.
细节 目标 该测试目标是显示同步交互和异步交互之间的区别。 例如,在较弱的硬件上,数据库写入速度较慢。 结果,工人在等待同步操作结果时执行的有用性较低的工作。 另外,给出了在golang上执行任务的示例 ...
nlog-log4net-性能比较小型应用程序,用于比较log4net和nlog记录器之间的性能代码用于:
A practitioner's comparison: The 1972 Stanford-Binet and the WISC-R A PRACTITIONER’S COMPARISON : THE 1972 STANFORD-BINET AND THE WISC-R PAUL L. EVANS AND BERT 0. RICHMOND Professional ...
比较Python单元测试工具 该存储库包含非常简单的示例,这些示例允许在实践Python标准模块与和单元测试工具进行比较。 但是,这并不是要对这些工具进行全面的比较。 单元测试 unittest兼容的测试位于文件中,可以从...
二元分类模型比较 在本笔记本中,我使用生成的二进制分类数据比较了Scikit-Learn算法 , , , , , 和简单的ANN在Keras和PyTorch中构建的预测精度。
Using a hands-on approach, you'll learn how to think in programming terms, how to use Objective-C to construct program logic, and how to synthesize it all into working apps. Gary Bennett, an ...
A performance comparison of main stream network simulators, including NS2, NS3, OPNet, OMNet++.
Linux assemblers: A comparison of GAS and NASM A side-by-side look at GNU Assembler (GAS) and Netwide Assembler (NASM).
The third part of the book looks at the measures used to assess the performance, in terms of speech quality and intelligibility, of speech enhancement methods. It also evaluates and compares several ...
A COMPARISON WITH LOW-ACHIEVING AND AVERAGE PEERS KENNETH W. MERRELL Central Washington University This study compared groups of learning-disabled (LD; n = 64),l ow-achieving (LA; n = 46), and ...