Application testing with Apache JMeter

How does the Java-based performance testing tool work?
web development
22 July 2024
A company can save time and money by testing the performance of an application early in the development phase, whether it is a web service, database, or server-side system. One of the best solutions for performance testing is the open-source, Java-based performance testing tool, Apache JMeter, which LogiNet has tested and implemented in medium and large enterprise environments.
Performance testing in the early stages of development is crucial to determine how reliably the system can operate in an end-user environment. The focus is on how the application or web service performs under varying workloads and traffic levels. This helps to detect problems early, prevent potential downtimes due to unexpected loads, and address issues at a lower cost.
For testing, we can choose from numerous tools, both licensed and open-source software. One of the best solutions is JMeter: the free and open-source load and performance testing tool developed by the Apache Foundation. It is suitable for testing the performance of various types of applications, web services, databases, and other server-side systems. It allows for the measurement of response times, load capacity, and other performance metrics, optimizing application performance.

What are the advantages of JMeter?

  • Wide-ranging support: JMeter is easily downloadable from the official website. It supports various protocols and technologies, including HTTP, HTTPS, JDBC, FTP, JMS, LDAP, SOAP, and many others, making it suitable for testing a wide range of systems and applications.
  • User traffic simulation: It allows for the simulation of user traffic, i.e., how an application would respond to multiple users simultaneously or how much load a given server can handle.
  • Functional testing: JMeter is capable of testing not only performance but also functional operations. This means it can automate the testing of HTTP requests, database queries, or other types of requests.
  • Graphical User Interface (GUI): JMeter has a graphical user interface that makes it easy to configure, run tests, and analyze results. Additionally, it offers a command-line mode for test automation.
  • Extensibility: JMeter has a modular structure that allows for extending and customizing its functionality to meet user needs.

Step-by-step guide to effective performance testing

  • Defining the test environment: Before starting the testing of the given system, it is advisable to define the test environment in collaboration with the client. This includes understanding what we know about the users, how and from where they will access the system, their physical connection (connection speed, etc.), what client-side applications they will use, what hardware the server and client have, and what processor and memory they are working with.
  • Defining exit criteria: It is worth determining the exit points using performance metrics, i.e. when we consider the entire process complete. Specific criteria might include that the website can handle 1000 users and still be accessible within an acceptable response time, or that the response time should be under 40 ms even after 500 users.
  • Planning tests: When planning tests, the entire test scenario must be defined: what users will we subject the system to? What user cases should we simulate? What test data will we use? How will we generate these? Here, every detail for later implementation is planned.
  • Configuring the test environment: To subject a system to load, we must prepare the test environment. It is advisable to create an environment as similar as possible, ideally identical, to the production environment to ensure the test provides accurate data. Appropriate monitoring tools need to be selected.

Creating a test

Adding a Thread Group to the Test Plan. This allows you to set parameters such as the number of users, the ramp-up time (e.g., reaching full load in 10 minutes), the total number of iterations, and the duration of the test. Once this is set up, you can add requests (e.g., HTTP requests) under it. To view the results of the test run, you need to add a listener.

JMeter functions

We can organize the test under different controllers according to the logic we want to implement. These can include If Controller, Loop Controller, While Controller, Switch Controller, Transaction Controller, Random Controller, Once Only Controller, and ForEach Controller.
Assertions are one of JMeter's fundamental functions, allowing automatic verification of responses during the test. These can be organized under Requests. Response Assertion enables verification of responses during the test, checking if the responses contain the expected pattern or expression. For JSON testing, there is a separate JSON Assertion that allows verification of JSON responses.
Timers are useful for regulating the timing between requests during tests. They help simulate real-world delays between unexpected requests. You can place them between different Requests or Request groups, for example, to wait 20 seconds before the next Request, wait for a random amount of time, or specify a random time between 10 and 20 seconds before the next Request is executed.
Pre-processors are elements that perform preparatory tasks before Samplers. They allow data preparation, variable setting, HTTP request preparation, etc. These can be written in different languages. JMeter offers various types of Pre-Processors (e.g., User Parameters Pre-Processor, JSR223 Pre-Processor, BeanShell Pre-Processor), each serving different purposes. Pre-processors are generally placed under Sampler elements in the test plan, allowing specific preparatory tasks to be performed before the given Sampler.
Post-Processors are similar to Pre-Processors, but they perform processing tasks after Samplers. They allow response processing, data extraction, and variable setting. JMeter offers various types of Post-Processors, each serving different purposes, such as extracting data from responses (e.g., Regular Expression Extractor, JSON Extractor, XPath Extractor). Post-processors are generally placed under Sampler elements in the test plan, allowing specific processing tasks to be performed after the given Sampler. The most common use of Post-Processors is data extraction from responses, for example, using the Regular Expression Extractor to extract certain data from an HTML or JSON response.
There are different types of Config Elements. For example, HTTP Request Default, User Defined Variables, HTTP Cookie Manager, HTTP Header Manager, and CSV Data Set Config.
The Test Fragment element is a special type of controller that is at the same level as the Thread Group in the Test Plan. It differs from the Thread Group in that it is only executed when referenced by a Module Controller or a Include Controller. This element is exclusively used for code reuse within test plans.
JMeter can be extended with plugins, which can be obtained from the plugins–manager.jar page and used in our tests.
It is possible to record the operation of the site, i.e., capture HTTP or HTTPS traffic - this is done with the Test Script Recorder tool, which can create JMeter test plans. The created test plans can be further refined according to the functionality to be tested.
There are several options for running tests:
  • GUI mode: In this mode, tests can be easily created, edited, and run through a user-friendly interface.
  • Command-line (CLI) mode: This mode allows for the automation and execution of tests from the command line. This can be particularly useful in CI/CD processes.
  • Distributed testing: JMeter tests can be run on one or more machines, which can act as test servers. In this mode, parts of the tests are distributed and run in parallel, which can increase the load and result in better performance.

Analyzing results

Two important metrics are obtained after processing the test results (CSV):
  • Throughput: The capability of the system or process to pass or handle a certain amount of data or processes within a given period.
  • Error rate: The number of failed requests out of the total requests during the test run.
It is possible to generate an HTML report, which can be done from both the GUI and CLI modes.
Report Contents:
  • Summary: A summary of the results
  • Project summary: A short description of the software tested
  • Measurement methodology: Detailed explanation of the measurement methodology
  • Results: Evaluation of the runs, analysis of throughput and error rates
  • Server-side measurements: Graphs from server monitoring applications (e.g., Grafana): CPU, Memory, Disk

Examples of JMeter tests

  • For one enterprise client, the goal was to determine the impact of a threefold traffic increase and the necessary server capacity enhancement. Results indicated the need for database server performance improvements and an additional web server.
  • Another enterprise client required improvement and enhancement of inherited load tests, testing various application parts with multiple tests.
We work with software development solutions that are customised, cost-effective, extensible and secure. Learn more about our technology solutions!

Let's talk about

your project

Drop us a message about your digital product development project and we will get back to you within 2 days.
We'd love to hear the details about your ideas and goals, so that our experts can guide you from the first meeting.
John Radford
Client Services Director UK