Frequently Asked Questions

The two answer different questions.

Checkmk monitors systems: Is the server up? Is the database reachable? How full is the disk? Every light can be green while the business reports that checkout in the shop has been stuck for two hours.

Robotmk closes exactly that gap. It brings synthetic monitoring to Checkmk: automated tests that drive applications the way a real user would — log in, click, submit the form, verify the result. What is monitored is then no longer only the infrastructure, but the user experience.

No second tool is required for this: test results appear as regular services in Checkmk — with the same rules, notifications and dashboards as every other service.

Still open? Submit the question via the chat widget. Or book a free clarity call →

They are two separate projects that work together closely.

Checkmk is developed and sold by Checkmk GmbH. Synthetic Monitoring is an integral part of the product there — including support and documentation.

robotmk.org is this site: an independent project run by ELABIT GmbH. I invented Robotmk and work part-time as a product manager at Checkmk on the integration. In parallel, I use Robotmk as a consultant in customer projects.

That dual role is the essential point: field experience from projects flows back into product development — and the resulting product knowledge is passed on here.

Still open? Submit the question via the chat widget. Or book a free clarity call →

No. Robotmk is part of Checkmk and is licensed exclusively through checkmk.com — Synthetic Monitoring is a subscription extension to the Checkmk installation.

This site provides the surrounding context: knowledge, field experience and, on request, support with the rollout.

Still open? Submit the question via the chat widget. Or book a free clarity call →

Not any more. Robotmk started as an open source project on GitHub and was for years one of the most installed Checkmk extensions.

With the integration into Checkmk it became a commercial product, developed, maintained and supported by Checkmk GmbH. The earlier open source version (v1) is no longer being developed.

The Robot Framework world itself — the framework and its entire library ecosystem — is untouched by this and remains fully open source.

Still open? Submit the question via the chat widget. Or book a free clarity call →

That is understandable — the decision was not an easy one.

The background: a project of this size requires dependable development, tested releases and reliable support. That cannot be done on the side. A substantial part of today’s feature set — the deep Checkmk integration, automatic environment management, robust result processing — would not have come into existence as a spare-time open source package.

The commercial basis is therefore not the price paid for less openness, but the precondition for the current level of functionality and quality.

Still open? Submit the question via the chat widget. Or book a free clarity call →

None. This expectation is common, so to be explicit:

Robotmk, or Synthetic Monitoring, is the solution for integrating Robot Framework test results into Checkmk. Writing those tests remains the customer’s responsibility — and appropriately so, since knowledge of the relevant business processes sits within the organisation.

That does not mean starting from a blank page: the Robot Framework ecosystem provides ready-made libraries for practically every technology, and robotmk-starter contains CI-tested example test cases as a starting point.

Still open? Submit the question via the chat widget. Or book a free clarity call →

A number of suitable entry points are now available:

• Checkmk documentation — the authoritative reference for Synthetic Monitoring.
• Checkmk YouTube channel — including “Getting Started with Synthetic Monitoring” (set up a test client, write a first test) and “Integrating Robot Framework tests in Checkmk” (configure Robotmk, discover services, set up monitoring).
• Robot Framework Slack — a notably helpful community.
• This blog — tutorials, best practices and news from the field.
• robotmk-starter — CI-tested boilerplate test cases for reference.
• Robotmk Academy — my online course, tailored to Synthetic Monitoring with Robot Framework and Checkmk. It also prepares for the RFCP exam (Robot Framework Certified Professional). Details under Services.

Still open? Submit the question via the chat widget. Or book a free clarity call →

No. This is a deliberate decision, not a missing feature.

The question usually stems from reservations about “programming”. Robot Framework has no graphical test interface; tests are written as code. Thanks to the keyword syntax, however, that code is exceptionally readable — one of the reasons Robot Framework came about in the first place.

The argument against a recorder lies in how a test is structured. It consists of three layers:

• the test logic (what needs to be done),
• the test technique (how it is implemented),
• the test data (what inputs are used).

A recorder conflates all three. The result looks serviceable at first but does not scale: every similar test has to be recorded again, and any change to the application, the logic, the technique or the data requires re-recording all affected tests. The codebase grows unmaintainable quickly.

Robot Framework code can instead be layered: the top keyword layer describes only the logic (“WHAT is being tested?”), the layer below it the sub-steps with the test data, the bottom layer the technical implementation. Changes then affect exactly one layer.

An analogy from hi-fi illustrates the principle: a system built from separate components — source, amplifier, speakers — assigns one job to each component. Each can be repaired, replaced and modernised, and such systems last accordingly. Test recorders correspond to all-in-one units: convenient at first, but barely repairable once something fails, because everything is built into one box.

Still open? Submit the question via the chat widget. Or book a free clarity call →

A suite is a text file containing Robot Framework code describing one or more tests (also called test cases). Robotmk executes these suites on the monitored hosts — on Windows as well as Linux.

Suites serve primarily to group test cases.

Still open? Submit the question via the chat widget. Or book a free clarity call →

A test is a single scenario that checks one specific aspect of an application. A suite can contain multiple test cases, which are executed one after the other.

Test cases should be kept as granular as possible: one test case = one scenario. This facilitates maintenance, makes the results more meaningful, and allows for more targeted alerting.

Still open? Submit the question via the chat widget. Or book a free clarity call →

Keywords are the building blocks that make up tests. They behave like functions in a programming language: they take parameters, can return values, and can be nested to any depth.

That nesting is precisely what makes Robot Framework tests readable and maintainable over time.

Still open? Submit the question via the chat widget. Or book a free clarity call →

The Robotmk check on the Checkmk server evaluates the raw data from the Robot Framework executions and discovers a dedicated service for each test — the test service.

This service reflects the state of the test (successful / failed), contains its human-readable HTML logfile, and can also alert on runtime exceedances.

The sum of all test services within a Checkmk subscription determines the scope for Synthetic Monitoring — it is the only figure that matters for licensing.

Still open? Submit the question via the chat widget. Or book a free clarity call →

Windows as well as Debian/Ubuntu test clients are supported, with at least 4 CPU cores and 8 GB RAM.

The client must be monitorable by Checkmk and should have internet access so the required Python environments can be built automatically.

The current, authoritative requirements are in the Checkmk documentation.

Still open? Submit the question via the chat widget. Or book a free clarity call →

All tests that drive the graphical interface of a locally installed application — SAP GUI, Citrix clients or classic Windows software, for example. Without a logged-in, unlocked user session there is no operable interface there. That is not a Robotmk limitation but a basic technical condition.

Anything that does not require a GUI runs headless — that is, without a desktop session. This includes web tests: Firefox, Chromium and WebKit can all run in headless mode.

Still open? Submit the question via the chat widget. Or book a free clarity call →

v1 was the community extension: an MKP installed and maintained independently. Python environments on the test clients had to be set up and kept current by hand.

v2 is Synthetic Monitoring as an integral part of Checkmk. The key differences:

• Built in, not bolted on — configured through the regular Checkmk rule management, no separate MKP.
• Automatic environment management via RCC: each suite declares in a YAML file what the test requires; the matching environment is created on the client automatically and reproducibly.
• Official support and documentation from Checkmk GmbH.
• Considerably more robust execution control and result processing.

v1 is no longer being developed.

Still open? Submit the question via the chat widget. Or book a free clarity call →

Incrementally — no cut-over date is required.

The v1 MKP can be installed on Checkmk 2.3+ in parallel and does not interfere with the integrated v2. Migration can therefore proceed suite by suite while the remaining estate continues unchanged.

The Robot Framework suites themselves need no changes. The only additional step: an environment specification (YAML) has to be created per suite so the matching environment can be generated automatically.

Still open? Submit the question via the chat widget. Or book a free clarity call →

Yes. Up to three test services can run in Checkmk without a subscription — in the so-called “free tier” that is enough to evaluate Synthetic Monitoring with real tests in one’s own environment at no cost.

One point to note: this allowance is a trial offer, not a permanent discount. Once a Synthetic Monitoring subscription is taken out, the three free test services no longer apply. From that point the usage counts as productive — and since every productively monitored test service represents delivered value, each one then counts towards the scope of the subscription.

The terms currently in force are documented on checkmk.com.

Still open? Submit the question via the chat widget. Or book a free clarity call →

Nothing — apart from a higher resolution in the test data.

This is precisely where costs rise with action-based competitors: there, billing is per test run or per action. With Checkmk the cost stays constant, because only the number of test services counts.

The significance of that difference is worked through in detail on Why Robotmk.

Still open? Submit the question via the chat widget. Or book a free clarity call →

Two figures from the existing system are sufficient:

1. the number of Metriken or states it delivers today, and
2. the average runtime of the underlying tests.

The first figure corresponds approximately to the number of test services required in Checkmk — and therefore to the scope of the subscription. Nothing further is needed for a sound initial estimate.

I’d be happy to help translate the figures into a specific Checkmk subscription during a Clarity Call.

Still open? Submit the question via the chat widget. Or book a free clarity call →

Locations are not a licensing dimension — no more than the number of Robotmk hosts in Checkmk. Billing is neither per location nor per host.

What counts is exclusively the total number of test services within a subscription. If the same suite runs at several locations, each will produce its own test services — but the price still follows this one, readily predictable figure.

Still open? Submit the question via the chat widget. Or book a free clarity call →

No.

The execution interval of the Robot Framework suites is completely independent of the Checkmk agent’s check interval. The agent merely collects whichever results were last available; how often it does so has no bearing on licensing.

Still open? Submit the question via the chat widget. Or book a free clarity call →

Yes, entirely. Robot Framework and its libraries are open source and freely available — browser automation, SAP, Citrix, REST, databases and much more. Getting started is possible immediately, without any prior purchase.

A subscription is required only for Robotmk — the integration layer that connects Robot Framework tests to Checkmk. Up to three test services can be run without a subscription for evaluation.

Still open? Submit the question via the chat widget. Or book a free clarity call →

Still open? Submit the question via the chat widget. Or book a free clarity call →