Software testing for complex systems

Midair tests software that runs on devices, across networks and on many hosts at once. Every run keeps its configuration, logs and verdicts.

Pick the kind of system you build. The page shows which tests matter for it and what Midair records.

Join the waitlist Pick your system

What software testing for complex systems is.One fault, many conditions.

Software testing checks that a system does what its specification says, under the conditions it meets in use.

A complex system runs on hardware you do not fully control, across several nodes, under limits of time and memory. A defect can depend on the device, the load or the order of events.

So one passing run proves less than it seems. A useful test records the build, the hosts and the configuration next to the verdict, and can run again under the same conditions.

Our article covers the challenges and strategies in more depth: Software testing for complex systems.

Pick your system.See the plan for it.

The test levels, the tools and the record change with the system. Choose the one closest to yours.

Embedded device

Test levels that matter

1. Code analysis
2. Unit
3. Emulator or VM
4. Real hardware
5. Several hosts
6. Load and duration
7. Deployed units

What Midair runs

• Visao checks C, C++ and assembly with your cross toolchain, before the code reaches a board.
• A host next to the board flashes it, switches its power and watches the serial console during each test.
• Delta inspects the firmware root file system offline, or the running device, against a baseline.

What the run records

• Console and kernel logs, with lines such as Kernel panic marked.
• The firmware, suite and test environment commits.
• Expected and obtained verdicts for every test.

Guides for this systemFirmware testing services Embedded systems testing Hardware-software integration testing

Network product

Test levels that matter

1. Code analysis
2. Unit
3. Emulator or VM
4. Real hardware
5. Several hosts
6. Load and duration
7. Deployed units

What Midair runs

• Test Environment coordinates agents on several hosts in one run: the device, its peers and the traffic sources.
• A Wi-Fi test joins the bench network, gets an address over DHCP and fails on any packet loss.
• Delta lists open ports, routes and interfaces on the device and compares them with a baseline.

What the run records

• The bench settings of the run, taken from environment variables.
• Printouts such as round-trip times, attached to the test that made them.
• Ports and services that appeared or went missing since the baseline.

Guides for this systemNetwork hardware testing IoT device testing Cybersecurity testing

Distributed service

Test levels that matter

1. Code analysis
2. Unit
3. Emulator or VM
4. Real hardware
5. Several hosts
6. Load and duration
7. Deployed units

What Midair runs

• One run starts agents on many hosts and drives them in the order the scenario needs.
• Planned runs repeat the same scenario at night or several times a day.
• An AI assistant connected through MCP starts runs and reads failures and log pages.

What the run records

• One result tree across all hosts, with the failures list on top.
• Log pages you open by record number, by time or by test.
• A new record for every start. Midair never writes over a run.

Guides for this systemPerformance testing Test automation Static code analysis

Server or desktop software

Test levels that matter

1. Code analysis
2. Unit
3. Emulator or VM
4. Real hardware
5. Several hosts
6. Load and duration
7. Deployed units

What Midair runs

• Visao analyzes code in nineteen languages, including C, C++ and Rust, and can fail the build on new findings.
• Test agents run on Linux and Windows hosts, so one suite covers both.
• Delta checks installed binaries for hardening such as stack canaries, PIE and RELRO.

What the run records

• Code findings and test failures in one problem list.
• Parameters and error lines for each failed test.
• Findings in SARIF, JSON or GitLab SAST format for your other tools.

Guides for this systemStatic code analysis Static code analysis tools Test automation

Anatomy of a run record.Seven parts, one record.

Every start of a test run makes a new record. Choose a part to see what it holds.

run 518 · node lab-2 · wifi-tsExample

01 / 07

Configuration

The run parameters and the bench settings the run started with. A later edit of the template does not change them.

Why it matters

You tell two runs apart by their settings, not by memory.

02 / 07

Pinned commits

The test suite, the code under test and the test environment, each pinned to a branch, a tag or a commit.

Why it matters

You can run exactly the same thing again months later.

03 / 07

Result tree

Packages and sessions are branches, tests are leaves. Printouts stay attached to the test that made them.

Why it matters

A failure sits in its place in the suite, with its own output.

04 / 07

Expected and obtained verdicts

Each test has an expected verdict in trc.xml. A test passes when the obtained verdict matches it.

Why it matters

A known failure does not hide a new one.

05 / 07

Logs

The raw TE log, turned into text and served in pages by record number, by time or by node of the result tree. An AI timeline and review are optional.

Why it matters

You read the minute around the failure, not the whole log.

06 / 07

Device evidence

Console output, the kernel log and the Delta report from the device: what appeared and what went missing since the baseline.

Why it matters

You see what the device did, next to what the test saw.

07 / 07

Findings

Visao findings on the same commit of the code, in one problem list with the test failures.

Why it matters

Code findings and run results describe the same build.

Questions.Short answers.

What is software testing for complex systems?

It is testing of software that depends on hardware, a network or many hosts. Each result is read together with the conditions that produced it: the build, the devices, the configuration and the timing.

How is it different from testing one application?

One application is mostly tested by input and output. A complex system also needs the test to set up hosts, devices and the network, and to record them, because the result depends on them.

Which systems does Midair test?

Embedded devices, network products, distributed services and server or desktop software. Test agents run on Linux and Windows hosts. A device under test needs no agent: a host next to it reaches it.

Can we start before the hardware is ready?

Yes, in part. Visao analyzes the source without running it, tests can run in QEMU, and Delta can inspect a firmware root file system offline.

Does Midair fit our CI?

Yes. Runs start from your CI, for example with the Jenkins plugin, or on a schedule. An AI assistant can start runs and read failures through MCP.

Does Midair certify our software?

No. Midair produces evidence: run records, verdicts and findings. Test Environment tools can turn a suite and its results into an HTML or PDF report for your own review or audit.

Related pages.Guides and reading.

GuideEmbedded systems testingTest levels from one workstation to a fleet of devices. GuideNetwork hardware testingRouters, switches and gateways under real traffic. GuideEmbedded systems validationBehavior of the whole system across code, devices and runtime conditions. BlogStatic and dynamic code analysis comparedWhat each method finds, and why a complex system needs both. Embedded testing diaryReproducible testsWhat has to be recorded around a run so that a failure can be reproduced.

Start with one system and one suite.Add the rest when it works.

Compare the plans on the pricing page. Prices are monthly, in euros.

Join the waitlist