Where agentic AI can be leveraged for test system development – lessons learned

Lessons learned from an internal development project for test system applications.

Custom test system SW development-Editing Python with Scope

AI helps test system development but needs an experienced engineer for best results

If we had to describe our experience in one sentence it would be:

Agentic AI can be a useful aid in test code development but it is best used in the hands of an experienced test engineer.

James Campbell, PhD, Client-focused Analytical Engineer, Viewpoint Systems

Our rebuild of Pulsar made clear that agentic AI can easily boost productivity in code development by roughly a factor of 3 to 5. However, the other aspects of application development, such as understanding and documenting requirements, designing, testing, and commissioning, require a human test engineer to do this work. Thus, the overall productivity boost is on the order of 10s of percent, not 100s of percent.

These numbers are rough and based on a single, albeit complicated, development effort and thus may not apply to your situation. But the boost is certainly not the 10X speculated by some people in the past year or so.

Still, even a 10% efficiency boost is worthwhile. And, we expect that over time AI will be able to aid the test engineer increasingly in other areas of the design and build of a test system. Thus, in our opinion, test engineers should definitely be leveraging AI, even if just as an exploration as to what is possible.

Lessons learned from our internal Pulsar-rebuild project

We were especially impressed with the ability to compress the code development effort. To meet the timing and data comms requirements for Pulsar’s RT loop, the code needs specific C++ function calls and certain OS configurations. On an unfamiliar µP, locating this information manually might take a couple weeks of scouring through dense µP and Linux manuals. Agentic AI compressed this effort into a few days.

We also learned some best practices for using agentic AI for test system software development:

Practice: competency in using the LLM and IDE tools takes time.
Design experience: developer knowledge of software development methodologies should be used to create “guardrails” to the design and details output by the agentic AI.
Clear prompts: feeding specific and detailed instructions and requirements is critical to success.
Trust but verify: a test engineer must test the code output by the agentic AI. The code may look correct at first glance but typically contains errors, such as missed edge cases or downright doesn’t work as intended.
Agentic AI cannot solve all problems: Humans still add value in a couple of areas. For example, the engineer can steer an LLM away from “going down a rabbit hole” by losing sight of the “big picture”. Or, as another example, the engineer can recognize when the LLM is “misbehaving” and should be redirected. This situation can happen if the LLM is misinterpreting the prompts and is working on a problem other than the one at hand. (Note that clear prompts will minimize this situation.)

It’s important to point out that only by using agentic AI on a meaningful project will test engineers gain the experience needed to follow the best practices outlined in this bulleted list.

Trust but verify is a required philosophy that must be applied to all AI generated results.

Experience is needed for the best outcomes with AI

Test engineers must gain experience with agentic AI to work best together in order to be successful. The outputs from LLMs behind the agentic AI tool are only as good as the context and detail in the prompts provided to them. To create such prompts, the test engineer requires excellent comprehension of the problem the LLMs are being used to solve. And the result is only as good as the verification to assure correctness. Both these aspects require a test engineer with experience in test system development and associated skills in testing the tester.

In other words, in our opinion, an inexperienced test engineer will likely be unsuccessful in creating a solid test system with agentic AI tools. While they might be lucky in getting a system to function, it’s less likely that the test system will be robust enough for a production environment. For example:

Unexpected inputs may cause errors because the inexperienced test engineer didn’t mention those situations to the LLM.
Flexibility and modularity in the architecture may be missing, making updates and changes difficult as the test system matures.

Test engineers with less experience may be more proficient using agentic AI tools, but they lack the experience in understanding the architectural and contextual details needed for prompt creation, as well as the ability and intuition to perform solid verification and validation of the LLM outputs.

Experienced test engineers may be less familiar with using agentic AI tools and structuring LLM prompts. However, they’ve spent years designing both the hardware and software components of a test system and will excel in architectural design and verification and validation skills.

Here’s a compressed summary of the role of the engineer and AI for each phase of test system development as we see them at the moment:

	Primarily engineer	Engineer interacts with AI	Engineer gives tasks to agentic AI
Requirements Gathering	Experienced engineer digs into the details.	AI can assist experienced engineer by creating a summary of test system needs which can guide the engineer through the details
Architecture and Design	Experienced engineer designs hardware and software components
Coding		Engineer reviews code output by AI for correctness	Experienced engineer works with agentic AI to manage entire coding project
Unit Test		Engineer asks AI to create unit tests for specific code modules
Acceptance testing	Experienced engineer assures that requirements traceability matrix is complete	Engineer can get assist from AI to help outline test procedure (including hardware elements) and organize test code
Documentation		Engineer can ask AI to create BOM for hardware and software components (with version numbers and dependencies)

The statements in this table are brief partly because we expect that AI will continue to improve, perhaps very significantly, in coming years. We expect that the cells in this table will move to the right column over time. The only caveat will be the physical nature of control and measurement hardware that will (for a long time, we think) need a human to select, configure, connect, and test. For test systems that connect to physical things, this hardware is obviously a necessity.

The bottom line is that we believe it behooves test managers to build teams of both inexperienced and experienced test engineers working with AI to grow capability and explore possible use cases to improve efficiency in designing and building test systems.

Leveraging LabVIEW in with AI

As of September 2026, agentic AI for LabVIEW is nascent. The platform, Nigel, is at an early development stage, having been announced in July 2025 and a functional version released in July 2026.

We can expect additional features to be introduced over time, so test managers and engineers should pay attention to Nigel as its capabilities grow. But, even at this early stage, some features make Nigel unique among LLMs and thus should be considered as an important part of a test engineer’s toolbox.

Integration with LabVIEW+ Suite applications

For LabVIEW and test engineering, Nigel can be used for:

in-context advice,
code generation,
and as an AI agent for workflow orchestration.

Furthermore, it can help integrate NI’s hardware into your application, which is a bonus compared to more generic LLMs.

One aspect that makes Nigel attractive for LabVIEW development is that it’s aware of NI’s other platforms in the LabVIEW+ Suite, such as TestStand, VeriStand, and InstrumentStudio. For example, it can parse and edit TestStand sequences.

Data privacy and security protection

Currently (September 2026), Nigel is hosted on Microsoft’s Azure cloud and uses Azure’s OpenAI models. Several data protection practices are employed to ensure that your proprietary data and projects are kept from prying eyes.

Encryption: Prompts are sent to the Nigel Local Service (NLS) on your PC for formatting and encryption via HTTPS. The encrypted data is then sent to the Nigel Cloud Service (NCS) where the prompt is formatted into a query to the LLM, and a response is generated.A similar path in reverse returns the encrypted response.
Not used for training: Client data sent to Nigel is not used as additional training material for the LLM, is not shared with Microsoft, and is deleted from the cloud when a session is closed, including personally identifiable user info.
Data retention: Retention exists only for the session to provide context during the session, and can be removed at any time by deleting the chat.
Data location: Nigel cloud resides only on US-based Microsoft servers.
Responses are focused on test: Nigel has been trained by NI on test system design, methodologies, data analysis, and more, combined with knowledge of NI hardware. Off-topic queries will, as much as possible, result in Nigel telling the user that responses are restricted to test and measurement.

See https://www.ni.com/en/support/security/product-security/ni-nigel-end-user-data.html for more detail.

Next steps

Our experience rebuilding Pulsar convinced us that agentic AI is an important tool for boosting efficiency in building test systems.

We recommend that test managers and engineers cautiously start experimenting with using agentic AI tools in their test applications. But do not expect anything close to the 10X speedups that were bantered about recently.

Discuss with test engineers, manufacturing managers, and test system integrators. AI platforms are advancing rapidly and we can all learn from the experience of each other. If you’d like to chat about leveraging AI for test system development, feel free to reach out. We’re here to help.

Yes, I’d like to chat about leveraging AI for test system development »

In learning mode? Check these out: