Don't Confuse Validation Builds With a Pilot Run

Lately, we have been receiving a number of inquiries that follow the same pattern. Each one comes with a detailed email and a product requirements document (PRD) that is clearly written by AI. There are no CAD files and no product data, but the request is to start a 50 to 100 unit pilot build within 45 days.

Aside from asking whether product data exists, my first question is whether the product is released for production. In other words, will the product change after the pilot build? The answer is almost always “maybe, it depends on the market feedback.” Right away, I know this is not a pilot build. If the market has not been tested, how can the product go into production?

AI can write a detailed PRD, but it cannot give you the market feedback needed to move into production. It also does not excuse anyone from following the correct product development process. The right process is still validation builds first (EVT, DVT, and PVT), then a pilot run, which tests the production environment.

In this post, I want to clarify the difference between validation builds and a pilot run, and when each one is needed.

Validation builds test the idea and the market

Validation builds prove three things: the idea works, the product can be built, and people actually want it. That last part is easy to skip. Validation units can go to beta testers to find out whether anyone cares about the product before money goes into production tooling.

Each validation build answers a different question.

EVT (Engineering Validation Test). Does the design work? I build EVT units with prototype processes such as CNC machining, 3D printing, soft tooling, and hand assembly. Quantities are small, and testing is functional: does it fit, does it power on, does it do what the PRD says. EVT always produces a list of design changes, and these are the units that go to early users for feedback.

DVT (Design Validation Test). Does the design meet every requirement? DVT parts are made from production-intent CAD, often from first shots off production tooling. Units go through environmental, drop, thermal, cosmetic, and certification testing, and more units can go out to beta testers. The changes should be smaller than at EVT. The goal at the end of DVT is a frozen design.

PVT (Production Validation Test). Can the factory build the product consistently at the planned rate? PVT units are built on the production line with production tooling, fixtures, test stations, and operators. We measure yield, cycle time, and defect types. Design changes at this stage are expensive and should be rare. The output is a product ready for production release.

A pilot run tests production

A pilot run starts after the product is released for production. The tools are already opened and the production line is set up. Now it is time to test everything: find the bottlenecks, and confirm the entire process from material procurement to warehousing runs the way it should.

If a major issue comes up during a pilot run, the line stops and the issue is dealt with right away. The pilot run is not the time to find out whether customers like the product. That question should have been answered during the validation builds.

Side by side

	Validation build	Pilot run
What it tests	The idea, the design, and the market	The production process
Production status	Not released	Released for production
Tooling	Prototype or first shots	Opened production tooling
Expected changes	Design changes	Process fixes
Who gets the units	Engineers and beta testers	Customers

AI does not validate the product

AI is useful, and it makes a lot of work more efficient. At OpusFab, we use AI for instant quotes, DFM feedback, and more. It works for us because it is built on many years of manufacturing data.

What AI does not tell us is whether people will buy a product, or how they feel when they use it. That only comes from putting validation units in front of real users and talking to them.

Conclusion

A pilot run is not the place to find out whether a product works or whether anyone wants it. Validation builds answer those questions. EVT proves the design works, DVT proves it meets every requirement, and PVT proves the factory can build it. Only after the product is released for production does a pilot run make sense, because the pilot run tests the production process, not the product.

If the answer to “will the product change after the pilot build?” is “maybe,” the product is still in validation. Skipping those steps is how a team ends up with production tooling for a product no one actually cares about.

Use OpusFab to get to market faster, with instant quotes and DFM feedback at every build, while still going through every step.