What should the role of experiments be in ICME?

While one goal of ICME is to reduce the time and cost of generating test data, I think ICME will itself depend on experimental data, particularly in the near term. Experiments will be needed to provide modeling inputs (model parameters and boundary conditions), to assess uncertainty, and ultimately validate ICME methods and tools. Also, these experiments will be motivated by need to identify materials mechanisms and uncover relationships using DOE to complement model development efforts. It's important that we continue to pursue resources to strengthen the laboratory facilities and expertise needed to design and carryout experiments and characterization that generate high quality data, even as we continue to foster ICME maturation.

Experimental validation is important for any modeling study.

But at least as important is a strong connection to experiments at the outset to ensure that one is asking the right questions to begin with. Without this, one can go off and "do modeling" of something interesting and generate completely irrelevant results, or make assumptions which are not grounded in the physical reality. Either way, the outcome is wasted effort.

One analogy is the use of data for neural networks. You need a training set of experimental data to develop the model, and a second set to test/validate it. With physics-based models as well, it is imperative to have a good sense of how the model "should" behave before starting, for which one needs a close connection to experiments.

This is especially important for ICME, where we couple models at different lengthscales. When we add model to model, errors can propagate and grow, and it's essential to have experimental data with which to benchmark the results at every level from the outset.

A problem we're facing is statistical model validation. One of my "Design of Experiments" texts advised limiting the cost of the first set of experiments to no more than 25% of your budget so you had money left when you discovered the lurking variable you hadn't planned to address in the first iteration. Are there rules of thumb in terms of how much of the total available time and money should be allocated to model validation vs. model development? For those in industries not supported by tax dollars--how have you tackled model validation on limited budgets, particularly when moving toward a reliability based product performance model that requires validation of a model that is sensitive to tails of the statistical distribution of the inputs?