A conscious effort is underway to explore the paradigm of Set-Based Design (SBD) for development of next generation US Navy ships. The Electric Ships Research and Development Consortium (ESRDC) funded through the Office of Naval Research (ONR) is responsible for developing a state-of-the-art design environment namely, Smart Ships Systems Design (S3D) [1] wherein one focus area is to incorporate SBD functionalities. Impetus and efforts to develop SBD enablers, to be used within a concurrent and collaborative environment like S3D are in its infancy. The fundamental SBD task is of feasible-design space reduction subject to user driven requirements and constraints. Recent work done by authors highlights some potential tools which informs background work to better understand the design domain, impact factors and pertinent metrics, followed by performing design-space reduction [2]-[5]. Well-established techniques used in the product development arena, identified by authors as potential candidates to facilitate SBD are: 1. Quality function deployment, particularly the House of Quality (HOQ) [6] 2. Robust design using reduced factorial Taguchi method (TM) [7] Leveraging these efforts, the next step is to make detailed and rigorous evaluations to better understand the applicability of the identified tools. This paper will illustrate the analysis approach to help reduce the set of feasible-designs with representative examples using devices within the notional breaker-less MVDC architecture [8], [9]. An important collection of outcomes of the proposed methodology are as follows:

• Flexibility to choose design factors that become pivots to progressively reduce the feasible sets.
• Equipment scaling relations to provide visualization of design across power levels.
• Inform further research into adapting the combination of HOQ and TM to explore development of a potentially single or suite of tools as an integrated or stand-alone functionality for S3D.
• Potential for automation of methodology making it a promising option for use within a collaborative and concurrent design environment.

While the work presented in this paper forms a preliminary evaluation using a relatively limited list of devices, future work will aim to expand analysis to cover several energy storage types and power converter system topologies. The output from this work is anticipated to feed into building the S3D device database with useful scaling related information which will enable designers to conduct trade-off studies.

References:
[1] J. Chalfant, M. Ferrante, C. Chryssostomidis, B. Langland, “Task 2: Collaborative System Design Environment: Integration with LEAPS”, Technical Report submitted to the Office of Naval Research, February 2013.
[2] R. Soman, C. Wiegand, T. Toshon et al, “Contribution of Heat Sinks to Overall Size of Modular Multilevel Converters,” Proceedings of ASNE 2017 Intelligent Ships Symposium XII.
[3] T. Toshon, R. Soman, C. Wiegand et al, “Component Level Decomposition Approach to Develop Selection Metrics for Shipboard Power Converter Systems,” Proceedings of ASNE 2017 Intelligent Ships Symposium XII.
[4] T.Toshon, R.R. Soman, C.T. Wiegand et al, “Set-Based Design for Naval Shipboard Power Systems Using Pertinent Metrics from Product Development Tools,” Proceedings of the IEEE Electric Ships Technologies Symposium 2017.
[5] R. Soman, C. Wiegand, T. Toshon et al, “Investigation of Product Development Tools to Aid Naval Shipboard Power Systems Design” Proceedings of the IEEE Electric Ships Technologies Symposium 2017.
[6] X. G. Luo, C. K. Kwong, J. F. Tang and F. Q. Sun, “QFD-Based Product Planning With Consumer Choice Analysis,” in IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 45, no. 3, pp. 454-461, March 2015.
[7] Taguchi, Genichi, “Introduction to quality engineering: designing quality into products and processes.” 1986.
[8] R. M. Cuzner and D. A. Esmaili, “Fault tolerant shipboard MVDC architectures,” 2015 International Conference on Electrical Systems for Aircraft, Railway, Ship Propulsion and Road Vehicles (ESARS), Aachen, 2015, pp. 1-6.
[9] Steurer, M.; Bogdan, F.; Bosworth, M.; Faruque, O.; Hauer, J.; Schoder, K.; Sloderbeck, M.; Soto, D.; Sun, K.; Winkelnkemper, M.; Schwager, L.; Blaszczyk, P., “Multifunctional megawatt scale medium voltage DC test bed based on modular multilevel converter (MMC) technology,” in ESARS, 2015 International Conference on , vol., no., pp.1-6, 3-5 March 2015

It is great to remember the 1980’s as a time when the Navy developed the Aegis Combat System and one of its most successful platforms - the Arleigh Burke class DDG-51. In those days, Hull, Mechanical and Electrical (HM&E) systems development was largely separated from Combat System work. Today, it is clear that the Navy’s surface combatant designs will largely be driven by new, much more power-hungry weapons and sensors. The ship platform solution must be an optimized combination of power generation, conversion, control and another new variable: energy storage. At the same time, the Navy needs to engineer, model, and simulate in accordance with real-world scenarios to ensure power is available for the ultimate combat effectiveness of future systems. Interoperability with the fleet and new fleet assets such as unmanned vehicles present both an opportunity and a challenge. The scale of the problem is intractable as HM&E architecture and topology options in the thousands, combined with operational load cases can produce billions of design and mission scenarios to evaluate for a single platform. (1) The Navy has already made considerable investment in the enablement of Set Based Design via the Rapid Ship Development Environment (RSDE), LEAPS and the ASSET Tools. Looking beyond the HM&E community, more modeling and simulation capabilities have been developed that can and must be leveraged. The question is: how can the Navy institutionalize, industrialize and expand RSDE types of capabilities to a much broader field of experts within the community? This paper will explore the implications of the advent of recent technologies that can help to drive even greater innovation and collaboration across the Navy R&D community: Simulation Data Science, Cloud and platform-based computing. Simulation Data Science is an enabler for innovation. It is a family of applications and technologies to capture and manage information, standardize simulation processes, leverage data analytics, & integrate the practice of simulation into the enterprise. By utilizing the cloud and platform-based tools and integration, Simulation Data Science takes advantage of a variety of core assets common to all applications that sit on the platform. Using a common search engine and database, along with communities, dashboards and other core platform aspects, the solution takes a significant form. Openness is also a key characteristic. Favored legacy tools and compute environments can be preserved and brought forward. Through the use of process applications, Simulation Data Science supports multiple ways to conduct simulation. Users that are doing their work in an ad-hoc manner are supported with minimal disruption but what they have done is captured. Common methods and best practices or processes can be developed that can be standardized within the organization. When process/method development is established, there is an application to help build forms and templates for easy democratization of these processes/methods to others within and outside the organization. By using a simulation dashboard, existing simulations can be reused to address things like variable changes without having to redo all the work done previously. Workflows from methods/processes can be automated using study applications. When automation of design space evaluation (e.g., Design of Experiments) is done, an extensive Results Analytics facility can be used to decipher the resulting cloud to make better and more insightful decisions. This enables faster ways to innovate, develop and democratize simulation for a broader audience within the Navy enterprise. At the same time, it provides a single source of truth that can be governed as loosely or as stringently as needed. Simulation Data Science contributes to Sustainable Innovation and transforms simulation from a specialist role to a broader audience within the product innovation and development process. With the enablement of cloud and platform-based environments along with Simulation Data Science, we see changes such as the centralization of information and assets, capturing and sharing best practices, and having the ability to fully explore design and subsystem variants more efficiently. This enables certain value to be recognized from these changes, such as reduced cycle time and rapid design convergence via democratized simulation within an easily accessible single source of truth. All of this leads to the ultimate goal of realizing program value in more innovative results, better performing systems, better understood risks, increased predictability in the timing and cost of a program, team efficiency and effective operational behavior. Innovation is fundamental to enabling the Navy to achieve its goals. However, it does not simply happen. It will take both the cultural and the operational parts of the organization to be supportive. It also requires putting knowledge and information into the hands of people empowered to use it for wise decision-making. This is overarching goal of Simulation Data Science. The Navy can now leverage the commercial availability of this technology.

(1) Mavris D, Balchanos M, A Decision Support Environment for Integrated Design of Power and Thermal Architectures in All-Electric Naval Surface Combatants , ASNE Intelligent Ships Symposium, May 25, 2017

Set Based Design is a method for discovering insights that inform design tradeoffs, requirements definition, and concept selection. These discoveries are made by first identifying the sets of solutions that represent feasible regions of the design space. Next, those candidate sets are analyzed to derive insights that enable decision makers to understand the relationships between competing objectives, costs, and the design decisions that drive those outcomes. Set Based Design is particularly valuable for addressing critical capability gaps that demand new cutting-edge technologies. The Navy is continually advancing Combat, Power, and Energy Systems (CPES) that require novel solutions to address increased power demands and electrical loads. In turn, this creates the need for evolved modeling and simulation capabilities to address these new domains. The Set Based Design approach requires insight into how requirements, threats, environment, tactics and technical solutions together influence effectiveness and cost. Most importantly, this insight must be used to explore the option space that exists within the bounds of what is both affordable and technically feasible. The process of balancing capability and cost involves generating (via computational models or prototyping) and analyzing many design variants. Accomplishing this task can be especially challenging for novel concepts as the models needed to generate them either do not yet exist or they are not yet validated for those regions of the concept space. What happens when the tools needed to generate the solutions sets do not yet exist? Or what if the tools exist but do not yet provide the desired level of confidence? And what if the suite of tools available represents disparate levels of fidelity? What if the only way to get a complete representation of a design is to accept low fidelity estimates of some characteristics alongside higher fidelity estimates of others? This paper addresses these questions and offers practical solutions for conducting a Set Based Design study that is constrained by the limitations of the tool suite available at the time of the study. The paper will include an example of these approaches being applied to a requirements analysis of a hypothetical frigate design. This example will be used to demonstrate the following key concepts:

• Integrating data from different model types (combining agent based mission effectiveness models with sizing and synthesis models, and subsystem performance models)
• Finding the appropriate balance between model efficiency and model fidelity (trading off the ability to explore a greater region of design space against the desire for greater confidence in the estimations)
• Generating and communicating insights from the data collected

Set-Based Design (SBD) has been developed as an alternative to Point-Based Design. SBD has been successfully used by Toyota in the automotive industry and currently has significant interest in the Department of the Navy and other DoD organizations. SBD defines the design value, develops sets of design solutions that span the design space, evaluates the design sets, delays design decisions to eliminate sets until adequate information is available, and documents the rationale for eliminating sets. Our research focuses on developing trade-off analytics to support SBD decision-making to define value, evaluate sets, incorporate new information, and eliminate sets. Our approach is to align SBD terms with common systems engineering terms; identify the necessary and sufficient conditions for SBD evaluation; develop a broad mathematical formulation for SBD evaluation; and illustrate how trade-off analytics can support SBD decision-making. We report on demonstration models that provide trade-off analytics to support SBD decision-making. We conclude that trade-off analytics provides a consistent and credible tool to implement SBD.

Foundational to Systems Engineering is the ability to inform the decision making process through requirements decomposition and traceability. Historically this process has been plagued with the need to make decisions without the design knowledge to fully decompose requirements and provide traceability. A Set-Based design approach provides a multi-criteria methodology to support systems engineering and enables progressive requirements traceability to properly inform design and requirements decisions as a part of the acquisition process. This paper discusses the role that Set Based Design plays in support of Systems Engineering as used within the Department of Defense. The paper discusses the challenges of point design from a requirements development perspective and the role Set Based Design plays in systems level requirements development. The final discussion is a hypothetical application of Set Based design for machinery systems design and integration in support of a surface combatant acquisition.

Set-Based Design (SBD) has been derived from a Toyota Motor Corporation approach now being applied in other more general engineering design processes. Engineering design processes can be described and classified in terms of the types of reasoning used. The three basic reasoning approaches are inductive, deductive, and abductive. Engineering design has also been described in terms of a composite class called retroductive reasoning. The difference between the reasoning approaches is in what is initially given and what is generated during the application of the process. Our research will define the engineering reasoning behind SBD. Our approach is to present various traditional engineering design processes in terms of engineering reasoning classes, and to create a definition of the SBD reasoning approach. We describe the approaches in the context of decomposition-based design processes typically applied to systems, and formulate the approach used in SBD. To complete our argument, we examine cases in which SBD is employed, and study the degree that our definitions align with the reasoning underlying the design process. The resulting formalized description of SBD will assist engineers in formulating future applications more consistently.

The Netherlands Defence Materiel Organisation (DMO) is responsible for the procurement and sustainment of all weapon systems for all branches of the Netherlands Armed Forces. Within DMO, the Division of Maritime Systems provides engineering support to both the Royal Netherlands Navy and the Defence Staff. The division is involved from day one of a warship procurement project, and supports policy formulation, budgeting, and the requirement formulation for new warships via concept exploration, feasibility studies and costing. To conduct these studies, the DMO has invested in a set of early stage ship design tools to efficiently conduct concept exploration and feasibility studies that benefit from the set-based design methodology. The toolset includes ship synthesis tools, performance prediction tools, costing models, operational models and data-visualisation. Purpose is twofold. First, to conduct an integral exploration of a wide range of alternative warships to learn how requirements, ship design, effectiveness and cost interact, in order to identify desirable alternatives early on. Second, to define the most desirable alternatives in more detail to gain a better understanding of their performance, cost and risk. Several applications were conducted recently in support of the planned renewal of the Royal Netherlands Navy fleet. They included studies into motherships for unmanned mine-countermeasures systems, anti-submarine-frigates, diesel-electric submarines and a combat support ship. The paper will outline the process and tools used by DMO, discusses several of the recent applications and lists important insights gained from these applications.

The purpose of the study is to develop a set-based design framework which integrates high-fidelity physics-based simulation tools early in ship concept design, and to apply it to practical ship design problems. Any ship design methodology, either based on successive refinement of an initial concept (traditional design-spiral based), or innovative methodologies such as set-based [1~3], must address and harmonize a broad spectrum of mission requirements, as well as technical requirements [4~6] considering objectives of cost, risk, and overall mission effectiveness. All this, of course, while ensuring the technical consistency of the design as a whole. Brown et. al. have successfully developed a C&RE (concept and requirements exploration) framework [5] to optimize the concept design of monohull and trimaran naval ships [4]. A great advantage of this C&RE framework is the mathematical reduction of broad design variable sets for convergence to a set of non-dominated (Pareto) designs using set-based methods and a multi-objective genetic algorithm. Discipline-specific explorations are performed independently, following the principles of set-based design. Feasible design space sets and individual discipline response surfaces developed in the discipline explorations are applied and integrated to identify overall feasible non-dominated designs. However, one of the common drawbacks of the current C&RE framework and similar concept design frameworks is that design synthesis and analysis are performed using low-fidelity models with limited accuracy, and the resulting designs and their assessment can be greatly different from designs created and analyzed using high-fidelity analysis [7]. The contributions of the study are several to solve these issues: 1) high-fidelity physics simulations are integrated early in the design through a variable-fidelity response surface, 2) coupled sensitivities are calculated to determine the coupling strengths among the subsystems and for dimension reduction, 3) a Bayesian probability-based fidelity indicator predicts the required fidelity to analyze new design candidate points, and 4) adaptive sampling techniques of filtering and infilling samples are used to further increase the efficiency of the design framework. A brief schematic of the proposed design framework is shown in Fig. 1 with three objective functions with N design variables. In the full paper, variable-fidelity (VF) hydrodynamics analysis methods including linear inviscid panel method, Euler, and URANS solvers are used. CFD solution examples analyzed by the VF are shown in Fig. 2. A Bayesian fidelity indicator [12] and the adaptive sampling criteria [11] were developed in an author’s previous study [12] and will be directly utilized in the full paper. Major modifications to the C&RE frameworks will be made in a full paper which cover 1) the inclusion of the high-fidelity physics simulation per design iteration based on the probabilistic value of the dynamic fidelity indicator, 2) the variable-fidelity response surface, and 3) adaptive infill and filtering sampling criteria. Example application of the design method is chosen for the design of hullform and propular will be carried out which include their flow interactions.

References
[1] Singer, D. J., Doerry, N., Buckley, M. E., “What is Set-Based Design?, ” Naval Engineers Journal, Vol. 121, No. 4, pp. 31–43, 2009.
[2] Doerry N., Earnesty, M., Weaver, C., Banko, J., Myers, J., Browne, D., Hopkins, M., and Balestrini, S., “Using Set-Based Design in Concept Exploration”, SNAME Chesapeake Section Technical Meeting, Arlington VA, Sep 2014
[3] Doerry, N., “Measuring Diversity in Set-Based Design,” ASNE Day 2015. March 4-5, 2015. Arlington VA
[4] Brown, A., J., “Multi-Objective Design of a Trimaran Surface Combatant,” ASNE Sea-Basing Symposium, 23–24 March 2006
[5] Brown, A., J., and Sajdak W. J., (2015) “Still Re-Engineering the Naval Ship Concept Design Process,” Naval Engineers Journal, 127(1):49-61, March 2015
[6] Brown, A. J., (2013) “Application of Operational Effectiveness Models in Naval Ship Concept Exploration and Design,” 3rd International Ship Design & Naval Engineering Congress, 13 - 15 March 2013, Cartagena, Columbia
[7] Allison, D., L., Morris, C. C., Schetz, J. A., Kapania, R. K., and Watson, L. T., (2015) “Reevaluating conceptual design fidelity: An efficient supersonic air vehicle design case,” J. of Aerospace Engineering, July 20, 2015, doi: 10.1177/0954410015595
[8] Bonfiglio L., Perdikaris P., Brizzolara S. (2016) “Multi-Fidelity Optimization of high speed SWATHs”, To appear in Proceedings of SNAME Maritime Technology Conference, Nov. 2016, Seattle (WA).
[9] Bonfiglio L., Vernengo G., Brizzolara S., Bruzzone D. (2016) “A hybrid RANSE – strip theory method for the prediction of ship motions”, Proceedings of 3rd Int. Conference on Maritime Technology and Engineering, Martech, July 2016, Lisbon (P).
[10] http://www.darpa.mil/program/equips
[11] Yi, S., Kwon, H., and Choi, S. (2014) “Efficient Global Optimization using a Multi-Point and Multi-objective Infill Sampling Criteria,” AIAA SciTech, 52nd Aerospace Sciences Meeting, National Harbor, MD, 13th-17th Jan 2014
[12] Jo, Y., Yi, S., Lee, D., and Choi, S., (2016),Adaptive Variable-Fidelity Analysis and Design Using Dynamic Fidelity Indicators, AIAA Journal, Vol. 54 Iss 11 pp. 3564-3579, October14,
[13] Choi, S., Lee, K, and Alonso, J., “Helicopter Rotor Design Using a Time-Spectral and Adjoint-Based Method”, Journal of Aircraft, Vol. 51, Iss. 2, pp. 412–423
[14] Choi, S., Datta, A., and Alonso, J., (2011) “Prediction of Helicopter Rotor Loads Using Time-Spectral Computational Fluid Dynamics and an Exact Fluid Structure Interface”, Journal of American Helicopter Society, Vol. 56, Iss. 4, pp. 042001-1 – 042001-16

Set-based design has grown in popularity and status thanks to several successful implementations of the design method within the Navy. The method is now being considered for application during future ship, submarine, unmanned underwater vehicle, and unmanned surface vehicle programs. However, set-based design is still viewed by some circles of the naval design community with uncertainty, negativity, and as being primarily a buzz word. The goal of this paper is to educate readers on the set-based design method and to discuss the pros and cons of set-based design in the context of a notional ship design study. Two design teams set out to design a notional surface combatant given the same set of design requirements; one team using the set-based design method and the other following the traditional point-based design method. The design study also included two mid-design requirement changes to test the robustness of the respective design methods along with an additional mid-life upgrade requirement. This paper will focus on the adaptability of the set-based and point-based designs to accommodating the mid-life upgrade and comment on the advantages of using set-based design principles for naval ship design.

An emerging concept to overcome the challenges of exploring a large, complex and expensive solution space such as with DoD systems is to employ Set-Based Design tightly coupled to Model-Based Systems Engineering, and to treat the design process formally as a sequential decision process. In this paradigm, designers start with a broad set of potential solutions, use lower fidelity models and analysis techniques in the design process to reduce the size of the set, and then sequentially advance through smaller sets of solutions using models of increasing fidelity that provide tighter bounds until a solution is selected. This paper summarizes recent experience in both empirical application of the method to ongoing design efforts, and theoretical developments in the formalization of Sequential Decision Process.

Today’s complex systems have to meet hundreds of top level product acceptance requirements. It is well known (Stoll, 1999) that decisions taken early in the design process tend to have the largest impact on whether these requirements are met or not. However, current system verification and validation practices focus on comparing virtual prototype behavior with physical prototype behavior, which is not available early in the design process. This significantly increases the risk that errors are found late in the development process, which is the leading cause of program cost overruns (DeWeck 2012). To address this issue, a virtual prototype metric called the Probabilistic Certificate of Correctness (Van der Velden 2012) was developed. This metric computes the probability that the actual physical prototype will meet its benchmark acceptance tests, based on virtual prototype behavior simulations with known confidence and verified model assumptions. To rigorously and efficiently compute PCC it is important to account for all sources of uncertainty in a scalable manner, including model verification and behavior simulation accuracy and precision, manufacturing tolerances, context uncertainty, human factors and confidence in the stochastic sampling itself. The PCC metric was developed as part of the DARPA Adaptive Vehicle Make program (DARPA 2012) and is deployed during the design of the Fast Adaptive Next-Generation Ground Vehicle. To illustrate how PCC is deployed in practice, it was applied it to the verification of a ModelicaTM model that computes the fuel efficiency of a Hybrid Electrical Vehicle with a simulated driving cycle. A ModelicaTM verification package was developed to enable model authors to verify that the modelled or simulated behavior stays within the intended verification bounds and to capture the effects of simulated behavior uncertainty. We will show that the consistent use of such tools has to potential to address the problem of trust in simulation results through a “correct-by-simulation” verification of complex assemblies of behavior models. An automated simulation framework was used to generate the stochastic dynamic behavior samples that are used to compute the PCC metric. This simulation framework submits hundreds of randomized configuration samples to a HPC cluster for efficient calculation of the PCC metric with a known confidence interval. To address ease of use concerns with respect to this complex stochastic simulation process, we deployed a dynamic simulation process flow whereby PCC metrics could be flexibly defined using spreadsheets for any simulation model. This approach removed the need for the end-user to understand the detailed workings of this complex tool. When requirements are certified with deterministic behavior simulations, t twice as many physical prototype cycles may be needed as compared to a product certified with a high PCC (PCC>0.9). Thus PCC is an effective way to reduce overall development cost as well as decrease time-to-market.