Science & Technology (Commonwealth Union) – As artificial intelligence (AI) continues to accelerate to need to enhance it while keeping in check, such as unintended hacking. The enhancement of data mining and simulations for innovations may open a new chapter.
Engineers frequently rely on vision-language models to create innovative designs, including parts for aircraft and vehicles. To evaluate how these components might behave under real-world conditions, they use established computer-aided design (CAD) tools to transform the concepts into detailed 3D models, which can then be tested through virtual simulations such as crash and strength assessments.
Researchers from the Massachusetts Institute of Technology (MIT) and other institutions have developed a new approach that enables vision-language models to automatically translate 2D designs into CAD programs with significantly greater accuracy and functionality than previous methods, while requiring far less computational power.
By making AI-based CAD generation more effective and efficient, the technology could accelerate the prototyping workflow, lower development expenses, and help engineers discover promising design solutions that may otherwise go unnoticed.
The system creates additional training data by analysing the model’s attempts to convert 2D images into CAD code. It identifies and corrects errors made during the process, then combines these improved results with successful outputs to build a stronger dataset for future performance.
The system uses this information to train the model to identify and correct particular errors, while also helping it overcome complex challenges that it may not be able to solve independently.
The lead author Giorgio Giannone, a research affiliate at MIT’s Design Computation and Digital Engineering (DeCoDE) Lab and a principal research scientist on the AI Innovation Team at Red Hat indicated that engineers should be able to apply our framework to a CAD model that is not performing well, define the available computing resources, and allow the system to handle the improvement process automatically — using the model’s own failures as valuable data for future training.
The paper’s authors also include Anna Claire Doris, a mechanical engineering graduate student at MIT; Amin Heyrani Nobari, an MIT postdoctoral researcher; Kai Xu from Red Hat; and co-senior authors Akash Srivastava, director of Core AI at IBM and principal investigator at the MIT-IBM Watson AI Lab; and Faez Ahmed, associate professor of mechanical engineering at MIT, head of the DeCoDE Lab, and principal investigator at the MIT-IBM Watson AI Lab. The study was recently presented at the International Conference on Machine Learning.
Ahmed indicated that almost every physical object we use today, from aircraft to household devices, starts as a CAD model and industries are looking for AI technologies that can accelerate the design process, but current systems often generate basic shapes that are not suitable for real-world applications. He further pointed out that the most exciting aspect of this research is that it enables many image-to-CAD code models to improve through their own experiences, learning from mistakes instead of relying solely on additional human-generated examples and this moves us closer to reliable AI-powered design tools that can become part of everyday engineering workflows.
Researchers are aiming to develop vision-language models (VLMs) capable of generating CAD designs. These models are designed to interpret a 2D image along with accompanying descriptive text and produce Python code that can run within CAD software to create a 3D representation of a real-world object.
The team investigated the difficulties involved in applying current VLMs to CAD generation and found that their performance is primarily restricted by the shortage of diverse, high-quality CAD datasets needed for effective training.
To address this limitation, the researchers explored ways to generate additional training material for teaching AI systems how to create CAD models through a technique called data augmentation.
“We want to obtain data augmentation that is informed by the model itself,” added Giannone.
