Deriving Design Feature Vectors for Patent Images Using Convolutional Neural Networks

The patent database is often used by designers to search for inspirational stimuli for innovative design opportunities because of the large size, extensive variety, and the massive quantity of design information contained in patent documents. Growing work on design-by-analogy has adopted various vectorization approaches for associating design documents. However, they only focused on text analysis and ignored visual information. Research in engineering design and cognitive psychology has shown that visual stimuli may benefit design ideation. In this study, we focus on visual design stimuli and automatically derive the vector space and the design feature vectors representing design images. The automatic vectorization approach uses a novel convolutional neural network architecture named Dual-Visual Geometry Group (VGG) aiming to accomplish two tasks: visual material-type prediction and international patent classification (IPC) section-label predictions. The derived feature vectors that embed both visual characteristics and technology-related knowledge can be potentially utilized to guide the retrieval and use of near-field and far-field design stimuli according to their vector distances. We report the accuracy of the training tasks and also use a case study to demonstrate the advantages of design image retrievals based on our model.