论文标题
Conchshell：将图片变成钢琴音乐的生成对抗网络
ConchShell: A Generative Adversarial Networks that Turns Pictures into Piano Music
论文作者
论文摘要
我们提出了Conchshell，这是一种多模式生成的对抗框架，将图片作为网络输入并生成与图片上下文相匹配的钢琴音乐样本。受i3d的启发，我们引入了一种新型的图像特征表示方法：时间跨卷神经网络（TCNN），该方法用于在时间维度中伪造图像的特征。尽管我们的图像数据仅包括六个类别，但我们提出的框架将具有创新性和商业意义。该项目将为工作提供技术思想，例如3D游戏语音Over，短视频配乐以及实时的元式背景音乐。我们还发布了一个新的数据集，即Beach-Ocean-Piano数据集（BOPD）1，其中包含超过3,000张图像和超过1,500架钢琴件。该数据集将支持多模式图像到音乐研究。
We present ConchShell, a multi-modal generative adversarial framework that takes pictures as input to the network and generates piano music samples that match the picture context. Inspired by I3D, we introduce a novel image feature representation method: time-convolutional neural network (TCNN), which is used to forge features for images in the temporal dimension. Although our image data consists of only six categories, our proposed framework will be innovative and commercially meaningful. The project will provide technical ideas for work such as 3D game voice overs, short-video soundtracks, and real-time generation of metaverse background music.We have also released a new dataset, the Beach-Ocean-Piano Dataset (BOPD) 1, which contains more than 3,000 images and more than 1,500 piano pieces. This dataset will support multimodal image-to-music research.
