Machine Learning for Creativity and Design

NeurIPS 2023 Workshop

Dec 16 2023, in-person (New Orleans)

Image credit: Salvaging the beauty left behind (2022). Keito Takaishi, Asuka Ishii, Kazufumi Shibuya, Nao Tokui.

If you have any questions, please contact us at

You can also follow us on Twitter for updates: @ML4CDworkshop


Machine co-creativity continues to grow and attract a wider audience to machine learning. Generative models, for example, have enabled new types of media creation across language, images, and music including recent advances, such as large language models like ChatGPT/Bard/Claude, high quality text to images models like Stable Diffusion/MidJourney, plus AudioLM and MusicLM. This one-day workshop will broadly explore topics in the applications of machine learning to creativity and design, which includes:

We will aim to balance addressing the technical issues and challenges of applying the latest Machine Learning models and techniques to creativity and design with the philosophical and cultural issues that surround this area of research.

The goal of this workshop is to bring together researchers and artists interested in exploring the intersection of human creativity and machine learning. This workshop solicits viewpoints from artists and technology developers to look beyond technical issues to better understand the needs of artists and creators.

Invited Speakers


The following schedule is tentative.

Time (CST) Duration Event
8:15 AM 0:15:00 Welcome and Introduction
8:30 AM 0:30:00 Invited Talk 1 - Tianwei Yin
9:00 AM 0:25:00 Invited Talk 2 - Misha Konstantinov & Daria Bakshandaeva (remote)
9:30 AM 0:25:00 Invited Talk 3 - Alexander Mordvintsev & Ettore Randazzo (remote)
10:00 AM 0:30:00 Art gallery / Coffee Break / Social
10:30 AM 0:30:00 Panel Discussion
11:00 AM 1:00:00 Paper & Artwork Spotlights
12:00 PM 1:00:00 Lunch
1:00 PM 0:30:00 Invited Talk 4 - Aleksander Holynski
1:30 PM 0:25:00 Invited Talk 5 - Richard Zhang (remote)
2:00 PM 0:30:00 Invited Talk 6 - Cristóbal Valenzuela
2:30 PM 0:30:00 Panel / Open Discussion
3:00 PM 0:30:00 Art / Coffee Break / Social
3:30 PM 1:00:00 Poster session
4:30 PM 0:10:00 Conclusion
4:40 PM 1:00:00 Free-formed discussion

Accepted Papers

This year we accepted 32 papers from a total of 80 paper submissions, resulting in a 40% acceptance rate. Among accepted papers, 7 are selected as spotlight papers, marked with * below.

# Title Authors
1 Latent Painter Su, Shih-Chieh
2 * Minecraft-ify: Minecraft Style Image Generation with Text-guided Image Editing for In-Game Application Kim, Bumsoo; Byun, Sanghyun; Jung, Yonghoon; Shin, Wonseop; Ul Amin, Sareer; Seo, Sanghyun
3 BioSpark: An End-to-end Generative System for Biological-Analogical Inspirations and Ideation Kang, Hyeonsu; Lin, David Chuan-En; Martelaro, Nikolas; Kittur, Aniket; Chen, Yan-Ying; Hong, Matthew K
4 Interactive Machine Learning for Generative Models Shimizu, Junichi; olowe, ireti; Broad, Terence; Vigliensoni, Gabriel; Thattai Ravikumar, Prashanth; Fiebrink, Rebecca
5 * SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions Lee, Yuseung; Kim, Kunho; Kim, Hyunjin; Sung, Minhyuk
6 * Real-time Animation Generation and Control on Rigged Models via Large Language Models Huang, Han; De La Torre Romo, Fernanda; Fang, Cathy Mengying; Banburski, Andrzej; Amores, Judith; Lanier, Jaron
7 The Interface for Symbolic Music Loop Generation Conditioned on Musical Metadata Han, Sangjun; Ihm, Hyeongrae; Lim, Woohyung
8 Weaving ML with Human Aesthetic Assessments to Augment Design Space Exploration Jeon, Youngseung; Hong, Matthew K; Chen, Yan-Ying; Murakami, Kalani; Li, Jonathan; Chen, Xiang ‘Anthony’; Klenk, Matthew
9 * Envisioning Distant Worlds: Training a Latent Diffusion Model with NASA’s Exoplanet Data Beaty, Marissa; Broad, Terence
10 Zero2Story: Novel Generation Framework for Anyone Park, Chansung; Lee, Youngbin; Han, Sangjoon; Lee, Jungue
11 On the Distillation of Stories for Transferring Narrative Arcs in Collections of Independent Media Ashley, Dylan R; Herrmann, Vincent; Friggstad, Zachary; Schmidhuber, Jürgen
12 Breaking Barriers to Creative Expression: Co-Designing and Implementing an Accessible Text-to-Image Interface Taheri, Atieh; Izadi, Mohammad; Shriram, Gururaj; Rostamzadeh, Negar; Kane, Shaun
13 Multi-Subject Personalization Jain, Arushi; Paliwal, Shubham Singh; Sharma, Monika; Jamwal, Vikram; Vig, Lovekesh
14 CalliPaint: Chinese Calligraphy Inpainting with Diffusion Model Liao, Qisheng; Wang, Zhinuo; Abdul-Mageed, Muhammad; Xia, Gus
15 CAD-LLM: Large Language Model for CAD Generation Wu, Sifan; Khasahmadi, Amir Hosein; Katz, Mor; Jayaraman, Pradeep Kumar; Pu, Yewen; Willis, Karl D.D.; Liu, Bang
16 Unrolling Virtual Worlds for Immersive Experiences Tikhonov, Alexey; Repushko, Anton
17 Setting Switcher: Changing genre-settings in text-based game environments populated by generative agents Wood, Oliver H; Fiebrink, Rebecca
18 * LEDITS++: Limitless Image Editing using Text-to-Image Models Brack, Manuel; Tsaban, Linoy; Kornmeier, Katharina; Passos, Apolinário; Friedrich, Felix; Schramowski, Patrick; Kersting, Kristian
19 V2Meow: Meowing to the Visual Beat via Music Generation Li, Judith Yue; Su, Kun; Huang, Qingqing; Kuzmin, Dima; Lee, Joonseok; Donahue, Chris; Sha, Fei; Jansen, Aren; Wang, Yu; Verzetti, Mauro; Denk, Timo I
20 DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models Wang, Tsun-Hsuan; Zheng, Juntian; Ma, Pingchuan; Du, Yilun; Kim, Byungchul; Spielberg, Andrew; Tenenbaum, Joshua; Gan, Chuang; Rus, Daniela
21 An Ontology of Co-Creative AI Systems Lin, Zhiyu; Riedl, Mark
22 ObjectComposer: Consistent Generation of Multiple Objects Without Fine-tuning Helbling, Alec F; Montoya, Evan; Chau, Duen Horng
23 HARP: Bringing Deep Learning to the DAW with Hosted, Asynchronous, Remote Processing Flores Garcia, Hugo F ; Benetatos, Christodoulos; O’Reilly, Patrick; Aguilar, Aldo; Duan, Zhiyao; Pardo, Bryan
24 Contextual Alchemy: A Framework for Enhanced Readability through Cross-Domain Entity Alignment Shahid, Simra; Srikanth, Nikitha; Jandial, Surgan; Krishnamurthy, Balaji
25 JamSketch Deep α: Towards Musical Improvisation based on Human-machine Collaboration Kitahara, Tetsuro; Yonamine, Akio
26 Hacking Generative Models with Differentiable Network Bending Aldegheri, Giacomo; Rogalska, Alina; Youssef, Ahmed; Iofinova, Eugenia
27 Lasagna: Layered Score Distillation for Disentangled Image Editing Bashkirova, Dina; Ray, Arijit; Mallick, Rupayan; Bargal, Sarah; Zhang, Jianming; Krishna, Ranjay; Saenko, Kate
28 CommonCanvas: An Open Diffusion Model Trained with Creative-Commons Images Gokaslan, Aaron K; Cooper, A. Feder; Collins, Jasmine; Seguin, Landan; Jacobson, Austin; patel, Mihir; Frankle, Jonathan; Stephenson, Cory; Kuleshov, Volodymyr
29 * Personalized Comic Story Generation PENG, WENXUAN; Schaldenbrand, Peter; Oh, Jean
30 Combating the “Sameness” in AI Art: Reflections on the Interactive AI Installation Fencing Hallucination Qiu, Weihao; legrady, george
31 SynthScribe: Deep Multimodal Tools for Synthesizer Sound Retrieval and Exploration Brade, Stephen; Oore, Sageev; Wang, Bryan; Sousa, Mauricio; Grossman, Tovi; Newsome, Greg
32 * WordArt Designer API: User-Driven Artistic Typography Synthesis with Large Language Models on ModelScope HE, JUN-YAN; Cheng , Zhi-Qi; Li, Chenyang; Sun, Jingdong; Xiang, Wangmeng; Hu, Yusen; Lin, Xianhui; Kang, Xiaoyang; Jin, Zengke; Luo, Bin; Geng, Yifeng; Xie, Xuansong; Zhou, Jingren

Accepted Artworks

This year we accepted 27 artworks to the gallery from a total of 52 artwork submissions, resulting in a 51.9% acceptance rate. Five exceptional artworks were awarded a spotlight, where they have the option to speak 5 min about the artwork during the workshop. All artworks will also be shown at the workshop in the form of an art show video screening.

Special thanks to the artwork jury: Eunsu Kang, Terence Broad, Yvonne Fang, August von Trapp, Yingtao Tian, Lia Coleman, Tom White, and Hannah Johnston.

Artwork Spotlights

Artwork Title Artist Names
Visions of Destruction Canet Sola, Mar*; Guljajeva, Varvara
Unreal Pareidolia -shadows- Allen, Scott*
Beneath the Surface Ozoani, Ezinwanne*; Tsaban, Linoy
mememormee Pettee, Mariel*
Scores for the Virtual Epstein, Vadim*; Winkler, Christoph

All Artworks

Artwork Title Artist Names
EvoGen: Evolutionary Diffusion Model Prompt Generation Petersen, Magnus*
Envisioning Distant Worlds Beaty, Marissa*; Broad, Terence
Movies Never Made Wilke, J. Brad*
mememormee Pettee, Mariel*
Gender Tapestry Rosenbaum, J*
Like a natural phenomenon Shibuya, Kazufumi*
THE RAVEN Lu, Wei*; Luo, Xuehui
Mega Giga Kazuki, Abe*
Potential Form Kazuki, Abe*
Attraction of image Ishii, Asuka*; Nagata, Kazuki
Tick Tock Epstein, Vadim*
Scores for the Virtual Epstein, Vadim*; Winkler, Christoph
Neuracappella Ng, Zihou*
GPT-ME Meshi, Avital*
Peekaboo Meshi, Avital*
SURVA.I.LANCE Thompson, Ryan*
nocturneInteractivePerformance1.txt Oore, Sageev*; D’Eon, Jason; Miller, Finlay; Becker, Nic; Lowe, Scott C; Oore, Daniel
You don’t know what this is Shibuya, Kazufumi*
John Connor Project (BIGYUKI × Qosmo) Tokui, Nao; BIGYUKI; Nakajima, Ryosuke; Han, Alex; Sugano, Kaoru
The Unknown Porres, Diego*
Beneath the Surface Ozoani, Ezinwanne*; Tsaban, Linoy
Unreal Pareidolia -shadows- Allen, Scott*
Fragments of Autumn Park, Jihyeon*
Subsumption No. 1 Tallon, Tina*; Beacham, James
Visions of Destruction Canet Sola, Mar*; Guljajeva, Varvara
Mythos Synthscribe Remix: Deus in Machina (2023) Oore, Daniel; Staniland, Andrew; Brade, Stephen; Wang, Bryan; Sousa, Mauricio; Newsome, Gregory Lee; Grossman, Tovi; Oore, Sageev*
Imaginary-shell Sano, Fushi*

How to attend

A registration ticket must be purchased on This will allow you to attend the workshop in-person. The workshop will be held on Saturday Dec 16 at the New Orleans Ernest N. Morial Convention Center.

Call for Papers

We invite submissions in the form of papers. The deadline for paper submissions is Friday Oct 6 Wednesday Sep 27 at 11:59pm (anywhere on earth).

To Submit a Paper: We invite participants to submit 2-page papers in the NeurIPS camera-ready format (with author names visible), to be submitted to our CMT portal.

Topics may include (but are not limited to):

We encourage all authors to consider the ethical implications of their work. This can be discussed in a 1-paragraph section at the end of the paper and will not count towards the page limit.

In your submission, you may also indicate whether you would like to present a demo of your work during the workshop (if applicable).

Papers will be reviewed by committee members in a single-blind process, and accepted authors can present at the workshop in the form of a short talk, panel, and/or demo. At least one author of each accepted paper must register for and attend the workshop. Accepted papers will appear on the workshop website. Please note that we do not adhere to a formal peer review process and are normally unable to provide detailed feedback on individual submissions. We encourage submission of work that has been previously published, and work in progress on relevant topics of the workshop. This workshop will not have an official proceedings.

References and any supplementary materials provided do not count as part of the 2-page limit. However, it will be the reviewers’ discretion to read the supplementary materials.

Call for Artwork

We welcome submission of artwork that has been created using machine learning (autonomously or with humans). The deadline for artwork submissions is Nov 07 Oct 30 at 11:59pm (anywhere on earth).

We invite art submissions in any medium, including but not limited to:

This year we are asking for submissions to consist of:

Place these in a single zip archive called

Submit this zip file through our CMT portal.

The reason for this specific submission format is because the workshop will screen the accepted art pieces as a video slideshow (which is why we ask for the landscape format). This video of all the accepted artworks will be posted on our website as well. During the workshop, we also may invite a few select creators of accepted artwork to participate in the form of a short talk.

Important Dates

Previous years:

Connection with NeurIPS Creative AI Track

As you may have seen, NeurIPS is having their first creative AI track this year, for artistic submissions. We are excited that there is more interest in and visibility for creative AI now.

Both the organizers of this ML4CD workshop and the NeurIPS Creative AI track share the same goal: giving creative AI a platform! In doing so, we are actively collaborating with them, and are in close communication with the Creative AI Track chairs.

Since this is the first year that the NeurIPS Creative AI track is occurring, this first year our calls for art are separate, but in future years we may collaborate in a more direct way.

Their call for art already occurred, but you can submit to either/both!

For more in-depth specific differences between this ML4CD workshop’s call for art and the NeurIPS Creative AI Track, we have detailed them below:


If you have any questions, please contact us at