MACHINE LEARNING FOR ARTISTS
Scripps College MS038
Fall 2021
MW 09:35-10:50AM
Location: Steele Hall, Room 229
COURSE STAFF
Professor: Douglas Goodwin
Email: dgoodwin@gmail.com
COURSE OVERVIEW
Machine learning (ML) is transforming technology and society through applications in automatic translation, speech recognition, transportation, and beyond. While ML is becoming ubiquitous in our daily lives, it remains daunting to non-specialists. This course bridges this divide by introducing ML concepts to students without prior experience, providing templates for immediate hands-on work with ML tools and frameworks.
Students will study and remake artworks by leading ML artists including Mario Klingemann, Anna Ridler, Sougwen Chung, Memo Akten, Helena Sarin, and Tom White. The course explores various techniques and frameworks such as image segmentation, CycleGAN, pix2pix, and TensorFlow. Students will develop an independent project in the final third of the course.
Prerequisite: Any programming experience, especially with p5.js, ml5.js, and Python.
LEARNING OBJECTIVES
By the end of this course, students will be able to:
- Develop intuition for core machine learning concepts and algorithms
- Apply ML algorithms to real-time interaction in media art projects
- Collect custom datasets and train machine learning models
- Generate media using ML: words, sound, and images
- Critically discuss social impact and ethics of ML applications
- Navigate the landscape of ML-generated new media art
REQUIRED MATERIALS
- Free P5.js editor account
- Free GitHub account
- Free Google Colab account with Google Drive integration
RECOMMENDED READINGS
- The Information: A History, a Theory, a Flood by James Gleick
- Aesthetic Programming by Winnie Soon & Geoff Cox
- Artificial Intelligence: A Guide for Thinking Humans by Melanie Mitchell
ASSESSMENT
- 40% Weekly assignments
- 40% Final project (proposal and completion)
- 20% Attendance + Participation
Final Project Evaluation Criteria:
- Perseverance
- Faithfulness to proposal
- Creative engagement
- Critical thinking application
- Preparation and time management
- Success of finished piece
Rubric for Creative Projects
Based on Kristin Hughes' "Learning and Grading Rubric"
Percentage Calculation: Total points × 5 = Final percentage
(Example: 15 points = 75%)
COURSE SCHEDULE
Module 1: Introduction and Fundamentals
- Introduction to ML with ml5.js
- Image Classification with Teachable Machine
- Body tracking and pose estimation
- Face and hand tracking
Module 2: Advanced Techniques
- Style Transfer
- Pix2pix implementation
- DIY Neural Networks
- ML on Google Colab
Module 3: Critical Applications
- Embedding bias
- Adversarial attacks
- Ethics and social impact
- Final project development
Course Schedule
| Week | Date | Theme | Required Reading | Required Videos | Assignments |
|---|---|---|---|---|---|
| 1 | 08/30 | Introduction to ML with ml5.js | - | - Hilary Mason ML explanations - Beginner's Guide to ML with ml5.js |
- Image classifier demo - YOLO webcam detection |
| 09/01 | ML Art Foundations | - Awesome Machine Learning Art - AI4D workshop links |
- Anna Ridler, Nicer Tuesdays | - Build on image classifier demo - Publish to GitHub |
|
| 2 | 09/06 | LABOR DAY | - | - | - List interesting ML projects/artists |
| 09/08 | Image Classification | - What is Teachable Machine? - KNN classifier guide |
- Teachable Machine 1 - Image Classification |
- KNN Image Classifier project | |
| 3 | 09/13 | Body Tracking | - Hello-ml5 - PoseNet documentation |
- Alyosha Efros: Evolution as Data | - Discord: Submit pose estimation project |
| 09/15 | PoseNet Deep Dive | - ML5js: PoseNet guide | - Dan Shiffman: PoseNet tutorial | - Browser experiment with poses | |
| 4 | 09/20 | Face & Hand Tracking | - ml5.js face-api reference - ml5.js handpose guide |
- Rebecca Fiebrink: ML for Creators | - Discord: Submit face/hand tracking project |
| 09/22 | Advanced Pose Estimation | - | - Shiffman: PoseNet deep dive | - | |
| 5 | 09/27 | Style Transfer | - Ml5.js StyleTransfer docs | - Shiry Ginosar: Creative Material | - Discord: Combined style/pose project |
| 09/29 | Style Transfer Applications | - | - Gene Kogan: Visualization | - | |
| 6 | 10/04 | pix2pix | - Gene Kogan's pix2pix tutorial | - Efros & Isola: Socratic debate | - Discord: pix2pix + pose project |
| 10/06 | pix2pix Workshop | - | - | - | |
| 7 | 10/11 | DIY Neural Networks | - | - Tom White: Neural Abstractions | - Discord: DIY Neural Network project |
| 10/13 | Neural Networks Workshop | - | - | - | |
| 8 | 10/18 | FALL BREAK | - | - | - |
| 10/20 | FALL BREAK | - | - | - | |
| 9 | 10/25 | ML on Google Colab | - | - Jun-Yan Zhu: Efficient GANs | - |
| 10/27 | CycleGAN | - | - Jesse Engel: Magenta | - | |
| 10 | 11/01 | Embedding Bias | - | - Sofia Crespo: Artificial Biodiversity | - |
| 11/03 | Adversarial Attacks | - | - Joel Simon: Creative Networks | - | |
| 11 | 11/08 | Advanced Models | - | - Ian Goodfellow: Sequence Modeling | - |
| 11/10 | RNNs & GANs | - | - | - | |
| 12 | 11/15 | Motion Models | - | - | - |
| 11/17 | Motion Workshop | - | - | - | |
| 13-15 | 11/22-12/08 | Final Projects | - | - | Final Project Development |
Notes:
- All readings and videos should be completed before class
- Project submissions should be posted to Discord by specified dates
- Check course page for detailed assignment requirements
- Additional resources and examples available on course website
POLICIES
COVID-19 Policy
Masks required during class. Please consult Scripps College return-to-campus plan for latest guidance.
Attendance
- More than six unexcused absences will result in course failure
- Regular attendance required for both lectures and labs
Changes to the Syllabus
This syllabus is subject to change. Students are responsible for staying informed about updates, which will be communicated via Discord. All assignments should be prepared for the scheduled class day unless noted otherwise.
Zoom Notice
Class meetings held on Zoom may be recorded for educational purposes. These recordings are protected and used solely to support course facilitation. Students and participants are prohibited from making their own recordings. Students needing special accommodations for recordings should coordinate with the instructor.
Academic Integrity
Academic integrity is crucial to our educational mission. Academic dishonesty, including but not limited to cheating, fabrication, plagiarism, multiple submissions, or facilitating misconduct, undermines trust and learning. Plagiarism, the presentation of another author's words or ideas as your own, is a serious offense. Consult your instructor with any questions about documentation or quotations before submission.
Accessibility and Accommodations
Our goal is to make learning accessible for all students. If you face any issues with course materials or requirements, please contact me to discuss potential solutions. Students with disabilities are encouraged to consult the Office of Accessible Education for guidance and official accommodations. If you have approved accommodations, let's meet to devise an implementation plan. We are committed to meeting accessibility standards and welcome your feedback on improving access to course materials.
Inclusive Environment
We are dedicated to an equitable and inclusive learning environment, free from discrimination and harassment such as sexual violence, dating violence, and stalking, which violate college policies and legal standards. This policy applies to all individuals associated with the college. Violations can result in disciplinary actions, including expulsion or termination. A climate of mutual respect and open dialogue is essential, and speech is protected if it doesn't constitute harassment or discrimination.
Diversity
Our diverse community is a strength that fuels innovation and enriches education. We commit to reflecting this diversity within our student body and workforce, ensuring access for talented individuals from all backgrounds. By fostering an environment where ideas are shared respectfully, we enhance innovation and leadership development. We focus on removing barriers for underrepresented groups in all institutional activities.