-
Notifications
You must be signed in to change notification settings - Fork 0
Homework Ellen 04
| Section 04 Info | |
|---|---|
| Meeting Time | Mon 12:20pm - 2:50pm |
| Location | Room 411 |
| Contact |
ellen.nickles@nyu.edu On weekdays I aim to respond within 24 hours |
| Student Hours | Fri 1:30pm – 3:30pm Sign up here |
| Support |
Resident office hours (schedule) The Coding Lab (schedule or drop-in help, starts 9/22) How to ask code-related questions (examples) |
| ICM Help Sessions | Tue 3:00pm - 4:00pm @ Room 426 |
| Additional Materials |
Submit creative assignments Assignment responses Our Miro board Course syllabus p5.js Reference / Web Editor The Coding Train's Intro to p5.js Allison Parrish’s Creative Coding Notes Patt Vira's Make Art with Code xin xin's Intro to p5.js Jack B. Du's My Coding Show Qtv's Creative Coding tutorials in Chinese (also on bilibili) HappyCoding.io |
- About Our Section
- Week 8 • Oct 27 • Images, Pixels, Color
- Week 9 • Nov 3 • Video and Pose Estimation
- Week 10 • Nov 10 • Sound Synthesis
- Week 11 • Nov 17 • Data of Sound, Texts, APIs
- Week 12 • Nov 24 • Final Project Proposals
- Week 13 • Dec 1 • Final Project User Testing
- Week 14 • Dec 8 • Final Project Presentations
Please read the ICM Media Syllabus for the course description, learning objectives, and all policies.
In addition, our course section embraces an active-learning approach with the belief that programming is best understood by doing. In keeping with this approach, class time will be structured as a workshop and seminar instead of a lecture, relying heavily on group discussion and participation. Weekly tutorials before our class sessions will introduce fundamental coding concepts. During class, supplemental code examples will be presented and discussed before hands-on work together. Time will also be allotted to share our weekly assignments. Examples and assignments will iteratively build upon lessons learned in preceding classes. I encourage you to reach out to me outside of class to ask questions, share your ideas and feedback, and discuss topics in detail.
All assignments will be posted on this page, and all are required. Unless otherwise stated, assignments are due the night before class so that I have a chance to review your questions before we meet. All assignments are expected to be submitted by the stated deadline. If you anticipate any challenges meeting the deadline, please reach out to me so that we can consider your options together.
Submitted assignments will be marked as complete (full credit), partially complete (half credit), or incomplete (no credit) in accordance with the Assignment Policy.
Before class, read and/or watch the assigned tutorials about computational media topics and programming techniques to practice together when we meet in class.
There are worksheets* to practice techniques introduced in this course. You are expected to add links to your p5 sketches directly in these documents.
Each week there is an open-ended exercise to apply the concepts that we practice. You are expected to document your process, include a link to your p5 sketch, and answer these questions to complete the assignment:
- What did you do?
- What worked?
- What didn’t work, and what steps did you take to try to solve the issue?**
- Did any code-related questions come up for you?
- Be sure to cite your sources.
Submit the creative assignments using our homework form*.
A big part of learning at ITP is learning from each other. Share your work and in return you'll see everyone else's! Here is the spreadsheet with everyone's responses*.
*Log in to your NYU account to access the document.
**If you are struggling with the weekly assignments, put your energy into writing about what didn't work and how you tried to solve it. It is totally okay to vent your frustrations. Document your work to reference later and also to help you synthesize what you did and tried to do.
Adding on to the course AI Statement, cite your use of AI tools in accordance with our Citation Guidelines (see below).
In accordance with the Statement of Academic Integrity, you are expected to cite your sources and note how you used them. For code references, a link to the online source along with a brief description will suffice. If you use any generative AI tools, e.g. ChatGPT, Claude, Gemini, Copilot, etc., please include the model name and version with description of why and how you used the model.
- Overview of ICM Media
- Image files
- Objects with images
- Pixel regions
- Pixel manipulation
- Syllabus resources
- Image Manipulation with p5 by Adam Driggers, IMA Low Res '25
- Media resources
- Supported media type and formats
- Image sources: Pxhere | Unsplash | Wikimedia Commons and/or your own?
p5 examples
- Rotating shapes/images demo series
- All the ways to create images
- Getting started with image files
- Animating images
- Objects with the same image
- Working with many images
- Pixel and pixel region info
- Objects of pixel regions
- Demonstrating alpha
- Canvas pixel array
- Image pixel array - change pixel color
- Pixelated image using pixel array - get pixel color to draw a shape
- Color
Related projects
- Parts of Whole assignment by Callie, Surya, and Omi
- gämma by p1xelfool
- Kim Asendorf
- #dominicanwomangooglesearch by Joiri Minaya
- Photomontage* by Sohei Nishino
- Four Artworks for The Myth of Asian American Identity* by Kensuke Koike, Photographs by Tommy Kha
- The Constructed Self by Karen Navarro
- The Moment We Meet by Hsin-Chien Huang
- Joiners by David Hockney
- Garbage Selfie by Jenny Odell
- First 1/3 of Coding Challenge: Slitscan Time Displacement
Effect using the
copy()function with images by Coding Train -
Interactive Easter Egg using
copy()by Nahuel Gerth - Pushin' Pusheen Pixels Tutorial by August Luhrs
- 2D Water Ripple Coding Challenge by Coding Train, p5 version by Lisa Jamhoury
- *Free NYTimes subscription via NYU
- PRACTICE • Week 8 Worksheet
- CREATE • PARTS OF A WHOLE
- Create and/or manipulate an image (or images) using pixel regions and/or
individual pixels to create an alternative reality than depicted in the
source image(s). Can you describe in 1-3 keywords how the result feels
different from the source image(s)? Document
your process and result. Work with any kind of imagery you want.
Whatever you choose, you might consider these elements of a
photograph
in your computational approach:
- Techniques and attributes: Light, focus, time, motion, vantage point/point of view, framing, cropping, composition
- Composition: Shape, line, angle, color and tone, pattern, depth, overall composition
- Content: Subject, background, foreground, people, story, mood/feeling, symbol/metaphor, style, genre, artist's intention/purpose, meaning
- Submit on our homework form.
- Create and/or manipulate an image (or images) using pixel regions and/or
individual pixels to create an alternative reality than depicted in the
source image(s). Can you describe in 1-3 keywords how the result feels
different from the source image(s)? Document
your process and result. Work with any kind of imagery you want.
Whatever you choose, you might consider these elements of a
photograph
in your computational approach:
- READ
- Working with video tutorial by Allison Parrish
- WATCH
- Coding Train Video 11.1 on capturing your webcam ~10min
- Video with machine learning pose estimation
- Technical context: A visual and intuitive understanding of deep learning from Octavio Good (0:00 - 9:40) ~10min
- Coding Train:
-
(Introducing) ml5.js 1.0 (using FaceMesh)
- Part 1: Intro by Dan Shiffman ~20min
- Part 2: Guest Conductor Patt Vira ~35min (OPTIONAL, uses vectors)
- Pose Estimation with ml5.js ~22min
- Hand Pose Detection with ml5.js ~16min
-
(Introducing) ml5.js 1.0 (using FaceMesh)
- (OPTIONAL) Coding Train: Live Video and createCapture on video photo booths, pixel mirrors, painting with pixels, and slit-scan video ~1hr 20min
- Video
- Webcam
- Pose estimation with ml5.js
- BodyPose
- HandPose
- FaceMesh
- Related projects
Syllabus
Media Files
- Supported media type and formats
- Videvo | Videezy or your own
- Downsize video files: VLC (free) > File > Convert / Stream > Customize > Video codec
- p5.MediaElement methods (.play(), .stop(), .onended(), and more!)
ml5 Pose Estimation
- BodyPose Reference
- HandPose Reference
- FaceMesh Reference
- Thanks to IMA Intro to ML for the Arts (Fall 2024) for these resources!
Thanks to IMA Intro to ML for the Arts (Fall 2024) for many of these examples and related project resources!
p5 examples
- Video
- Webcam
- ml5 Pose Estimation
- BodyPose
- Getting started / Basics with MoveNet
- MoveNet Keypoints (example 2) | Skeleton
- BlazePose Keypoints | Skeleton
- Coding Train examples scroll down
- HandPose
- Getting started
- Single image
- Parts
- Keypoints
- Particles by Jack B. Du
- Quadrilateral by Jack B. Du
- Quadrilateral with texture by Jack B. Du
- Pinch to paint
- Start-stop detecting (method also available for BodyPose and FaceMesh)
- Coding Train examples scroll down
- FaceMesh
- Single image
- Face keypoints
- Face bounding box
- Parts keypoints
- Parts bounding box
- Shapes from parts
- Lip contours
- Triangulation by Jack B. Du
- Face masks by Jack B. Du
- Image texture
- Dora Do's ml5 workshop for IMA Low Res Summer 2025
- BodyPose
Related projects
- Video and pixels (no AI)
- Video photo booths, pixel mirrors, painting with pixels, and slit-scan video from The Coding Train
- Flickering Existence by Liu Chang
- p5VideoKit - dashboard for mixing video in browser by John Henry Thompson
- Online Live Performance by Yuguang Zhang and Nuntinee Tansrisakul
- Misc
- PoseNet Sketchbook by Maya Man (PoseNet preceded BodyPose)
- Drawing with Body by Casher
- Bubbles by Nahuel Gerth
- Spring Experiments by Jasmine Nackash
- Mirrors / Masks
- Interactive Art by Danny Rozin
- Más Que la Cara by Zach Lieberman
- Aspire Mirror | The Algorithmic Justice League by Joy Buolamwini
- Move Mirror by Google Creative Lab
- Customizable AR face masks - Made with TensorFlow.js (and FaceMesh) by Samarth Gulati and Praveen Sinha
- Coding Train's Happy Halloween Mask | Code
- Movement / Performance
- Doppelgänger by NUUM Collective
- MAXforum: Art Of Movement Mapping (talk) Artist/Programmer Lisa Jamhoury, Artist/Technologist Mimi Yin, and Neurobiologist Dr. Ryan York discuss the creative constraints and possibilities of working with AI and movement mapping on humans (and other organisms!)
- Body, Movement, Language: AI Sketches With Bill T. Jones by Google Creative Lab
- Ballet Rotoscope by Euphrates
- Messa di Voce by Golan Levin, Zach Lieberman, Jaap Blonk, and Joan La Barbara
- Animation
- The Treachery of Sanctuary by Chris Milk
- Pose Animator by Shan Huang
- Puppet Parade - Interactive Kinect Puppets by Design I/O
- Text / Type
- Text Rain by Romy Achituv & Camille Utterback
- Touch Type: Interactive Typography by Richard Yee
- Finger Talk by Future Sketches
- Magnet Poetry by Lea Naisberg
- Embodied Typography by Nahuel Gerth
- Finger Numerals by Nahuel Gerth
- Fingerspelling by Hello Monday
- Games
- Quaruntine and Beat pose by Charlie Gerard
- ITP Web Olympic by Vibert Thio
- Body assignment (see below) by Jesse, Dean, Callie, and Surya
- NYTimes Opinion The Racist History Behind Facial Recognition: When will we finally learn we cannot predict people’s character from their appearance?
- No worksheet this week!
- CREATE • BODY
- The idea this week is to center an experience around the body--yours, someone else's, or a particular audience.
- Choose the tools and techniques that make the most sense for the concept, keeping in mind that pose estimation extends choices for interactive design beyond the mouse and keyboard, which prioritize the hands and fingers for communication and expression in a very specific way.
- Some questions to start brainstorming:
- What are the private, public, and performative aspects of bodies?
- How are bodies imagined, idealized, reflected, altered?
- What is the personal and cultural significance of masks for ritual, to explore identity, hide, entertain, frighten...
- In what ways can you use the body for expressive input? Consider choices for the part(s), position, distance between parts, direction of movement, and speed.
- Submit on our homework form.
- WATCH • Coding Train Videos 17.1 - 17.3, 17.5 - 17.8 on sound (code) ~90min
- Share homework
- Quick survey for next week
- Mic input
- Sound output (samples and oscillators)
- Making notes
- Timing and patterns
- Related projects
- Syllabus Resources
- What Is an Oscillator in Music?
- The Math Behind Music and Sound Synthesis
- Media resources
- Record or generate your own!
- Orchestral Sample Library
- Another Orchestral Sample Library
- Free Music Archive
- Freesound
- BBC Sound Effects
- Sound of Picture
- Blackhole for routing computer sound
- Supported media type and formats
- Audacity for audio editing (free)
- Record or generate your own!
p5 examples
- Mic input
- Sound output
- Making notes
- Timing and patterns
Related projects
- Articles / podcasts
- Sonic Alchemy: Exploring the Art of Generative Sound
- Music Is More Than Just Sound*
- NYTimes Magazine Sonic Voyages (Listen to laughing rats in NY!) browser version*
- Can Brown Noise Turn Off Your Brain?*
- The Handbook for Sonic Happiness
- Writing Music in Light
- *Free NYTimes subscription via NYU
- jazz.computer by Yotam Mann and Sarah Rothberg
- Plink by DinahMo
- Patatap and Typatone by Jonathan Brandel
- Music Eclipticalis by Brian Foo
- Troupe-like in p5.js by Cassie Tarakajian
- ...based on Trope app by Brian Eno and Peter Chilvers
- Symphonies of the Planets by Joohyun Park
- Chrome Music Lab
- Pose estimation-related
- Sidewalk Orchestra by Cristobal Valenzuela
- Code Music by Tero Parviainen
- Semi-Conductor by Google Creative Lab
- Sampler and BodySynth by Use All Five & Google Creative Lab
- Mouth-Controlled Synthesizer with FaceMesh by Jack B. Du. code
- Melody Painter with HandPose by Jack B. Du. code
- BodyMouth by Kat Mustatea
- Imogen Heap: NPR Music Tiny Desk Concert
- PRACTICE • Week 10 Worksheet - last one!
- COMPOSE • Create a 30-60s sound composition or generative music system. Option
to work with partner. No visuals. Just sound. Some ideas to get started:
- Design a melody using this process and figure out an algorithmic way to generate it. Or, How Generative Music Works by Tero Parviainen.
- Record bits of spoken word and loop them to create music. See "Record and loop" example above. Why repeating words sound like music to your brain.
- Use sound samples and manipulate their playback rate to control pitch instead of the oscillator.
- Try implementing a different scale with different pitch ratios. More about scales and Music Theory: The TL;DR Version 2022.
- Play with Timbre and make use of p5 Sound's post-processing features: Delay, Filter, Reverb, Convolver, etc.
- Maybe the composition tells a symphonic story with several characters.
- Compose a sonic piece that is not music.
- Use audio navigational cues to implement a vision accessibility-feature for a video game. Student example by Kea Lia Pedersen.
- Submit on our homework form.
- READ
- Allison Parrish's tutorial on Text and Type
- Review these syllabus examples and tutorials for Text and Data. We'll cover a couple topics, but you might want to go further with the materials posted here.
- WATCH
- Coding Train videos 17.9-17.11 on sound analysis (code) ~36min
- Listen to homework
- Data
- of Sound
- of Text
- as JSON
- from API
- Final project overview
Resources, p5 examples, and related projects
- Resources
- p5 examples
- Amplitude of sample
- FFT Waveform
- FFT Analyze
- p5 audio visualization collection by Brian Ho (Low Res)
- Related projects
Resources
p5 examples
- Font examples
- Array vs string
- Dancing characters
- Random story starters
- Draw with lines from a poem
- Word and keyword count
- Draw one word at a time
- Work with text outline
- Carrie Wang's Scrolling Text and Drawing the Text Array
- Allison Parrish's Interactive Type Examples
- Generative Design P.3. Type Examples
Related projects
- Programming Design Systems: Typography by Rune Madsen
- Space Type Generator
- Phase by Elias Hanzer | related article
- Type + Code Series by Yeohyun Ahn
- Generative Typography course and student gallery by Lynne Yun (ITP alum) and Kevin Yeh
- Generative Typography by Ze Wang
- DIA
- the html review
- Type Dynamics by Lust
- Text Rain by Romy Achituv & Camille Utterback
- Legible City by Jeffrey Shaw and Dirk Groeneveld
- How to Make a Dadaist Poem
- Generative Word Games by The Oulipo Group
- Writing with the Machine with Robin Sloan
- Google Experiments AI + Writing Collection
Resources
- Syllabus resources
- JSON resources
- Working with JSON
- JSON Formatter Chrome Extension
- Formatting and Validating JSON
- Coding Train Intro to Data and APIs (includes working with Wordnik, NYTimes, Giphy, and Wikipedia APIs)
- Data sources
p5 examples
Related projects
- IMG_0001 by Riley Walz
- YouTime by Callie Page documentation
- Shirley Wu
- Jason Snell
- Letter Frequency by Jasmine Nackash
- The Library of Missing Datasets by Mimi Onuoha
- We Need Us by Julie Freeman
- Listen to Wikipedia by Stephen LaPorte and Mahmoud Hashemi
- See, also: Our favorite visualizations built on Wikipedia data curated by Hatnote
- Numbers Series: performance and sound installations by Mendi and Keith Obadike (related Reading the Numbers of Stop-and-Frisk)
- Smoke and Mirrors: Methane Data x Big Oil Ads since the first Earth Day by Beatie Wolfe
- MethaneSAT
- Climate Stripes by Ed Hawkins
- Air Quality Stripes
- The True Colors of America’s Political Spectrum Are Gray and Green*
- Dear Data by Giorgia Lupi and Stefanie Posavec
- Sleep Drawings by Laurie Frick
- Heartmonic and Eunoia (Version 2) by Lisa Park
- The Data Garden Project Gallery (of p5 sketches)
- Visual Essays by The Pudding
- 45 Ways to Communicate Two Quantities
- *Free NYTimes subscription via NYU
- DESCRIPTION
- Your final project is an open-ended creative project that builds off or is
inspired by the concepts in this class. It is also an opportunity to push
your abilities to produce something that utilizes what you have learned. The
final project includes:
- presenting a proposal to our class in Week 12,
- user testing a prototype of the project with our class in Week 13,
- and documenting your project in a blog post (or Notion, Google Doc, etc.) including your code contributions to the project, when the project is shared during final presentations in Week 14.
- Focus on concept - There is no requirement to use a particular aspect of programming. Focus on an idea that excites you and choose the best programming concepts and tools to help you realize it.
- Build or Create Anew - You can take something you’ve already made and develop it further or create something entirely new.
- Final projects can be one part of a larger project integrated with a different class.
- Option to Collaborate - Final projects can be collaborations with anyone in any class. Group projects are welcome and encouraged! It is expected that everyone will contribute and document their own code to the project.
- Scope - It’s okay to keep things simple and small in scope. For example, if your project idea is a big one, document the larger idea and implement (and document) a smaller piece of it.
- Audience - Also consider making a final project for a small audience such as our class or for a single person like a family member, friend, or yourself as a way to focus your idea and design process.
- Your final project is an open-ended creative project that builds off or is
inspired by the concepts in this class. It is also an opportunity to push
your abilities to produce something that utilizes what you have learned. The
final project includes:
- EXAMPLES from ITP/IMA/Low Res Spring Show 2020 | Winter Show 2020 | Spring Show 2021
- FINAL PROJECT PROPOSAL
- PART 1 • Research your project and collect some notes, such as:
- Inspirations • How did you become interested in this idea? quotes, photographs, products, projects, people, music, political events, social ills
- Source material • Drawings, images, videos, sounds, text
- Code • Your own sketches and/or other people's sketches
- Ideas for title • And maybe a 1-sentence description?
- Context • What is the story? Who's it for? How will people experience it? Is it interactive? Is it practical? Is it for fun? Is it emotional? Is it to provoke something?
- Questions for your classmates • What are you unsure of, conceptually and technically? On what aspect(s) would you like feedback?
- PART 2 • Add two (2) to three (3) slides to this deck to share your proposal. Consider using the list above as an outline to illustrate what’s in your head(s), e.g. include some keywords, images, or embed video, etc. Everyone will a few minutes each to share in our next class.
- PART 1 • Research your project and collect some notes, such as:
- Final project proposals
- Discuss final project user testing
- FINAL PROJECT PROTOTYPE
- PART 1 • Create a working prototype of your final project to user test in
class next week.
- If your project is screen-based, be prepared to share a live sketch, full-screen if possible.
- If your project involves some type of physical interaction, ideally bring it to class or show a short video (~2 min)
- PART 2 • Prepare for user testing
- Observing how others interact with your project is an incredibly valuable part of the design process to understand what’s working (and not) in order to improve the experience.
- Sometimes unexpected user behaviors and interaction outcomes provide insights for developing projects in entirely new directions.
- User testing is not about getting everything perfect the first time! It’s about learning, iterating, and improving based on these real-world insights.
- Consider the kind of feedback that will be most useful for your
project's development, and prepare at least two (2) open-ended, neutral
questions for your user testers that focus on specific aspects of your
project you're unsure of or wanting to know more about. This type of
question often leads to more genuine and useful feedback than "yes/no" or
leading questions. Some examples to consider:
- What do you think the main goal or message of this project is?
- What did you think was the first step?
- Did any specific part of the experience (visuals, sound, interaction) evoke a particular reaction?
- Did any particular part of the project capture your attention or make you curious? Why?
- Did you notice any parts that didn’t work the way you expected?
- Was there any point where you felt like you wanted to stop or didn’t know what to do next?
- PART 3 • Document your prototype and your user-testing questions
- Document what you created for this stage. Aim to clearly communicate your progress, what’s working so far, and what you’re still figuring out.
- Your documentation should include:
- A written description of what you built for your prototype and what specific aspect(s) you focused on developing this week.
- Any relevant media such as photos, screenshots, short videos, and code that help show the current state of your project.
- Your user-testing plan, including the two (or more) open-ended questions you prepared for our in-class testing.
- Submit on our homework form.
- PART 1 • Create a working prototype of your final project to user test in
class next week.
- User testing
- Discuss next week
- FINAL PROJECT
- PART 1 • Complete your final project.
-
PART 2 • Documentation. Write your final project documentation in the form of a blog post
(or Notion, Google Doc, etc.). It's up to you to figure out how to best
document your project. Here are some guidelines if you aren't sure what to
include:
- Title and a brief written description
- Visual documentation such as a sketch running online, images, video, etc.
- References such as links to related projects, code samples, etc.
- What were some of the successes and challenges? What did you learn?
- What do you find interesting about what you did?
- How do you feel about WHY you want to use code in your work now compared with the beginning of the semester?
- How did coding it make the final product different from using software?
- If you are struggling with your project and can't get things to work, you should feel free to put your energy into writing about what didn't work (and vent any frustrations).
- Describe next steps to further develop the project.
- If the project is a collaboration, each person needs to contribute code and submit a post documenting their contributions to the project.
- Be sure to cite the sources of any material / code you use with the exception of examples provided by the instructors or demonstrated for the course.
- Submit on our homework form.
-
PART 3 • Presentation (3 Slides in 3 Minutes)*. To ensure everyone has
time to present and receive feedback next week, we will stick to a
structured format. Please add three (3) slides to this deck to demonstrate
your project:
- Slide 1 • Context. What is the story? Who is it for? Include the project title and a 1-sentence description.
-
Slide 2 • Demo. Embed a video or screen recording of the project in
action (~1 min).
- NOTE: You may do a live demo, but please have a video here as a backup. You may tab out to show a live sketch running in fullscreen mode only if it is loaded and ready to go instantly.
- See these tips on how to create a first‐person perspective demo of your final from Mimi Yin.
- Slide 3 • Computation. Highlight a specific aspect of the code. Show us a snippet of logic, an algorithm, or a technical hurdle you solved to emphasize the computational aspects of the work.
- *NOTE: If collaborating with a partner in our class, you should each have a separate "Computation" slide, so your presentation will be four (4) slides long.
- Welcome and general notes
- Final project presentations 🙌
- Course evaluation