LCC 6310: Computation as an Expressive Medium
Instructor: Michael Mateas
Email: michael.mateas@lcc.gatech.edu or michaelm@cc.gatech.edu
Office: Skiles 361
TA: Mayhew Seavey
Email: gtg630n@mail.gatech.edu
Lab hours
Tues:
Wed:
Thurs:
Fri:
Lecture: TTh
Lab: Friday
Office Hours: Email for appointments
Description
The goal of this course is to learn Java programming in the context of an art and design practice, that is, to understand computation as an expressive medium. We will juxtapose reading and discussion of seminal articles in computational media (from the New Media Reader) with Java programming projects designed to exercise specific technical skills as well as encourage conceptual explorations in computational art and design. Anyone working in new media will eventually be involved on interdisciplinary projects in which the ability to program will be a strong asset, if not a necessity. Even if in your future career as an artist or designer programming is not a large part of your practice, this course will empower you to communicate confidently with programmers, and thus deepen your interdisciplinary collaborations. And perhaps a few of you will become fascinated with the expressive possibilities opened up by programming, and will choose to make computation one of your primary media.
Books
The New Media Reader
Edited by Noah Wardrip-Fruin and Nick Montfort
Head First Java
Bert Bates and Kathy Sierra
Projects
There are six projects. Each of the projects will explore an expressive possibility of computational systems and exercise specific Java programming skills. With each project we will read and discuss a number of seminal readings in new media where the conceptual explorations in the readings intersect with the project. For the later projects you will turn in a project description approximately half way through the project describing your design, that is, what you intend to accomplish with the project, what experience you're trying to create.
In addition to the projects, there are smaller weekly assignments. These weekly assignments are designed to exercise specific programming and software design skills.
Each student is required to show one of his or her projects at the IDT demo day at the end of the semester.
Courseware
Assignments and projects appear on two different courseware websites.
Assignments: view; login and upload
Projects: view; login and upload
Grading
There are 105 points possible in the class.
Projects: 60 points (6 projects, 10 per project)
Weekly assignments: 30 points (6 assignments, 5 per assignment)
Helpful citizen: 10 points extra credit
Lab
Every Friday from
Notes on Reading Head First Java
Head First Java is written with the assumption that you're using raw Java, not Processing on top of Java. There are therefore a couple of changes necessary to make examples from Head First Java run within Processing.
First, whenever you see System.out.print or System.out.println, replace this with print or println. Processing's print and println commands print to the scrolling pane in the bottom of the Processing environment. Java's System.out.print (and println) commands print to the console window that was used to start the Java program (in this case Processing), a window that probably doesn't exist unless you started Processing from the command line. If this doesn't make too much sense, don't worry about it; just do the replacement.
Second, you'll see that every example in Head First Java has the method public static void main(String[] args) somewhere in it. For a "raw" Java program, this is the entry point, the place where Java starts executing. For a Processing program, it just starts from the top of the file if we don't have a setup() method, or the setup() method if we do. Whatever code you see in the main() method can be moved into the Processing top level (not in a method, or in the setup() method). If a class only contains a main method (this is true of any Head First Java classes that have TopLevel in the name (e.g. TestTopLevel), then you can get rid of the class entirely. In Head First Java, any class with TopLevel in the name is just there to provide an entry point; the class doesn't have any additional state (ie. class variables) that you have to worry about (in a sense it's not a "real" class). If the main() method appears in a class that has variables, and possibly other methods, then you have to keep the rest of the class, and just move the code from main() into the Processing top level (not in a method, or in the setup() method). Here are some examples.
Additional
Programming Resources
Sun's Java site has a nice tutorial.
The docs describing the Java class library are available for online viewing and for download (near bottom of page).
Programming
Environments
We will use Processing, an environment built on top of Java. All code you write in Processing is Java, but Processing provides a number of built in classes that make graphical operations easy, as well as a programming environment that supports script-like Java programming (that is, easily being able to write snippets of code and immediately execute them). Processing has been installed on the Macs and PCs in the IDT lab. Additionally, it will probably be useful to have Processing on your own machine. Please download it here. Q&A bboards for using Processing can be found here.
Later in the class we will look at "raw" Java, that is, what you need to do to make stand-alone Java programs that don't use Processing. This will also help us to understand how Processing works, and how you might build something like Processing yourself.
Tuesday, August 24
Overview of class.
Introduction to Processing.
Programming concepts: using Processing, syntax, drawing primitives, coordinate systems, variables.
Thursday, August 26
Programming concepts: using Processing, syntax, drawing primitives, coordinate systems, variables.
Discuss: From Software: Exhibition at the Jewish Museum - NMR pp. 247-257.
Discuss: Four Selections by Experiments in Art and Technology - NMR pp. 210-226.
Assignment 1
|
A1-01 |
Draw three lines. |
|
A1-02 |
Draw five lines. |
|
A1-03 |
Draw three ellipses. |
|
A1-04 |
Control the position of two lines with one variable. |
|
A1-05 |
Control the position and size of two lines with two variables. |
|
A1-06 |
Control the properties of two shapes with two variables. |
|
A1-07 |
Create a simple, regular pattern with six lines. |
|
A1-08 |
Program your pattern from Exercise A7 using while(). |
|
A1-09 |
Draw a layered form with two new loops. |
|
A1-10 |
Redo Exercise A1-05 using mouseX and mouseY as the variables. |
|
A1-11 |
Draw two visual elements that each move in relation to the mouse in a different way. |
|
A1-12 |
Draw three visual elements that each move in relation to the mouse in a different way. |
|
A1-13 |
Move a visual element across the screen. When it disappears off the edge, move it back into the frame. |
|
A1-14 |
Draw a visual element that moves in relation to the mouse, but with a different relation when the mouse is pressed. |
|
A1-15 |
Using if and else, make the mouse perform different actions when in different parts of the window. |
|
A1-16 |
Develop a kinetic image which responds to the mouse. |
Due: Friday
September 3
Friday, August 27
Tuesday, August 31
Programming concepts: loops, if-then, arrays
Discuss: Man-Computer Symbiosis - J.C.R. Licklider, NMR pp. 73-82.
Discuss: Personal Dynamic Media - Alan Kay & Adele Goldberg, NMR pp. 391-404.
Thursday, September 2
Programming concepts: built-in processing methods, input (including mouse), arrays, defining your own methods.
Head First Java chapter 3 discusses variables and arrays.
Project 1
From the central heartbeat of the central processor, to the obsessive timestamping of files and blog entries, to ever present clock displays, time is a fundamental feature of computation. Display the progress of time in a non-traditional way. It is OK to consider large temporal scales (e.g. seasons), but smaller temporal scales should also be displayed (or be available to be displayed, perhaps as a function of user input). You may make use of mouse input if you wish.
Due:
Friday, September 10
Clock Examples:
In
the Processing examples under Input: seconds_minutes_hours and milliseconds.
The conceptual strategy in both cases is to procedurally generate a kinetic
abstract image where the dynamic changes are linked to time. From the point of
view of this assignment, the milliseconds example is quite weak: it doesn't
give a sense of the passage of time in multiple scales; the changing grayscale
bars aren't connected to any intuitive, experiential sense of time passing.
Industrious
Clock. The conceptual strategy is to re-present the standard, numeric
presentation of time in a non-traditional medium (video representations of
paper and pencil).
Subway.
The conceptual strategy is to take a familiar rhythmic bodily experience and
use it to represent the passage of time. In this case, she takes the changing
rhythms of the subway (size and character of the crowd over the course of the
day, the swaying of bodies, changing patterns of light and dark in the subway
windows) and uses these to represent time.
Bubbles.
The conceptual strategy is to take a simulation of a physical or biological
process (in this case, bubbles in some fluid) and tie parameters of the
simulation (e.g. size and number of bubbles) to the time. A challenge with this
conceptual strategy is modifying the parameters of the simulation in such a way
that the changing simulation can still be read as the passage of time.
Maeda's
Clocks. A quicktime video of several screen-based clocks developed by
designer John Maeda. The conceptual strategy employed in this series of clocks
is to visually redesign standard hands or numeric representations of time.
Of course, your own design shouldn't be too similar to any of these examples.
Friday, September 3
Assignment 1 due
Tuesday, September 7
Programming concepts: built-in processing methods, input (including mouse), defining your own methods.
Head First Java chapter 2 introduces objects and methods.
Discuss: Sketchpad: A Man-Machine Graphical Communication Systems - Ivan Sutherland, NMR pp. 109-126.
Discuss: Direct Manipulation: A Step Beyond Programming Languages - Ben Schneiderman, NMR pp. 485-498.
Thursday, September 9
Programming concepts: classes
Head First Java chapters 4 and 5 talk in more detail about defining classes and objects.
Assignment 2
|
A2-01 |
Using beginShape() and endShape(), create a composition with five or more vertices. |
|
A2-02 |
Using beginShape() and endShape(), create a composition with ten or more vertices. |
|
A2-03 |
Create an image different from A2-02, but using the same vertex data. |
|
A2-04 |
Write a function with one parameter and demonstrate it visually. |
|
A2-05 |
Write a function for drawing triangles and visually demonstrate its flexibility. |
|
A2-06 |
Write a function with three or more parameters and visually demonstrate its flexibility. |
|
A2-07 |
Create a dynamic animation using the cos() function as a generator for motion. |
|
A2-08 |
Create a dynamic animation using the cos() and sin() function as a generator for motion. |
|
A2-09 |
Move two visual elements across the screen using the random() function as a generator of movement. Give each element a unique nonlinear motion. |
|
A2-10 |
Create an event that begins when the mouse is pressed and ends when the mouse is released. |
|
A2-11 |
Create a responsive image that behaves differently when the mouse is moving and the mouse is dragging. |
|
A2-12 |
Create a button that changes the color of the background when it is clicked. |
|
A2-13 |
Program your moving elements from A2-09 but use classes to represent the two visual elements. |
|
A2-14 |
Create a subclass of one of the asteroids classes that adds a new capability. Some examples of what you could do: create a subclass of Rocket (or ArmedRocket) that shoots flame when the thrusters are fired and/or plays a sound when thrusters are fired, create a subclass of Asteroid that know when it’s been hit (instead of doing this test in loop()), create a subclass of Asteroid that splits into two smaller Asteroids when it’s hit. |
Due:
Friday, September 17
Friday, September 10
Project 1 due
Tuesday, September 14
Programming concepts: classes, image manipulation
Discuss: A Cyborg Manifesto: Science, Technology, and Socialist-Feminism in the Late Twentieth Century - Donna Haraway, NMR pp. 515-542. (Here’s a Wired article about Haraway and the Cyborg Manifesto)
Discuss: The GNU Manifesto - Richard Stallman, NMR pp. 543-550.
Thursday, September 16
Programming concepts: classes, image manipulation
Project 2
The contemporary computer scene is dominated by the graphical user interface (GUI). For almost every task, from manipulating text, imagery, sound, video, to configuring a computer's operating system (e.g. control panels), from searching for and organizing information (e.g. the web), to the process of programming (e.g. integrated development environments), there are special purpose GUI tools supporting the task through analogies to embodied interaction with physical objects. But no tool is neutral; every tool bears the marks of the historical process of its creation, literally encoding the biases, dreams, and political realities of its creators, offering affordances for some interactions while making other interactions difficult or impossible to perform or even conceive. While the ability to program does not bring absolute freedom (you can never step outside of culture, and of course programming languages are themselves tools embedded in culture), it does open up a region of free play, allowing the artist to climb up and down the dizzying tower of abstraction and encode her own biases, dreams and political realities. What graphical tools would you create? Create your own drawing tool, emphasizing algorithmic generation/modification/manipulation. Explore the balance of control between the tool and the person using the tool. The tool should do something different when moving vs. dragging (moving with the mouse button down). The code for your tool should use at least one class.
Due:
Friday, September 24
Friday, September 17
Lab 4 introduces the BImage class.
Code for Lab 4 (imagetest1, imagetest2, imagewrapper)
Assignment
2 due
Tuesday, September 21
Programming concepts: classes, Java platform API (ArrayList)
Discuss: Happenings in the
Discuss: The Cut-Up Method of Brion Gysin - NMR pp. 89-91.
Discuss: Six Selections by the Oulipo - NMR pp. 147 - 189 (much of this is a paper poetry machine).
Thursday, September 23
Programming concepts: classes, Java platform API (ArrayList)
Assignment 3
|
A3-01 |
Create a subclass of BImage that implements a mosaic(int blockSize) method. The blockSize parameter specifies how big the mosaic block is (e.g. blockSize = 4 would mean the mosaic block size is 4 pixels by 4 pixels). The mosaic method should replace each block of pixels in the image (e.g. if blockSize = 4, each block of 4 by 4 pixels) with the average color value of the pixels in that block. Look at the Pixelate->Mosaic filter in photoshop for an example of what this image operation does. Demonstrate your new class by drawing an image with several different block sizes. |
|
A3-02 |
Write a small app that demonstrates kinetic text. Your app should allow the user to type something and move the text around in some why while they type. For example, the user might type text on a line, but slowly the words or letters start drifting apart, or perhaps the line starts bending, or the words and letters flutter to the bottom of the screen, etc. Of course you shouldn't exactly copy any of the typographic in Processing or that you find on the web (though using such examples for inspiration, as a place to start modifying code, etc. is fine). |
Due:
Friday, October 1
Friday, September 24
Project 2
due
Tuesday, September 28
Programming concepts: typography, text manipulation
Discuss: From Augmenting Human Intellect: A Conceptual Framework - Douglas Engelbart, NMR pp. 93-108.
Discuss: From Computer Lib/Dream Machines - Theodor H. Nelson, NMR pp. 301-339.
Thursday, September 30
Programming concepts: typography, text manipulation
Project 3
Literary machines are potential
literature, procedurally producing textual traces in response to interaction.
Examples of literary machines include interactive fiction, nodal hypertexts,
interactive poetry (often with animated typography), and chatterbots. Create a literary machine. The literary
machine must include algorithmic elements, such as animated typography,
generated text, conditional responses as a function of the previous interaction
trace. It must respond to external inputs (e.g. user interaction). Your piece
may include conjunctions of text and imagery.
Self
Portraits [as Other(s)] by Talan Memmott
Sky Scratchez by Mez and Talan Memmott (on a PC)
Young Hae-Chang Heavy Industries
Due:
Friday, October 8
Friday, October 1
Assignment
3 due
Tuesday, October 5
Programming concepts: typography, text manipulation
Discuss: As We May Think - Vannevar Bush, NMR pp. 35-47.
Discuss: Mythinformation - Langdon Winner, NMR pp. 587-598.
Thursday, October 7
Programming concepts: parsing html, recursion
HTML Parse Example (Image Collage)
Assignment 4
TBA
Due:
Friday, October 15
Friday, October 8
Project 3
due
Tuesday, October 12
Programming concepts: parsing html, recursion
Discuss: Nomadic Power and Cultural Resistance - Critical Art Ensemble, NMR pp. 781-790.
Discuss: The World-Wide Web - Tim Berners-Lee, et. al., NMR pp. 791-798.
Thursday, October 14
Programming concepts: parsing html, recursion
Project 4
Hypertext was conceived as a
computer-aided form of reading and writing whose structure matches that of the
human mind (a tangled web of association), thus enabling humans to make sense
of the exponential growth of knowledge experienced in the 20th
century. The World-Wide Web, while a rather anemic implementation of hypertext,
makes up for these deficiencies by providing us a sneak peak at what it might
be like to have a truly global repository of knowledge. But making sense of the
world is not just a matter of structure, but of process, of the dynamic
construction of meaning. And as we've been discovering together, computation is
fundamentally a process medium. What would you do to the web? Create an
applet that dynamically does something to one or more web pages (e.g. collage,
systematic distortion, re-layout, ironic superposition, etc.).
Mark Napier’s work (e.g.
look at Shred and Black and White)
Due:
Friday, October 29
Friday, October 15
Lab on debugging.
Tuesday, October 19
Fall recess
Thursday, October 21
Assignment 5 (non-graded)
|
A5-01 |
Modify image collage to, instead of grabbing images, grab headlines from several news sources and display them. This gives you practice in looking at the html source for multiple sites (e.g. New York Times, CNN), determining how a piece of information is represented, and writing the parse code to grab that piece of information. |
|
A5-02 |
Write an html parser that looks for keywords (you pick the keywords) in the text (not within a tag) of a page and counts how many times different keywords appear. You can imagine that this might be the beginning of an information visualizer that visualizes pages as a function of different keywords that appear. |
Friday, October 22
Lab 8 (on jar signing)
Tuesday, October 26
Answered programming questions related to Project 4.
HTMLParse4 (example code written during Q&A)
Thursday, October 28
Slides for Lecture 9 (process intensity)
Chris Crawford’s original essay on process intensity
Greg
Costikyan’s essay on interactivity and process intensity (scroll down to
Tuesday,
Michael Mateas’ essay on AI-based art (the essence of the computer as an expressive medium is computation)
Discuss: Computing Machinery and Human Intelligence - Alan Turing, NMR pp. 49-64.
Discuss: From Computing Power and Human Reason - Joseph Weizenbaum, NMR pp. 367-375.
Friday, October 29
Project 4
Due
Tuesday, November 2
Project 5
The field of artificial intelligence (AI) builds computational systems that model the intelligent behavior of people and animals. AI architectures can be extremely generative, able to produce complex responses to environmental changes, including user interaction. In computational art, AI approaches have been used to build work ranging from robotic sculpture, to drawing and painting generators, from generative interactive stories to music composition. In the popular art form of computer games, AI approaches are used extensively to build tactical and strategic opponents, non-player characters, and player modeling systems. In this project we will explore how a simple agent architecture, the Breitenberg vehicle, can produce complex agent behavior. Build a collection of Breitenberg vehicles that respond to each other, to objects in the environment, and to player interaction.
Due:
Tuesday, November 16
Discuss: From Soft Architecture Machines - Nicholas Negroponte, NMR pp. 353-366.
Discuss: From Plans and Situation Actions - Lucy Suchman, NMR pp 599-612.
Thursday, November 4
Assignment 6 (not graded)
|
A6-01 |
Modify the Braitenberg vehicle so that it has a different visual appearance. |
|
A6-02 |
Create multiple simultaneous Braitenberg vehicles that have different rules. Currently, there is only one type of vehicle active at a time. |
|
A6-03 |
Create a vehicle that responds to multiple sense modalities. The best way to do this is to overlay multiple grounds, each with its own source types (e.g. sound, light, heat, etc.). Create a class to represent sensory fields (grounds). |
|
A6-04 |
Make vehicles also be sources, so that vehicles respond to each other. One way to do this is to place a moving light source on each vehicle. You can choose to make the light source visible, or sum the moving sources into an invisible sensory field (if you don’t want glowing circles or some such appearing around vehicles). |
|
A6-05 |
Have your vehicles interact with the environment in some way. For example, when a vehicle runs into a source, perhaps it destroys the source. Other vehicles could create sources. Vehicles could lay trails that other vehicles respond to. A vehicle could have multiple ways of moving (flying under certain conditions, moving on the ground under other conditions, etc). |
Friday, November 5
Tuesday, November 9
Discuss: Using Computers: A Direction for Design - Terry Winograd and Frenando Flores, NMR pp. 551-561.
braitenberg3.zip (Defines a SensoryField class)
Thursday, November 11
Friday, November 12
Tuesday, November 16
Discuss: Video Games and Computer Holding Power - Sherry Turkle, NMR pp. 499-514.
Discuss: The Six Elements and the Causal Relations Among Them & Star Raiders: Dramatic Interaction in a Small World - Brenda Laurel, NMR pp. 563-573.
Discuss: From Theater of the Oppressed - Augusto Boal, NMR pp. 339-352.
Thursday, November 18
Project 6
While computer-based interactive games (a.k.a. video games) have been a pop-cultural force since the arcade scene of the late 1970s and early 1980s, in recent years video games have been recognized as a major emerging art form, poised to have as much cultural impact on the 21st century as cinema did on the 20th. The game industry is making Hollywood-sized amounts of money, with designers of the most popular games achieving a geeky sort of celebrity. Museums and galleries are offering exhibitions of “art games”, computer-scientists are beginning to treat games as technical objects worthy of serious attention, and in humanities departments around the world, games studies is a hot new topic. In short, something is going on, even if we don't yet know what it is or how to talk about it. What kind of game would you create? Create a simple game.
Due:
Friday, December 3
Friday, November 19
Tuesday, November 23
Discuss: The Lessons of Lucasfilm's Habitat- Chip Morningstar and F. Randall Farmer, NMR pp. 663-678.
Discuss: The Work of Culture in the Age of Cybernetic Systems - Bill Nichols, NMR pp. 625-642.
Thursday, November 25
Thanksgiving
Break
Tuesday, November 30
Thursday, December 2
Last day of
class!
Friday, December 3
Project 6
Due