this is a comment…
Ok, let's have a chat about lossless!
After registering and loggin-in to the wiki you'll have a link below called Discussion. You know what to do.
This is best for non-synchronous chats. If you want to go interactive we can use skype and I'll paste the session on this page. Sound good?
Here is the transcript of a chat session with John
Here is the transcript of a chat session with JP and John
On Nov 18, 2008, at 4:05 PM, J.P. Sniadecki wrote:
1. Spent a few cycles with the piece, and noticed that when i keep my eyes fixed on a point, i perceive the modulating motion vectors as white against a black background. But if I view the piece by moving my eyes from point to point, I perceive discrete and instantaneous flashes of primary colors within the projection laid over, as it were, the white motion vectors. What in the projection causes this strange effect to my visuality?
These things all use tricks for rendering the color spectrum. Some are spatial (e.g. thin lines of red|green|blue compose each pixel in an LCD screen, film grain too) others are temporal (DLP), others mix light, you get the idea. Now I'm starting to think about a new piece…
1. Selection: in planning this piece, did you know you wanted something black and white? If so, why? And why did you select Berkeley?
As I recall the choice was suggested by what MPEG encoding does to water. We were experimenting with watery work (13 Lakes, Waterworld, Moby Dick) when Rebecca suggested Esther WIlliams. We started looking at water ballet pieces and she remembered this amazing dream sequence about the waterfall. We found a copy and the rest is history.
The B&W is significant mainly because it worked so much better than the technicolor Esther WIlliams file worked. Once we saw the Berkeley, and Ernst was able to render the shrill treacly audio into that eerie haunting track we didn't think we coud do any better. Rebecca may have more.
hi doug, rebecca -
i'm interested in the hints dropped here and there about phenomenology and psychology in your description of the project (the way digital video and compression formats manage our perceptions) and your phrase “compression as phenomenology”… would you be willing to elaborate on this?
john
hi john,
I'm going to summarize our spiel quickly in this box. please ask me for expansion and clarifications. -d
digital media has been forced by the economics of media and hardware into new representational strategies. example: a 100 minute hollywood film is composed of 144000 discrete 35mm frames plus at least a stereo audio track. scanning these frames at “2k” (which is on the low end of a scan meant for standard definition) results in something like 1.700Tb of data (that's 12mb/frame). the best lossless techniques reduces that number to about 400 dvds. how do you reduce the data to 1/400th of the data contained in a 35mm print?
1. start by tossing out color that we can't see–this is prejudicing the information towards the sweet spots of our perceptual kit. [this is a branch of psychology called psychophysics.]
2. now describe the image in terms of instructions for reconstituting each compressed frame. eg: a solid black frame may be described as the color black painted over every pixel. call this an algorithm for reconstituting each frame. [this is a lossless compression called run length encoding [RLE], and there are numerous variations on this theme.]
these two strategies gets your data down to maybe 40 disks. that's a reduction by a factor of ten. that leaves another factor of ten to go. what next?
3. try a hybrid of 1&2: apply the perceptual approach and spatial approach over time. if a pixel doesn't need updating over time then tell it to stay the same as it was in the last frame. that creates temporal streams of colors that we don't notice because they're not changing. be sure to update the whole frame every two seconds or so and you have a pretty good image that fits on a dvd.
[the audio signal is also reduced by simplifying the waveforms. this is another radical reduction, but it is technically and metaphorically much closer to the way that cinema has always compressed audio]
the combo of color reduction and spatial & temporal RLE is radically different than the 24 resolved 35mm frames that are blasted on the screen every second as it is with film. digital media is constrained picture and sound data sent as instructions to a graphics chip and projector which result in a performance of light and color. this results is what we call a phenomenology of digital media.
we like the way that this relates to the material manipulations of light and sound that brakhage et alia promote. this provides another link to the aesthetic phenomenologies discussed in the canon of experimental film.
note: I'm going to move the comment over to compression_as_phenomenology
thanks for this run-down. i was looking at the searchers piece again today, and thinking about some of the things you said here.
could you remind me, again, what the exact formula was in preparing this video?
if i remember correctly, you removed frames at regular intervals from the DVD compression, and this forced each section of the image to sort of wander until it finds something definable to latch onto?
in any case, the effects amazing, and made me think a lot about figure/ground relations, or the relationship between the human figure and landscape.
some things i noticed:
whenever a figure in motion is far enough away from the camera such that small, erratic, or unpredictable movements aren't that visible (or such that they can get lumped into a larger, simpler movement), the result is a motion blur. for instance, all the shots where the posse is riding off in the distance..
but whenever movement is less uniform (more unpredictable, more local?), those isolated sections of the image lock into clarity. there are a few moments that come to mind:
- somebody tosses something to somebody else, and the trajectory of the object gets blocked out with crisp definition. - horses legs and especially hooves moving (making bits of landscape behind clear) - sudden movement that 'wipes' the screen with clear definition
a few other things:
whenever there is a pan or tilt, the new portion of the space that is discovered isn't defined - what we get is a streak of color information from one side of the frame.
then suddenly the image snaps into clarity after several seconds. why this time interval? is that the first clear frame from your sample?
then there were anomalous blocks that would appear, as if from a different image. bits of sky that pop up over john wayne's chest, or dark segments that pop up over the sky.
etc.
am i right in thinking that for this piece, the crucial element in the original media is movement (rate of acceleration, uniformity/predictability)?
Hi John,
The searchers piece (L3) started as a proper mpeg2 (DVD) file downloaded from the net. We wanted to reproduce and get control of the effects we had generated for L2 1). We noticed that the absence of resolved frames seemed to generate the most compelling imagery, so we started looking for ways to do that to a file we had already downloaded.
These resolved frames are known as I-frames in the lingo. They're metaphorically related to the keyframes animators use to control animation, so we call them keyframes. We found a way to toss out the keyframes and let the mpeg2 file try to connect the intermediate frames which depend on the keys to render a clean video image. Removing keyframes allows the image to slide and drift.
It's not exactly a motion blur (technical reasons) more like a stream of color that's torn from the background. the encoder knows that it should update the image but it doesn't have any image information to paint there, so it drags the background along the motion path. Think of the vectors you can see in L4.
The streaking is most evident with camera tilts. The blocks at the edge of the image are repeated since there isn't any new visual information to paint in there. So we have something that looks like stretched taffy.