Luciferjones

or Vertical Editing
Constructing meaning through video motion graphics and compositing.
Essay by Mike Jones

A character in the stage play Mongrels, by the eminent, but sadly late, Australian playwright, Nick Enright, proclaims that the definition of poetry is “maximum meaning in minimum content”.

For the conceptual, creative and intensely human process that the creation of poetry is, this is about as good a description of any. Certainly it encapsulates in broad terms the role of poetry as distinct from plain prose where information is laid out (generally) in a linear sequence so as to be easily understood. Poetry aims at something more complex, more symbolic, something deeper and ultimately something more demanding of the reader.

In the world of visual media, Motion Graphics and Compositing can be readily thought of as the video equivalent of poetry. Motion graphics is to poetry what video editing is to prose. Both construct meaning, tell a story or convey information. But where editing visual sequences most often creates a detailed and expanded construction of events; motion graphics and compositing creates less tangible but (potentially) more intrinsically engaging visual stimuli compressed into shorter periods of time.

Editing is the process of creating meaning horizontally along a timeline from beginning to end by placing and sequencing images next to each other in a specific order. Motion graphics and compositing is the process of creating meaning vertically by combining images together in the same space and same time.

Motion graphics is a broad term used to describe a wide collection of processes, tools and effects for combining and animating multiple images together. These techniques are not set in stone and are continually evolving and encompassing other media creation practice to the point now where motion graphics can encompass everything from simple overlays to 3D modelling.

So where do we see all this visual poetry? The most common, and often best, examples of motion graphics and compositing can be seen in title sequences to television dramas. Courtroom drama series The Practice has a very effective title sequence that combines headshots of each of the main cast members blended and superimposed with urban city images in a quickly cut montage. All these images have then been overlayed with flickering and broken ‘noise’ resemblant of very old 16mm film - scratches, dirt and jitter. Text has then been added along with further still photographic images of police arrests, handcuffs, courtrooms and the scales of justice. At no point are these images assembled in the usual linear sequence (that is, one after the other), rather all the shots and images are combined into the same visual ’space’; blurred and blended into each other.

Title sequences serve several purposes but at the core is the aim of conveying to the viewer the tone, feel, mood and atmosphere of the show they are about to see. The medium of TV is principally designed around efficiency. There is very little wasted time in a one hour TV drama. Scenes generally begin just before the key dramatic ‘moment’ and move quickly on to the next scene as soon as that key moment has passed. In this regard TV, due to time format (and the need to squeeze in ad-breaks) can’t afford to spend precious minutes and scenes establishing mood and tone for the viewer. TV title sequences aim to establish mood, tone and theme in the most immediately engaging and efficient means possible.

Compare The Practice title sequence to that of Buffy the Vampire Slayer and then to current affairs intros such as the excellent motion graphics sequence to the ABC’s 7:30 Report. All of these seek to quickly bring the viewer into the appropriate mindset for viewing the respective program. All do so in uniquely different ways that obviously seek to reflect both the audience demographic and the themes of the program.

Boiled down to essentials, the process of making a movie (in whatever media form that may be) involves two core theoretical elements; Creating Context and Making Meaning. The movie must make meaning by examining an issue, theme, story, idea that is meaningful and relevant to the viewer. And the movie must also create a context (time, place, character, situation) by which the viewer can experience or engage with the meaning. Comedy, tragedy, drama, documentary; it doesn’t matter. All good movies will seek to achieve these two central ideals.

Motion graphics and compositing tools and techniques provide the means for artists to make meaning and create context in some of the most concise, efficient, emotive and engaging ways available. As Nick Enright said, Maximum meaning, Minimum content.

All that out of the way, getting into and using motion graphics tools and techniques is, as with all digital media production, a fusion of both technical and creative knowledge. Mr. Left brain, I’d like to introduce you to Mrs. Right brain.

The Technical Stuff
No other art-form or creative endeavour has the potential to be so decidedly hamstrung by lack of technical knowledge as digital media and video production. All those great ideas floating around in your head are no good if the technical understandings are not there to allow them to be realised on the screen.

To fully appreciate how far this technology has come one need only look at the relative costs and means of motion graphics production. Less than ten years ago the act of creating multiple layers of animated video was the domain of hardware supported systems, costing tens of thousands of dollars, and buried away in dark corners of TV studios with production complexities understood only be a few sun-light starved technicians.

Large hardware based systems quickly gave way to more flexible, accessible (read: affordable) desktop software applications such as Adobe’s After Effects, and the power to create complex motion graphics entered a, relatively, more mainstream environment. That said, motion graphics tools, at this point, were still divorced from truly mainstream use in areas such as secondary education and domestic desktops.

Contrary to the popular belief that necessity is the mother of invention, the use and uptake of technology is quite often driven, not by the need to do something and subsequently developing the technology that will allow it. But, rather, by first having the means of doing and then finding the application that takes advantage of it. The best example of this being the ever upward trajectory of CPU speeds. In this case, rarely is it the software demanding faster processing but rather software developers rushing out new applications that take advantage of the new power available.

This has certainly been the case with non-linear digital editing systems. As soon as CPU speeds were fast enough a plethora of domestic level NLE’s became available, such as Apple’s very popular I-Movie, and before too long everyone was able to cut, copy and paste a movie together. A video editing utopia had arrived. But nobody told the CPU that nirvana had been reached, speeds didn’t stop their upward climb and very quickly it was realised that even a common, non-purpose built, computer held far more power than was actually needed to render DV footage. What to do with all that power, asked the software developers? What more could users want? The answer has been, for a great many non-linear editing systems that have hit the shelves in recent times, to embrace the world of motion graphics and compositing right on the editing timeline.

Motion Graphics is a very broad term encompassing an ever expanding range of techniques and tools but there are two core elements that are central to all motion graphics production; Keyframing and Transparency. Almost all the effects possible within a motion graphics application are created through combinations of these two elements. What’s more is that these two techniques are common to a great variety of software production tools aside from Motion Graphics, including Flash, 2D animation and 3D modelling.

Keyframing has its roots in traditional, hand-drawn, cell based animation. Here each individual frame is drawn through the progression of an animated scene, frame by frame for the full twenty five frames per second. Needless to say all animation, not least of all hand drawn, is a painstaking and time consuming process. It was common practice however to not have the head animator go through the task of creating each and every frame themselves, rather the head animator would draw only the keyframes. What defines a keyframe can vary but generally it involved the Key, or central moments or movements in a sequence. In practice it usually meant the head animator might draw only every tenth frame. The in-between frames would then be filled in by assistants and apprentices to complete the animated sequence. This process became known as tweening.

In the digital age, with assistance of our trusty computers, this process of creating keyframes and tweening them together is far easier, more flexible and certainly more powerful. Keyframing, in regard to computer graphics production, is still a means of animation but is a broader process that can be applied to an enormous variety of parameters aside from just movement. Colour, shape, position and effects are all elements that can potentially be keyframed to control their properties over time.

In motion graphics production keyframing works by setting a particular point in time as a keyframe and instructing the computer to give the clip or video element at that point a particular set of parameters: an example might be moving an image off screen to the left so it cant be seen. A second point is then created and the parameters altered: the image located on the opposite side, off screen to the right. When the video is played back the computer will tween between the keyframes to create a fluid animation; the image moves from left to right across the screen. A simple two keyframe movement animation.

From this simple beginning more and more complex animations can be built. A second keyframe timeline might be inserted to control a colour balance adjustment or effect. In this case the image begins off screen with normal colouring but as it moves across the screen from left to right its colour balance animates simultaneously becoming more saturated with red. Highly complex animations involving multiple layers and articulated movements are simply a progression of this simple process.

The second major motion graphics element, working closely with keyframing, is transparency. By managing, masking and creating areas and levels of transparency within images, layers of several images can be merged together to create new composite visual media.

There are essentially two ways to build transparency within an image. The first is to use a compositing mode that uses a pre-set formula to combine pixel colour values. In a digital RGB (Red, Green & Blue) colour array each of the 16+ million colours has a particular numerical value, the compositing mode combines these values in a particular way. For example using an Additive compositing mode adds the values of the two images together to create a new image. Subtract takes the values of one away form the other. Hard Light will allow only the strongest colours to show through. And so on.

In this way transparency between images that allows them to combine is created in a non-uniform manner that is unique to each image.

The other, more precise, manner of creating transparency is by use of an Alpha Channel. Computer screen and TV images are made of three colour channels; Red, Green and Blue. As opposed to the subtractive primary colours we all learnt about at school (Red, Blue and Yellow that when added together produce black) RGB are the additive primary colours that when all combined give white. To create transparency in an RGB image a fourth channel needs to be added, an Alpha channel.

The Alpha channel is itself an 8bit greyscale, black and white image that, rather than being visible itself, acts purely as an invisible mask to the image below dictating what areas of that visible image will be rendered transparent. The Alpha channel uses its black and white contrast to dictate this transparency - Black areas are see-through, White areas are opaque.

Alpha channels can be created in a number of ways. For simple picture-in-picture effects a basic geometric shape Alpha channel mask might be used to ‘cut out’ an area of an image and allow another to show through.

To create a mask of a less uniform shape more free-form lines or paths can be drawn on the image to dictate the bounds of the mask.

Aside from shape however, Alpha channels can also be created based on a particular pixel colour range rather than a defined shape. This process is called Chromakey, but is more commonly referred to as Blue Screen. In this technique an actor or subject is shot in front of a uniformly coloured screen (traditionally blue but more commonly green). The colour of the screen rather than a shape can then be selected to form the basis for an Alpha channel mask. Selecting a transparency mask by this method is very powerful as the transparency created is not fixed to a place or shape but will move with the image continuing to mask-out only the background blue or green areas of the image.

Motion graphics is a complex area but in essence there are just two major principals, keyframing and transparency that, combined with software effects plug-ins, serve as the building blocks for an infinite number of visual possibilities.

What was once very much the domain of dedicated software has now been firmly embraced as common tools encompassed by non-linear editing systems. And it stands to reason that if the tools are there, then before too long people are going to want to use them. But does the new found ability to construct video vertically suddenly render all the rules and principals of visual narrative obsolete?

The Creative Stuff
There are two core visual media theories that have survived the century of film making and continue to be the foundation of our visual understandings and literacy today, in the digital age. Mise En Scene, the act of ‘putting onto the stage’, encompasses all our notions of framing and composition. And Russian Montage, most notably articulated by Sergi Eisenstein, gives rise to all out sensibilities of sequence, actions and context. Never employed in isolation, these two schools of thought have together defined the way we now both create and articulate visual media.

But what of Motion Graphics? Does the recently popularised ability to fuse otherwise divergent media and imagery together suddenly render our current understandings of Mise en Scene and Montage obsolete?

When Sergi Eisenstein, and his compatriot Russian film theorist Lev Kulesov, first articulated the principals of Montage in the 1920’s they expressed the idea that meaning was not inherent within the frames of the images shown in a film. Rather they proposed that the image itself potentially held no meaning at all. Rather that meaning was constructed in the mind of the viewer through the sequencing of images in a defined order.

Kulesov famously held an experiment with some of his students where he showed two versions of a three-shot sequence. In the first version the sequence showed first a close-up of a man looking off-screen, a girl laying on the ground with an injured leg and a third, ‘reaction shot’ of the man again. Kulesov’s students, when asked to comment on the ‘narrative’ being conveyed by the sequence, remarked that the actor displayed a sense of pity and sorrow for the obviously distressed girl.

For the second sequence Kulesov replaced only the second of the three shots, the shot of the girl, and substituted it with a static image of a bowl of soup. To this his students commented that the actor displayed a sense of longing, poverty and hunger for the steaming bowl of soup.

What Kulesov was getting at here, and what Eisenstein would later articulate so well in Battleship Potemkin, was that meaning and context is not inherent in the images, it is constructed by the sequence and order in which images are shown.

On the surface this idea would seem to work against the principals of Motion Graphics where the central idea is to combine images and media together in the same ’space’ and show them, in effect, simultaneously.

Montage is essentially the art of creating context, a context which doesn’t necessarily exist within the image itself. In the example above the ‘context’ of hunger, poverty and longing did not inherently exist in the rather banal photo of a bowl of soup. That context was created in the minds of the viewers by the arrangement of the shot between those of an on-looking, blank-faced, person. An image too that in itself didn’t have a context until it had a bowl of soup next to it.

This idea is just as relevant and central to the construction of Motion Graphics. A video image of a person walking alone up a long set of stairs could mean anything. If that frame is then overlayed with the half transparent image of a set of slowly rotating scales of justice; which in turn has a third image overlayed, that of close-up shot of a handgun being fired, then all of sudden a very specific mood, tone, style and context has been created. The same scene and context could well have been created by showing via montage those three images in a sequential order, one after the other. But that would far less efficient and run against the notion of creating a compressed, poetic image. Maximum meaning, minimum content. What would have taken three separate images, three times as long can be done with Motion Graphics and compositing techniques as one shot in one third the time. Each layer of a Motion Graphics sequence, just as each shot in a Montage, has the potential to dramatically alter the meaning, interpretation and context for the layers around it.

The tools and techniques have varied (and will continue to evolve) but the principal of Montage, of constructing meaning through juxtaposition and sequence, is as relevant to visual media creation in the digital age as they have ever been.

Where traditionally Montage seeks to create meaning and context in the mind of the viewer ‘between’ the frames of a sequence, Mise en Scene is the practice of constructing complex meaning ‘within’ the individual frames of a film by the processes of composition and arrangement.

Mise en Scene seeks to employ a range of dramatic devices and media elements in the conveying of narrative within the frame borders of a particular shot. Principally this involves the composition of the shot itself; How far is the camera from the subject? How large or small is the space around the subject? Where is the subject placed within the frame? Where is the subject placed in the frame in regard to other actors or subjects? What objects are in the room with the subject? In asking these questions Mise en Scene is the first to seize upon the power of different shot types to convey meaning; Wide-shot, Mid-shot, Close-up, Two-shot, Pan, Track, low and high camera angles. All these can influence the viewer’s perception of the action taking place within the scene.

Along with composition however, the Mise en Scene also takes into account other media devices present within the space of the frame such as lighting and sound. Two scenes, otherwise identical, could be read very differently if one were starkly up-lit from below in red light and the other was flooded with fluorescent tubes from above.

In the creation of traditional film, Montage and Mise en Scene are never mutually exclusive but there is often a bias one way or the other depending on such factors as the particular style of the director or the genre of the film. In Motion Graphics the line between the two however is much finer. Mise en Scene takes a great many of its techniques from traditional two dimensional art-forms, principally painting and photography, in following and employing rules of form and visual aesthetic. Montage on the other hand is ‘pure cinema’, unique to the moving image. In the practice of creating Motion Graphics however the two meet on a much more uniform ground then ever before.

The compositing of images together into more complex, multi-layered compositions can serve in the ultimate form of Mise en Scene. Now not only are cinematic elements able to be arranged and composed based on their physical or spatial location within the frame but also their ‘metaphysical’ arrangement within the shot - their opacity, their movement, their merging with other elements, their position as a layer amid and amongst several layers. It can be said motion graphics and compositing adds a fourth dimension to the Mise en Scene; height, width, depth and now, layer.

Taking this a step further into a very post-modern perspective of creation through arrangement and re-contextualisation; motion graphics and compositing allows for the assembly of pre-existing video and graphic elements into a new media form. In this way each visual element used in the arrangement of the motion graphics composition carries with it its own original Mise en Scene - which in turn has its own connotations and context. Every time a visual element is added as a layer its’ own Mise en Scene is re-contextualised, re-interpreted and can ultimately find a new meaning in the newly assembled composition.

This can be viewed as very much akin to a great deal of contemporary, post-modern music particularly the art of turn-table DJ-ing and electronic arrangement. Here the DJ, as musician, uses not the notes of a particular instrument as the basis from which a performance or composition is built, but rather employs small music elements or sounds derived from existing recordings. Known as samples, the DJ might take the drum break from one record and the bass from another to combine into new musical piece. In this manner a DJ record is essentially a new record made from the assembled parts other records, re-contextualised, re-intrepreted, often unrecognisable from their original source.

This very closely resembles the possibilities and theory behind the re-contextualisation of visual elements in motion graphics that links very closely with both the cinematic theories of Mise en Scene and Montage.

Post-modern theory aside all the same rules and guidelines for visual aesthetics still apply in regard to motion graphics. The arrangement of an image using design principals of colour, shape and tone, photographic ideals of the rule of thirds and focal depth, identifying the strong points of an image and juxtaposition between simultaneous visual elements. The creation of motion graphics is as much an act of visual art and design as it is of video editing.

From an educational perspective, motion graphics and compositing provides an enormous scope for the exploration of the construction of meaning and visual narrative: making meaning and creating context. More broadly it goes to the heart of a true understanding of media literacy; how meaning, context and information can and is manipulated, controlled and effected through acts of arrangement, placement, sequence and juxtaposition. Moreover, motion graphics provides one of the most flexible and accessible means by which to create media that is open and conducive to lateral thinking and free creative thought.

As the tools for motion graphics creation become more and more commonly fused with standard video editing and media creation applications, potentially limitless opportunities exist for free experimentation with the moving image in new and exciting ways. Visual Poetry - Maximum meaning, minimum content.