Category Archives: digital genres

Digital Fiction, “transmedia” moments, and mood: What we can learn from the opening scene of Bela Tarr’s The Turin Horse

shazia hafiz ramji

On the train home after the first day of the Digital Pathways workshop, I was left with the question: “Why do I feel the way I feel when I read or watch something really great? Why don’t I feel it often? What exactly is it that I feel?” I thought about the conversations in class, and concluded that we discussed this feeling in terms of the “experiential,” and how this can be achieved by incorporating different media in a web platform that aggregates. I realized that I understood this ambiguous feeling i.e. the “experiential,” as the “mood.”

In Seven Types of Ambiguity, Empson writes: “Critics often say or imply casually that some poetic effect conveys a direct “physical” quality, something mysteriously intimate, something which it is strange a poet could convey, something like a sensation which is not attached to any one of the senses. This may only be a statement of how they themselves applied their conscious attention when reading the poem; thus a musical chord is a direct sensation, but not therefore unanalysable into its separate notes even at the moment of sensing…. But it may mean something more important, involving a distinction between “sensation” and “feeling”; that what the poet has conveyed is no assembly of grammatical meanings, capable of analysis, but a “mood”, an “atmosphere”, a “personality”, an attitude to life, an undifferentiated mode of being.

The opening scene from The Turin Horse by Bela Tarr is almost six minutes long. It is composed of one long shot. The first thing we see is a blank black screen, which is accompanied by a voice (the director’s) that tells us the story of Nietzsche witnessing a horse being whipped. Witnessing this scene supposedly drove Nietzsche to silence and madness. The director’s voice asks “But what of the horse?” So, we start the film (about the horse, the horse’s owner, and the owner’s daughter) without an image, but with a voice, and a story, that is a reference to another story.

The scene which fades in begins in the middle of the action. The action is a man (sitting in his horse-drawn cart), and his horse making their way through a stormy day. It sounds boring. But the scene isn’t just the image—the music completes the image: the horse’s legs are not in the frame, they only hint at strain, but the music—cellos and violins in somber notes—completes the image of the horse’s tendons and muscles under strain. I can go on and on trying to explain and analyze the nuances of this scene, but this would be missing Empson’s point. In my understanding, the reason it works is because the mood is very strong. Using Empson’s passage, this scene is undifferentiated and embodies an attitude to life (that of suffering, repetition, obligation, labour): there is a tightly woven unity to all the elements, down to the last few seconds during which the “real” sounds of the carriage on the path gradually replace the music. This element of surprise increases the immediacy. I know this, because it was “experiential”; my mood flipped, completely. I wept for a while, watched the opening scene many times, eventually watched the film, wrote a poem shortly after while weeping and eating potatoes as the characters did in the film, cancelled work the next day, and went to sleep at 8 a.m. I have never before or since missed work because of a piece of art.

Because of my interactive experience (interactive: generative, active response and participation) with this film, I want to argue that this multimedia scene is a forerunner to the successful and effective “transmedia” moment because of its mood, which is achieved through unity—created through ambiguity.

In class, we were constantly thinking of our senses, and how to create opportunities for interaction by engaging the senses, whether it was by clicking an image as a point of access, or hovering over the text, which expanded to a video.

On realizing how mood is an ambiguous undifferentiated feeling, I think that transmedia storytelling offers many opportunities, but to execute an awesome piece of fiction, there has to be ambiguity. This is how we get in. Ambiguity is simply an invitation to understand, and this is what interactivity does—it invites you to understand by making sense of it yourself, just like I responded to the scene from The Turin Horse by writing a poem.

Why not begin with complete ambiguity as the entry access point (instead of having an image, or text, or sound, or clicking a link, or any other form of interactivity that has already been established online)? Why not create an “atmosphere,” a space that makes you ask “What am I still doing here (on this site) even though I have to be out of the house in five minutes?”

The CBC Radio 3 magazine archives are effective in mimicking this mood. The page pops into music, and you’re simultaneously seeing the background. Before you can even ask “What is this?” you’re already feeling. There is no time to right click and “open in a new tab.” It’s just there; it’s immersive first—the ambiguity is an effect of totality that is very quick—and it is interactive second, but the realization of these two occur at the same time.

What does DF look like?

What does DF look like? Back in the 90s, the digital fiction works I was most familiar with were those published by Eastgate Systems: they were black and white works with little boxes around the words, indicating hypertext links. They didn’t look like much, really… the interesting part was the hypertext. Later, DF became more about multimedia, on the one hand, and code and algorithms on the other, the look changed; it became more self-consciously colourful, sometimes signalling its digital nature through the use of vibrant graphics and data visualizations. Often, green text on a black background evoked “computer.”

Some of this aesthetic was about the genre, and the late 20th century. Games from the same era share much of the look and feel. Some of that, though, merely reflected the presentational expression possible in one or other early authoring systems. Limited fonts, poor type rendering, 8-bit colour (and sound?)… many of these things are part of what people remember about DF’s early years.

As Flash became a standard toolkit for digital storytelling and expression, production values got a lot better. But 2000s-era Flash has its own characteristic look and feel; animated text, zooming views, standardized transitions were easy to achieve, and therefore became commonplace.

Steve Jobs ushered in a new era about 5 years ago, with the launch of Apple’s mobile platform and his banishing of Flash from iOS. That was arguably a good decision, as it lent a huge amount of energy to the development of open web-based standards to take Flash’s place. Today, we inherit a vast open web platform, built on simple open technologies like html, css, and javascript. The web platform is more than 20 years old now, and it begins to show the signs of maturity — in some areas at least.

The web’s agenda, stated or unstated, has been to absorb all other publishing and presentation systems. It has, I think, done so, at the expense of elegance; web authoring is still a bit of a dog’s breakfast, simply because there are so many options, so many toolkits and frameworks, and whatnot. But just about anything is possible now. Good typography is within easy reach on the web now, as is native support for animation, vector graphics, video, and audio. Ubiquitous frameworks like jQuery provide a solid foundation of basic interactive functionality. And designers have begun to think mobile-first, a move that is important not just because we all carry phones, but because it provides a way of throwing off the “web design” conventions and aesthetic of the first decades.

So what should Digital Fiction look like in 2014? Especially given that the open web platform places very few constraints on designers. The lack of constraints is both a blessing and a curse; those early experiments found a ready-made genre and style in the narrow limits afforded by early platforms. In the absence of such things, how do begin to design for wide-open media?

 

 

The Silence of the Ebooks

In the midst of an animated discussion with a grad student, weighed down with armfuls of books on media archaeology and criticism, I was utterly stumped by my inability to locate something I’d remembered reading last year: a fascinating chapter on John Cage’s 4’33 (of silence) and other interesting examples of blank media that still, somehow, have content. I struggled, I googled, I searched high and low on bookshelves, hard drives, gmails… all in vain. Eventually I turned to googling bare keywords in the hope that somebody, somewhere would refer to this text. Or maybe I dreamt it? Seemed plausible: a dream about a non-existent article about a non-existent song.

It turns out the truth wasn’t far off that. What I recalled reading was in fact a book – Craig Dworkin’s No Medium – which I read last summer, in one of the two pine muskoka chairs I had built and which sit in the shade of a katsura tree in my front yard. That part of the memory was crystal clear. How come I couldn’t remember what I’d read?

Eventually the details fell into place. I read it on the Kindle, which is of course why it didn’t turn up in any of my searches. And indeed, Craig Dworkin’s core argument, beautifully illustrated by literally hundreds of examples of blank, non-existent media, is that there is actually no such thing as a bare medium; rather, media are interpretive occasions, or at least eventful: their context is gathered, rather than given.

When I remember reading a book, I remember so much more than the text. I remember the physicality of the book, and where (and when) I was when I read it, and probably traces of whatever else was going on at the time. How the ebook messes with that memory! By erasing the physicality of a text, or at least by submerging it in a bland and anonymous container, the experience of reading – or rather, the experience of having read – is truncated.

If there is to be an embodied sense of a digital text, it must necessarily come from the text itself. I wonder if – putting ebooks aside – it would be possible to create an immersive digital storyworld rich enough to flesh out that part of the experience of reading, to provide the sense of a time and a place and a surrounding context. Is this perhaps easier with fiction than nonfiction? Because fiction provides a location, an immersive world to inhabit? And if so, how to provide that grounding in a nonfiction reading experience?

On Writing Digital Texts

Since 2001 I’ve been writing collaborative multimedia digital stories alongside my work as a writer of literary fiction.  Working in these parallel fields has served me well as writer, mostly accidentally. For example, one unintended consequence of being part of the small team that produces the digital story, ‘Inanimate Alice’, a work referred to elsewhere as ‘the world’s first born-digital transmedia pedagogical blockbuster for children’ (yes, we have a problem figuring out what to call these hybrid works), is that I often get asked to speak at digital publishing conferences. An unintended consequence of that is that I now know much more about digital publishing than I could have anticipated. Yet another unintended consequence is that I’ve recently become a digital publisher myself: working with a publishing consultant, I’ve created new ebook editions of four of my backlist literary novels under an imprint called, yes, Kate Pullinger Books (I suggested Kate Publishinger Books, but that was rejected).  So now I’m a transmedia collaborator, an author of literary fiction, and a publisher. C’est la vie.

One notable change over the past couple of years is that while these worlds which, as stated above, were largely parallel – in the same way that child psychologists observe toddlers engaging in ‘parallel play’: she has her toys, he has his, they are in the same room, occasionally they glance at each other suspiciously, but that’s about it – they have now begun to, well, not exactly merge, but at least they’ve grown up enough to exchange a few toys.

A few examples from my own work: firstly, a mainstream publisher playing in the realm of digital experimentation. My new novel, Landing Gear – literary fiction to the core – grew up out of www.flightpaths.net, the digital fiction I created with Chris Joseph. Doubleday, my Canadian publisher, was inspired by the novel’s pre-existing digital footprint to create a raw API from the first 30 pages of the novel. They then used that raw API to create an interactive map of the novel that pins extracts from the novel to the actual locations relevant to the text.

Second, two government bodies in two different countries investing in digital stories and digital pedagogy: ‘Inanimate Alice’, a project that has been not exactly dormant but certainly quiet for the last four years has just received two new tranches of funding to develop the next two episodes, and to create a companion set of interactive stories for language training in schools.

And lastly, my new digital collaboration, Letter to an Unknown Soldier, has been commissioned by 14-18NOW, a UK body set up to respond to the centenary of WW1 through a series of artists’ commissions. Letter to an Unknown Soldier is an attempt to create a new kind of war memorial, a digital memorial made of words. Open to everyone, it allows for collaboration on a massive scale; with its substantial budget and team of 18 people involved, it’s a far cry from my usual working method of me and a web artist, alone in front of our respective screens, communicating via email and Skype.

So, in conclusion – well, there isn’t really a conclusion. Writing is evolving. Reading is evolving. And publishing is evolving too. I, for one, find it incredibly exciting.

On Reading Digital Texts

“A BOOK [in the medium sense],” writes Charles Catton in a recent Medium piece on the so-called ‘enhanced’ ebook, “is a means of transporting the author’s words to the reader through the act of reading.” Whatever he might mean by “in the medium sense,” (see below) I think Catton has it backwards: a book is a means of transporting the reader into the author’s world. And I don’t mean “transporting” in some romantic, voodoo-ey sense of “I was utterly transported…” Rather, I think it’s high time we in the publishing world took stock of what it really means to read something.

Reading is not fundamentally about the delivery of information. It’s just not. Reading — be it escapist fiction or biomedical reference — is a profound experience of going inside somebody else’s mind. It is a transposition, a transubstantiation. It is virtual reality, as anyone who has ever read a novel can see, if they only stop to think about it. Even the “dipping in” kinds of reading we do when we read nonfiction and reference works is still this kind of experience; you internalize somebody else’s sentences and they unfold in your own head, (almost) as if it were your own experience.

The idea that books – or writing, or any other medium – is, as Catton’s subhead says, “a Means of Transport” has surely been thoroughly debunked by media theorists. Marshall McLuhan made himself famous forever by pointing this out, and the field of communications & media studies exists to explore the dynamics beyond this. Nor is writing a ‘delivery mechanism,’ as a generation of poststructuralist critics took pains to point out (Roland Barthes is my favourite here). Nor, even, is publishing a ‘delivery mechanism’ for content; read John Seely Brown & Paul Duguid’s seminal article “The Social Life of Documents” for a concise primer on how documents don’t just circulate in society; they bring society into being.

Reading is about experience, which is something that the people driving the ebook movement so far – publishers, retailers, and technologists alike –have so egregiously avoided, even while at the same time the discourse around User Experience Design is the ascendant on the open web. I’d go so far as to say that most of what’s “wrong” with the ebook world today is that things have been designed around the retail experience, the production experience, the gadget-sales experience, with very little attention to what reading means anymore.

And so we come to the ongoing debate about whether ebooks need to be “enhanced” or not. The pro side argues that ebooks are boring and that people would like extra stuff, like the additional features on a DVD, or at least links to the Wikipedia article. The con side counters that all that stuff disrupts and distracts from the immersive reading experience. As if that was a thing. As if the “sustained silent reading” we did in gradeschool were a course unto itself, like math or social studies.

As if we knew enough about reading to even begin to make a case like that. Well… we may not know much about reading, but we know what we like, right? For starters, we like what we already know. After all, what we know is of course the measure of everything we don’t know. And we can so easily imagine what we don’t like… the spectre of videos popping up just as we get to the good part is a good bugbear, worth repeating endlessly. We have all read a book, and so we are all qualified to sit back and armchair quarterback what might be the right thing to do for books.

Yet we don’t know much… other than what we’ve always known and liked. So, for instance, novels are 300 pages because they’ve always been 300 pages. Short stories are short enough so that 10 of them fit in a 300-page book. And even though people have begun to experiment with different-sized texts (e.g., ‘singles’, ‘shorts’, and the new ‘long form’) we still so underestimate the extent to which we conflate genre with format.

New formats are already here, all around us. The web-page article is a new format. Blogs are a new format. Wikipedia is a new format. Image-macro memes are a new format. But these things are also, and more importantly, new genres, which is something we don’t think about nearly as much. And these new genres are just the tip of the iceberg. Look around.

And yet we continue to discuss the book and the ebook as though these were God-given categories. Eoin Purcell, who wrote the post that inspired Catton to write his post, argues that the generic ebook is evidently “good enough” to capture 30% of the market, and considering the so-called ‘enhanced’ ebook, asks:

…whether anyone is looking for such innovations. The question those seeking to make more exciting and innovative products from books have to answer is straightforward; will those new products entice ebook readers away from ebooks, entice print readers from print books when ebooks didn’t do so, or entice new readers to read where ebooks and print books didn’t? —Eoin Purcell, How Different are Books Digitally?

As much as I like Eoin Purcell’s writing and perspectives, here he’s sadly invoking a zero-sum game and offering a narrowly functionalist assessment – not just of books, but of reading itself. We all already read vastly greater quantities of non-book, non-‘published’, non-traditional digital text, in a staggering array of both formats and genres. This is not in competition with the book-as-a-novel. Reading has already changed. The innovations are all around us. The better question at this point in time isn’t whether we want or need innovations, but which innovations we want. That’s our business now.

There is no pure, unmediated literary format. The traditional book is itself technology; it affords and enshrines a particular set of conventional reading experiences. It is historically and culturally situated; there’s nothing God-given or essential about it.

James Bridle today posted a thoughtful piece in Grafik, called Read Only, in which he writes,

Perhaps the line of the text is no longer a central metaphor we need to cling to. Perhaps it never should have been.

Bridle too weighs in on the so-called enhanced ebook, going to a place he’s been before, in which he distiguishes between the experience of the text itself from the experience of other kinds of media elements (as though they were alternatives), and in doing so praises apps like Instapaper and Readability for putting the emphasis on the literary experience alone. Bridle says, “the real value of these apps is how they remove all the crud of web design around a good piece of writing”

That’s all very well (and I live in Instapaper on a day-to-day basis), but it again papers over our cleaving to traditional text-only genres. It only takes a moment to recall the picture books we loved as kids, and the coffee-table books we shell out big bucks for still, to recognize that there are legitimate genres that mix text with, say, images. We don’t talk about photographs getting in the way of the text in a photo essay (or, for that matter, a photography book. Or for that matter, a listicle), because we collectively understand that a photo essay is a genre that is the result of the interplay of text and image.

But Bridle says something more sophisticated next, which belies his supposed adherence to the uninterrupted flow of words:

because text, however we may imagine it, is not linear, and it never has been. Just as the tiles of the slippy map are near-instantly rendered as we move over them, slotting into place before we are even aware of them, creating the illusion of a continuous plane seen through the narrow gap of the browser window, so the mind continually enacts understanding, rather than passively receiving it. That understanding is composed of many multi-layered understandings which go far beyond the text itself, into memory and association, into anticipation and extrapolation

That understanding, “composed of many multi-layered understandings”… that’s where the innovation in the reading experience is going to come from. As it always has before.  That’s where publishing needs to open up and look to writers, artists, and creators generally for some new perspective. That’s a major part of what this Creating Digital Fiction Workshop is interested to explore.

Digital fiction — what’s in a name?

When we were working on the initial descriptions of this workshop, we went back and forth over what to call it. Interactive fiction was used early on, but that term has come to be associated with the “twisty little passages” style of immersive text-game worlds — the Wikipedia article on IF gives some good detail. But as much as this is near and dear to my heart (I was a MUDder back in the day), it’s not what we’re doing this summer.

Kate Pullinger’s existing digital works—notably Inanimate Alice and Flight Paths—are ‘interactive,’ and ‘fiction,’ but they’re not immersive gameworlds in the sense of IF. They’re also multimedia fiction, though that buzzword has seen better days as well. So we’ve gone with “digital fiction,” which perhaps is ambiguous enough to provide a pretty big tent for us to work in.

Is there a better term?

UPDATE: just after I wrote this post, I discovered Kate’s 2006 piece in The Guardian, “Fact is, We need a better name for Digital Fiction