Author Archives: John Maxwell

What does DF look like?

What does DF look like? Back in the 90s, the digital fiction works I was most familiar with were those published by Eastgate Systems: they were black and white works with little boxes around the words, indicating hypertext links. They didn’t look like much, really… the interesting part was the hypertext. Later, DF became more about multimedia, on the one hand, and code and algorithms on the other, the look changed; it became more self-consciously colourful, sometimes signalling its digital nature through the use of vibrant graphics and data visualizations. Often, green text on a black background evoked “computer.”

Some of this aesthetic was about the genre, and the late 20th century. Games from the same era share much of the look and feel. Some of that, though, merely reflected the presentational expression possible in one or other early authoring systems. Limited fonts, poor type rendering, 8-bit colour (and sound?)… many of these things are part of what people remember about DF’s early years.

As Flash became a standard toolkit for digital storytelling and expression, production values got a lot better. But 2000s-era Flash has its own characteristic look and feel; animated text, zooming views, standardized transitions were easy to achieve, and therefore became commonplace.

Steve Jobs ushered in a new era about 5 years ago, with the launch of Apple’s mobile platform and his banishing of Flash from iOS. That was arguably a good decision, as it lent a huge amount of energy to the development of open web-based standards to take Flash’s place. Today, we inherit a vast open web platform, built on simple open technologies like html, css, and javascript. The web platform is more than 20 years old now, and it begins to show the signs of maturity — in some areas at least.

The web’s agenda, stated or unstated, has been to absorb all other publishing and presentation systems. It has, I think, done so, at the expense of elegance; web authoring is still a bit of a dog’s breakfast, simply because there are so many options, so many toolkits and frameworks, and whatnot. But just about anything is possible now. Good typography is within easy reach on the web now, as is native support for animation, vector graphics, video, and audio. Ubiquitous frameworks like jQuery provide a solid foundation of basic interactive functionality. And designers have begun to think mobile-first, a move that is important not just because we all carry phones, but because it provides a way of throwing off the “web design” conventions and aesthetic of the first decades.

So what should Digital Fiction look like in 2014? Especially given that the open web platform places very few constraints on designers. The lack of constraints is both a blessing and a curse; those early experiments found a ready-made genre and style in the narrow limits afforded by early platforms. In the absence of such things, how do begin to design for wide-open media?

 

 

Building Workflows for Authoring Digital Fiction

When Haig Armen and I set about building a technical infrastructure for our workshop (and for digital storytelling generally), we scoped out a presentation layer (focusing the reader’s experience) and an authoring layer (for the writer’s experience). We’ve happily been building upon Caleb Troughton’s excellent Deck.js framework, which provides an open, layered, web-native toolkit for building sequential stories. But the authoring environment has been a different story.

We thought that a good candidate for an authoring and content-management system would be the fabulously flexible Tiddlywiki, a “personal wiki” toolkit written entirely as a browser-based Javascript application, and very, very malleable. Malleable, yes, but all designs carry their own constraints, and over the past few weeks I’ve learned that the places we needed to go with this project don’t mesh well with Tiddlywiki’s assumptions. To put it briefly, Tiddlywiki attempts to provide an abstracted layer away from the usual building blocks of the web (HTML, CSS, Javascript), where what we wanted to do was to work with them directly. So I spent a couple of weeks learning where my assumptions conflicted with the assumptions built in to Tiddlywiki. Eventually, I got to a point where – despite my admiration for the way Tiddlywiki has been designed, and the flexibility it affords – I needed to move on.

I moved on – or rather back – to my document-production standby, Pandoc, a “swiss-army knife” for document conversion. Pandoc can convert just about any structured format into any other structured format. In our case, I need it to start with a simply formatted story, in plain text, with some annotations marking narrative structures, media, and events, and build that into the HTML+Javascript structures that Deck.js brings to life.

To do so, I built a custom HTML template that defines all the boilerplate CSS and Javascript needed by Deck.js (and its extensions). Next, I crafted some simple structural cues that allow sections to be defined, background media to be called, and on-screen formatting to be set up. The result is a very simple text file that an author creates, and a one-step build process to turn it into a rich media environment. That took only about an hour to craft, using Pandoc’s rich toolkit. What a difference compared with last week’s hacking!

My next steps are:

  1. to further tweak the Pandoc process so that writers can write less code (or at least write media directions with a minumum of formal syntax); and
  2. to embed the authoring and build process in a wiki or other web-based CMS so that we can have a collaborative writing experience rather than people working in isolation.

More to come! Stay tuned…

The Silence of the Ebooks

In the midst of an animated discussion with a grad student, weighed down with armfuls of books on media archaeology and criticism, I was utterly stumped by my inability to locate something I’d remembered reading last year: a fascinating chapter on John Cage’s 4’33 (of silence) and other interesting examples of blank media that still, somehow, have content. I struggled, I googled, I searched high and low on bookshelves, hard drives, gmails… all in vain. Eventually I turned to googling bare keywords in the hope that somebody, somewhere would refer to this text. Or maybe I dreamt it? Seemed plausible: a dream about a non-existent article about a non-existent song.

It turns out the truth wasn’t far off that. What I recalled reading was in fact a book – Craig Dworkin’s No Medium – which I read last summer, in one of the two pine muskoka chairs I had built and which sit in the shade of a katsura tree in my front yard. That part of the memory was crystal clear. How come I couldn’t remember what I’d read?

Eventually the details fell into place. I read it on the Kindle, which is of course why it didn’t turn up in any of my searches. And indeed, Craig Dworkin’s core argument, beautifully illustrated by literally hundreds of examples of blank, non-existent media, is that there is actually no such thing as a bare medium; rather, media are interpretive occasions, or at least eventful: their context is gathered, rather than given.

When I remember reading a book, I remember so much more than the text. I remember the physicality of the book, and where (and when) I was when I read it, and probably traces of whatever else was going on at the time. How the ebook messes with that memory! By erasing the physicality of a text, or at least by submerging it in a bland and anonymous container, the experience of reading – or rather, the experience of having read – is truncated.

If there is to be an embodied sense of a digital text, it must necessarily come from the text itself. I wonder if – putting ebooks aside – it would be possible to create an immersive digital storyworld rich enough to flesh out that part of the experience of reading, to provide the sense of a time and a place and a surrounding context. Is this perhaps easier with fiction than nonfiction? Because fiction provides a location, an immersive world to inhabit? And if so, how to provide that grounding in a nonfiction reading experience?

On Reading Digital Texts

“A BOOK [in the medium sense],” writes Charles Catton in a recent Medium piece on the so-called ‘enhanced’ ebook, “is a means of transporting the author’s words to the reader through the act of reading.” Whatever he might mean by “in the medium sense,” (see below) I think Catton has it backwards: a book is a means of transporting the reader into the author’s world. And I don’t mean “transporting” in some romantic, voodoo-ey sense of “I was utterly transported…” Rather, I think it’s high time we in the publishing world took stock of what it really means to read something.

Reading is not fundamentally about the delivery of information. It’s just not. Reading — be it escapist fiction or biomedical reference — is a profound experience of going inside somebody else’s mind. It is a transposition, a transubstantiation. It is virtual reality, as anyone who has ever read a novel can see, if they only stop to think about it. Even the “dipping in” kinds of reading we do when we read nonfiction and reference works is still this kind of experience; you internalize somebody else’s sentences and they unfold in your own head, (almost) as if it were your own experience.

The idea that books – or writing, or any other medium – is, as Catton’s subhead says, “a Means of Transport” has surely been thoroughly debunked by media theorists. Marshall McLuhan made himself famous forever by pointing this out, and the field of communications & media studies exists to explore the dynamics beyond this. Nor is writing a ‘delivery mechanism,’ as a generation of poststructuralist critics took pains to point out (Roland Barthes is my favourite here). Nor, even, is publishing a ‘delivery mechanism’ for content; read John Seely Brown & Paul Duguid’s seminal article “The Social Life of Documents” for a concise primer on how documents don’t just circulate in society; they bring society into being.

Reading is about experience, which is something that the people driving the ebook movement so far – publishers, retailers, and technologists alike –have so egregiously avoided, even while at the same time the discourse around User Experience Design is the ascendant on the open web. I’d go so far as to say that most of what’s “wrong” with the ebook world today is that things have been designed around the retail experience, the production experience, the gadget-sales experience, with very little attention to what reading means anymore.

And so we come to the ongoing debate about whether ebooks need to be “enhanced” or not. The pro side argues that ebooks are boring and that people would like extra stuff, like the additional features on a DVD, or at least links to the Wikipedia article. The con side counters that all that stuff disrupts and distracts from the immersive reading experience. As if that was a thing. As if the “sustained silent reading” we did in gradeschool were a course unto itself, like math or social studies.

As if we knew enough about reading to even begin to make a case like that. Well… we may not know much about reading, but we know what we like, right? For starters, we like what we already know. After all, what we know is of course the measure of everything we don’t know. And we can so easily imagine what we don’t like… the spectre of videos popping up just as we get to the good part is a good bugbear, worth repeating endlessly. We have all read a book, and so we are all qualified to sit back and armchair quarterback what might be the right thing to do for books.

Yet we don’t know much… other than what we’ve always known and liked. So, for instance, novels are 300 pages because they’ve always been 300 pages. Short stories are short enough so that 10 of them fit in a 300-page book. And even though people have begun to experiment with different-sized texts (e.g., ‘singles’, ‘shorts’, and the new ‘long form’) we still so underestimate the extent to which we conflate genre with format.

New formats are already here, all around us. The web-page article is a new format. Blogs are a new format. Wikipedia is a new format. Image-macro memes are a new format. But these things are also, and more importantly, new genres, which is something we don’t think about nearly as much. And these new genres are just the tip of the iceberg. Look around.

And yet we continue to discuss the book and the ebook as though these were God-given categories. Eoin Purcell, who wrote the post that inspired Catton to write his post, argues that the generic ebook is evidently “good enough” to capture 30% of the market, and considering the so-called ‘enhanced’ ebook, asks:

…whether anyone is looking for such innovations. The question those seeking to make more exciting and innovative products from books have to answer is straightforward; will those new products entice ebook readers away from ebooks, entice print readers from print books when ebooks didn’t do so, or entice new readers to read where ebooks and print books didn’t? —Eoin Purcell, How Different are Books Digitally?

As much as I like Eoin Purcell’s writing and perspectives, here he’s sadly invoking a zero-sum game and offering a narrowly functionalist assessment – not just of books, but of reading itself. We all already read vastly greater quantities of non-book, non-‘published’, non-traditional digital text, in a staggering array of both formats and genres. This is not in competition with the book-as-a-novel. Reading has already changed. The innovations are all around us. The better question at this point in time isn’t whether we want or need innovations, but which innovations we want. That’s our business now.

There is no pure, unmediated literary format. The traditional book is itself technology; it affords and enshrines a particular set of conventional reading experiences. It is historically and culturally situated; there’s nothing God-given or essential about it.

James Bridle today posted a thoughtful piece in Grafik, called Read Only, in which he writes,

Perhaps the line of the text is no longer a central metaphor we need to cling to. Perhaps it never should have been.

Bridle too weighs in on the so-called enhanced ebook, going to a place he’s been before, in which he distiguishes between the experience of the text itself from the experience of other kinds of media elements (as though they were alternatives), and in doing so praises apps like Instapaper and Readability for putting the emphasis on the literary experience alone. Bridle says, “the real value of these apps is how they remove all the crud of web design around a good piece of writing”

That’s all very well (and I live in Instapaper on a day-to-day basis), but it again papers over our cleaving to traditional text-only genres. It only takes a moment to recall the picture books we loved as kids, and the coffee-table books we shell out big bucks for still, to recognize that there are legitimate genres that mix text with, say, images. We don’t talk about photographs getting in the way of the text in a photo essay (or, for that matter, a photography book. Or for that matter, a listicle), because we collectively understand that a photo essay is a genre that is the result of the interplay of text and image.

But Bridle says something more sophisticated next, which belies his supposed adherence to the uninterrupted flow of words:

because text, however we may imagine it, is not linear, and it never has been. Just as the tiles of the slippy map are near-instantly rendered as we move over them, slotting into place before we are even aware of them, creating the illusion of a continuous plane seen through the narrow gap of the browser window, so the mind continually enacts understanding, rather than passively receiving it. That understanding is composed of many multi-layered understandings which go far beyond the text itself, into memory and association, into anticipation and extrapolation

That understanding, “composed of many multi-layered understandings”… that’s where the innovation in the reading experience is going to come from. As it always has before.  That’s where publishing needs to open up and look to writers, artists, and creators generally for some new perspective. That’s a major part of what this Creating Digital Fiction Workshop is interested to explore.

Of cards and decks

We’ve been thinking more about cards as the metaphor for the visual nodes of a story; everywhere the analogy gets reinforced, including today, when we had a good look at the excellent deck.js framework as a building block for our technology infrastructure. Deck.js was originally designed as an HTML5 slideshow/presentation tool, but it’s so nicely put together, modular, and well documented that we’ve begun adapting it as a basic platform for DF. A deck of slides or a deck of cards?

Similarly, we’ve been looking at the almost ineffably amazing TiddlyWiki5 as a content store and editorial management system; again, wiki’s origins were as a ‘deck of cards’ in software. TiddlyWiki5 is a stunning piece of recursive architecture, with the entire system built out of “tiddlers”–cards, that is –that hold content and/or the Javascript code that makes up the system itself. As Alan Kay would say, “it’s turtles all the way down.”

Beyond the technical details themselves, our goal is to put together a flexible system for assembling layers of media for digital fiction. We want to be able to support, on the fly, the creative directions set (or discovered) by our workshop participants, and to do so in an open, connective, and collaborative system that’s unconstrained by proprietary software or external limitations. That said, that such powerful toolkits are available on the open web (“Fork me on GitHub”) is a lovely thing.

Watch this space.

 

On Units of Meaning in DF

What’s the proper atomic unit of DF?

In last week’s post Towards a Technical Infrastructure, we worked from a model of an Episode, made up of a number of Scenes (a term with deep dramaturgical roots), where the majority of web-infrastructural things pertain—URL, content management hooks, media file associations, etc. But Scenes are made up of what? At this finer grained level, the more minute details of rhetoric, pacing, and media layers take hold; this is where a good deal of the “user experience” considerations are grounded. In our provisional model, we called these Shots, a term taken from cinema, but none of us were particularly happy with that term.

Kate suggested Clips as an alternative, a term which is similar enough to Shots, perhaps more of the era of YouTube than of 8mm film. As Kate pointed out, Clip “still has the weaponry association, but less so.” Haig noted that the CYOA platform Twinery.org uses the much more active term “Passages,” which evoeks both a section of text and a way through it.

We’re also aware of the resurgence of the term “cards” in the past year or two; from Twitter to Inkling, the card metaphor again finds its place, more than two decades after Apple’s unparalleled hypermedia toolkit, HyperCard. Indeed, the card metaphor has deep roots; Haig and I wrote a paper last year that looked into the substantial history and possible futures of cards and cardplay

Does Card make sense for the unit of immediate engagement in digital fiction? It certainly is more friendly and concrete in comparison with analytical language like “lexia” or “actemes,” or Aarseth’s “scriptons and textons”  And yet, a card is a static piece; the card metaphor denies any flux or dynamic play in the reader’s engagement. It is all about the node, and not about the link. By contrast, Clip, as Kate pointed out to me the other day, is also a verb; things can be clipped, and clipped together.

Sponsors and Scholarships

We’re excited to announce that we’ve added the University of Victoria’s Electronic Textual Cultures Lab, home of the Digital Humanities Summer Institute, as a sponsor and supporter of the Pathways workshop.

Between the ETCL and the Canadian Centre for Studies in Publishing (CCSP), we are pleased to be able to offer a limited number of scholarships to offset the tuition cost of the Pathways workshop.

As of May 15th, our scholarship applications are now closed. Registration for the workshop is still open

Towards a technical infrastructure

In planning for the Pathways workshop and the technical infrastructure required, we’ve been looking to a few historical precedents for guidance. We’ve taken stock of Kate Pullinger’s digital fiction projects, Inanimate Alice & Flight Paths, and also the original CBCRadio3 Digital Magazine circa 2002–2005, which Haig Armen was a part of.

All these publications were created in Flash—state of the art at the time—and have a good deal in common, architecturally. In particular, all three examples make use of the strategic layering imgtxtaudioof text, image (often animated), and audio to tell a story. All three play with the reader’s relative attention to these three modes; to what extent does the work direct or orchestrate the reader’s attention to one or other layer; and to what extent does it leave the reader free to attend as she likes.

 

We want first and foremost to create an HTML5-based environment capable of providing a platform like Flight Paths or a CBCR3 story, with that same weaving of media layers. To that end, Haig and I whiteboarded a provisional ‘object model’ for the environment, like so:

EPISODE, composed of one or more:

    - SCENE, which has the following:
        - straightforward URL
        - master audio track
        - and is composed of one or more

        - SHOT, which can have the following:
            - transitions in and out
            - triggers (for audio, for nav buttons to show, etc.)
            - maximum duration
            - text
            - image/animation
            - audio track
            - nav elements (buttons, game elements)

SHOTwbSo, a SHOT (of all the terminology we’ve used, I’m least happy with that one) has 3 visual layers: fg, mid, bg – each of which may be populated or not – as well as access to a SCENE-level background and audio for a 4th layer.

This chunking and layering of media elements is designed to allow a fair bit of flexibility in assembling orchestrated pieces. In many cases, not all the layers would be used; allowing a background image or soundtrack to simply flow across a sequence of shots.

The design of this will evolve over the next few weeks, and as we scaffold it into existence. So far, this isn’t terribly different than what HTML5 slideshow frameworks allow, but with more flexibility for bringing things in and out of focus.

Managing collaboration

The Pathways workshop is to be a collaborative experience; participants, supporting faculty, and creative director/digital dramaturge Kate Pullinger to conceive, author, and produce a work of networked fiction.

But how, in such an environment, are we to value and recognize intellectual contributions and intellectual property? If everyone involved is an “author,” then are we to be left with a tangle of competing IP claims for the resulting work?

Back in the 1980s, free software pioneer Richard Stallman famously confronted this problem, faced with managing the contributions of hundreds of contributors to the code for the Emacs text editor. Stallman’s solution to the problem was to turn copyright on its ear and create a ‘copyleft’ license, consigning the work itself to the commons, and putting the rights to reuse, rework, remix, and redistribute ahead of the right to control copying or use. The result was the GPL1.

That worked very well for software projects. But the same idea has been employed to good effect in Wikipedia—where an indeterminate number of contributors labour to make the entries there better and better over time. Rather than a tangle of IP claims from competing contributors, Wikipedia begins with a blanket CC-By-SA licence: essentially a ‘copyleft’ license like Stallman’s GPL.

But can this work in the world of fiction and creative writing? Examples are few and far between? Is a copyleft approach necessary to manage collaboration at scale, or are a number of networked private IP claims workable?


  1. The story of Stallman’s predicament and the solution that came out of it is retold nicely in Chris Kelty’s excellent book, Two Bits: The Cultural Significance of Free Software (Duke University Press, 2008)

CBC Radio3 Digital Magazine

CBC Radio3 was originally a magazine, long before it was a podcast or radio show. The project was an innovative Flash-based multimedia publication, published weekly in Vancouver. Pathways faculty member Haig Armen was a producer on the project.

According to its official archive (which features 105 issues!) at http://archive.cbcradio3.com/,

The CBC Radio 3 digital magazine ran from November 2002 until March 2005, garnering numerous accolades in Canada and abroad with its unique blend of music, journalism, literature and photography.

Over the course of its brief run, the magazine won over 20 awards, including one Art Directors Club award, two New York Festival Awards and three Communication Arts Awards, and was featured in several web and design books. In 2003, the groundbreaking website won three Webby Awards in a single year: Best Broadband Site, Best Radio Site and the People’s Voice Award for Best Broadband Site.

Digital fiction — what’s in a name?

When we were working on the initial descriptions of this workshop, we went back and forth over what to call it. Interactive fiction was used early on, but that term has come to be associated with the “twisty little passages” style of immersive text-game worlds — the Wikipedia article on IF gives some good detail. But as much as this is near and dear to my heart (I was a MUDder back in the day), it’s not what we’re doing this summer.

Kate Pullinger’s existing digital works—notably Inanimate Alice and Flight Paths—are ‘interactive,’ and ‘fiction,’ but they’re not immersive gameworlds in the sense of IF. They’re also multimedia fiction, though that buzzword has seen better days as well. So we’ve gone with “digital fiction,” which perhaps is ambiguous enough to provide a pretty big tent for us to work in.

Is there a better term?

UPDATE: just after I wrote this post, I discovered Kate’s 2006 piece in The Guardian, “Fact is, We need a better name for Digital Fiction

Flight Paths

Flight Paths: A Networked Novel was the second major digital fiction project from Kate Pullinger & Chris Joseph. Like Inanimate Alice, Flight Paths is episodic; Episode 1 was published online in 2007, and a sixth episode appeared in 2012. It orchestrates text, image, and audio in the delivery of the story of Yacub, a young Pakistani man who stows away in the landing gear of an airliner bound for London. He falls from the plane as it approaches Heathrow and crashes into Harriet’s car in a Sainsbury’s lot. Pullinger’s 2014 novel Landing Gear takes Yacub’s and Harriet’s stories—and their intersections—forward.

See http://flightpaths.net/

Inanimate Alice

Inanimate Alice, says Wikipedia, is “an interactive multimodal fiction,”  written and directed by Kate Pullinger and digital artist Chris Joseph. The first episode was released in 2005, and there are now four ‘official’ episodes on the Inanimate Alice website. The work has been extensively used in teaching digital literacy, inspiring students worldwide to create their own digital fiction and as such there are many more episodes out there.

The core story, which traces the experiences of Alice, who is an 8yr-old girl in Episode 1, as she moves with her parents to a variety of places around the world. The story is told via a combination of text, sound and image, with the reader controlling the pacing and direction of the story at various points.

A Guardian interview with Kate Pullinger in 2006 provides some context: http://www.theguardian.com/books/2006/dec/07/technology.internet

Announcing: Creating Digital Fiction with Kate Pullinger

A week-long digital fiction workshop from Publishing@SFU

MONDAY – FRIDAY, June 9-13, 2014
9:00am–6:00pm

Join us for this special week-long immersive writing/authoring workshop with award-winning author and digital fiction pioneer Kate Pullinger. This workshops is a unique learning opportunity aimed at writers who wish to explore digital fiction, developers who want to explore literary works, and publishers interested in new models for writing, reading, and collaborating in fiction.

Over the five days, participants will work together to collaboratively author a work of interactive, multimedia literature, which will subsequently be available online inviting further participation from a wider public.

Participants will work collaboratively with faculty to plan, compose, design, assemble, and promote the work over the course of the week. A series of short morning seminars with faculty will elaborate the dynamics, opportunities, and challenges of composing and producing for networked digital media. Afternoons will be devoted to collaborative work on writing, design and graphic production, audio and video production, and technical development. The goal for the week is the production of a prototype work which forms the basis for an ongoing, collaborative work which gathers an online audience.

FACULTY:

Kate Pullinger writes for both print and digital platforms.  Her new novel, Landing Gear, published in the spring of 2014,  takes the story told in Pullinger’s collaborative multimedia digital work, co-created with Chris Joseph, Flight Paths: A Networked Novel, and develops it further.  Her novel The Mistress of Nothing won the 2009 Governor General’s Literary Award for Fiction, one of Canada’s most prestigious literary prizes.  Her prize-winning digital fiction projects Inanimate Alice and Flight Paths: A Networked Novel have reached audiences around the world. Pullinger’s other books include A Little StrangerWeird Sister, The Last Time I Saw Jane, Where Does Kissing End?, which are all being published in new ebook editions in the spring of 2014.

John Maxwell is Associate Professor in the Publishing Program at SFU. His research & teaching focus is on the impact of digital technologies in the cultural sector (and particularly books and magazines), the history of digital media, and the emergence of digital genres and mythologies.

Haig Armen is one of Canada’s most respected and innovative digital designers. He is a faculty member in Design & Dynamic Media at Emily Carr University.

Ryan Nadel, with his company 8 Leaf Digital Productions, produces and designs digital media experiences in a broad range of sectors including education, graphic novels, and TV franchises, including the interactive companion to Art Spiegelman’s recent book MetaMaus and the transmedia campaign for the TV show Continuum.

Register here:

http://www.ccsp.sfu.ca/education/publishing-workshops/digital-media-workshops-2014/creating-digital-fiction-with-kate-pullinger/