1Approaching Media Architecture

In my contribution I want to approach media architecture – specifically media façades and urban screens – as urban interfaces. In particular, I want to do justice to the social (“inter”) and dynamic (“-ing”) aspect of interfacing: in time – “historical”, temporary, and processual – and place – located, positioning, communicating. When departing from the question of “access” – the question raised by this book series – inmy consideration of the specificities of media architecture, I wish to combine two perspectives. On the one hand, I consider the medium-specificity of architecture, and on the other, the architectural specificity of location-based and public forms of urban media. Hence, I take interface, or to be more precise, the verb interfacing as a concept that allows us to grasp the role of media architecture (and architectural media) in today’s information society – driven by the ambitions of shared access and of public participation: an open society of access for all. In particular, interfacing as a concept theorizes how “access” is always already an active form of exchange. Indeed, in the Age of Access, access is also a social issue, especially when taking place, literally, in urban public spaces. This literal, concrete “taking place” focuses the attention on the temporality as well as materiality of interfacing. The interface thus functions as a material object as well as a concept. We can call it an object-concept.

The interface as object-concept is connected to a set of other object-concepts that share overlapping traits. This overlap we find in comparison helps us to understand the specificity of the object we study. We can discern understandings of the interface as a material object via the metaphorical relationship with other objects, such as a membrane or skin, a surface or zone of simultaneous separation and contact.5Moreover, as the concept developed in light of new, digital technologies, we can see how the interface is often understood within frameworks of older, technologies of vision, most notably the screen. (Bolter/Grusin 2000; Manovich 2001) Indeed, as Shannon Mattern (2014) has more recently pointed out, the urban interface is most often imagined as a flat display or screen. It is not a coincidence but significant that these metaphorical objects – membrane, skin, mirror, or the architectural surface, window or aperture – are recurring terminology and central to our conception of both interface and screen. (Bolter/Gromala 2003; Friedberg 2006; Bruno 2014) In her work on media installation art, Kate Modloch phrases this double function of object and concept succinctly:

Screens themselves have the curious status of functioning simultaneously as immaterial thresholds onto another space and time and as solid material entities. The screen’s objecthood, however, is typically overlooked in daily life. (Modloch 2010: 4)

In other words, the quintessential and ubiquitous example for these overlapping and sometimes paradoxical qualities of the visual interface, the screen can be considered a material element of media architecture, and a metaphor, or metonym, for the larger and abstract interface quality of architecture, and a model for investigating the specific interactive spectatorial engagement with media architecture.

As such, I will develop my consideration of media architecture as material, communicative and performative architectural media, by examining the interfacing aspect of architecture via the screen as a model to think with. Hence, the term object-concept. Not only are urban screens, in their wide diversity, the most exemplary and visible instances of media architecture. Also, through the example of the screen and a theoretical approach to the screenic property of media architecture, I argue, we can analyze how media architecture as a wider category contributes to the social fabric of contemporary urban life.

But when we consider its status as between material object and concept, “screen” is too general a term. Traditionally conceived of as a framed surface on which to project selected and composed images, or from a realist perspective, a transparent window from which we can view the world, augmented digital screens come to us already replete with images. Moreover, in the course of watching them these images (can) continuously change and transform, under the influence of the actions we call interactions – whether human interactions, mediations between technologies or the processing of flows of data pushing and pulsing on our screens. In the following, I will explore the differentiated functions and manifestations of interfacing through analyzing a set of metaphors for, or meanings of the screen that foreground the object-concept status of the screen as interface: the screen as mirror, as interlocutor, and as surface for display. The interactive, digital screen requires a reconsideration of the formalist model of the picture frame, the realist model of the window, and the poststructuralist model of the mirror, as Kate Modloch summarizes (Modloch 2010, IV), that have been reigning for the cinematic and electronic, or televisual screen. In my book on screens and what I have called, the visual regime of navigation (Verhoeff, 2012a), I have analyzed the diversity of (digital) screens – whether fixed and architectural, or small and mobile – as a map, the screen as a (portable) gadget and the screen as skin, or site of touch, to investigate the mobile, haptic, interactive engagements afforded and invited by digital technologies and how this impacts screenic spectatorship. From this comparative approach and the question of the specificity of, what we may call urban spectatorship here I zoom in on some recent examples, each innovative, experimental and emphatically self-reflexive in its use of (media) technologies in their architectural design, and probe how they operate as mirrors, as interlocutors, as surfaces of display, and in this variety demonstrate how digital architectural screens operate as consoles for interfacing.

2Interface / Screen

But first, let’s return to the interface – the starting point of this inquiry in media architecture. We speak of interfaces easily, when pondering interactive and digital media technologies. Apioneer in thinking about the impact of digital technologies on, what he terms, our interface culture, Steven Johnson summarizes the interface as follows:

In its simplest sense, the word refers to software that shapes the interaction between user and computer. The interface serves as a kind of translator, mediating between the two parties, making one sensible to the other. In other words, the relationship governed by the interface is a semantic one, characterized by meaning and expression rather than physical force.” (Johnson 1997: 14)

This is close, if not identical to the essence of mediation. In her rich study on surfaces and the materiality of media, Giuliana Bruno reminds us about the etymological root of the word medium which

… refers to a condition of “betweenness” and a quality of “becoming” as a connective, pervasive, or enveloping substance. As an intertwining matter through which impressions are conveyed to the sense, a medium is a living environment of expression, transmission and storage. (Bruno, 2014: 4)

Following these ideas about the mediality of interfaces – whether as translation, expression, transmission or storage – and the materiality of media technologies – whether software, surfaces, substance or matter – I situate the term interface between abstract concept (interfacing) and material manifestation (interfaces), or object. “Interface” as a noun begets a remarkable concreteness in its use. Indicative of the ubiquity of the digital in our visual and material culture, we speak of mobile interfaces, architectural interfaces, digital interfaces, or haptic interfaces, etc. We speak in those terms when we indicate the material technologies that, in a stricter sense produce the interface between technology – whether digital, analogue or material – and subject.

This is perhaps where Alexander Galloway (2012) speaks of the interface effect – a set of processes rather than a (singular and fixed) object. His focus shifts the attention from media (fixed) objects to (on-going) practices of mediation:

Interfaces are not simply objects or boundary points. They are autonomous zones of activity. Interfaces are not things, but rather processes that effect a result of whatever kind. For this reason I will be speaking not so much about particular interface objects (screens, keyboards), but interface effects. And in speaking about them I will not be satisfied just to say an interface is defined in such and such a way, but to show how it exists that way for specific social and historical reasons. Interfaces themselves are effects, in that they bring about transformations in material states. (Galloway 2012: vii)

Here we can recognize an intersection of a spatial (“zones of activity”) and performative conception of the interface (“processes that effect”). Moreover, Galloway underscores the material and inherently social and historical nature of interfaces and of interfacing.

Branden Hookway (2014) also emphasizes that interfaces are inherently about interfacing. In his approach to interfacing as process, he stresses that interface is a form of relation. In his words, interfacing is essentially about the duality of relationality:

[…] the interface is that form of relation which is defined by the simultaneity and inseparability of its processes of separation and augmentation, of maintaining distinction while at the same time eliding it […] (Hookway 2014, 5)

This double logic of the interface – the distance implied in connection, and vice versa – is operative at the threshold of materiality or technology, which Hookway also describes in spatial terms:

The interface is a liminal or threshold condition that both delimits the space for a kind of inhabitation and opens up otherwise unavailable phenomena, conditions, situation, and territories for exploration, use, participation, and exploration. (Hookway 2014, 5)

While not conceived of as material object – the interface does “take place”: it has a spatial and temporal quality, which we recognize in Galloway’s words as well (“zones of activity”) above. Moreover,Hookway’s conception is architectural (“delimits the space for a kind of inhabitation”).

We can recognize the spatial thinking intersecting with the emphasis on (time-based) practice and process in the way in which Galloway points out both the essential layeredness of the interface:

While readily evident in things like screens and surfaces, the interface is ultimately something beyond the screen. It has only a superficial relationship to the surfaces of digital devices, those skins that beg to be touched. Rather, the interface is a general technique of mediation evident at all levels; indeed it facilitates the way of thinking that tends to pitch things in terms of “levels” or “layers” in the first place. […] Hence the interface is above all an allegorical device that will help us gain some perspective on culture in the age of information. (Galloway 2012, 54)

This layeredness implies both access and separation, and the self-referentiality of specific interfaces – here: architectural screens or surfaces – with respect to this layering. By means of allegory, Galloway then underscores the fundamental role these processes and practices of “interfacing” play in our culture. In this line, philosopher Jos de Mul also adds a fundamental aspect of interfacing that leads to a critical perspective from which to evaluate actual instances of interfaces:

Media are interfaces that mediate not only between us and our world (designation), but also between us and our fellow man (communication), and between us and ourselves (self-understanding). (De Mul 2009, 95)

From this viewpoint interfaces operate as means to communicate but also to self-reflect – on ourselves and our relation to the world around us. This points out the fundamental role of interfaces and processes of interfacing in the construction of the cultural fabric of our cities. Moreover, it suggests the way we navigate our cities, our world, via interfaces: by communicating with, and relating to our environment, we position ourselves in relation to the world, to others, and in this process we construct our conception of where and who we are.

This makes it not only relevant but also urgent to consider the architectural aspect of the interfacing use of screen. Architecture itself can be seen as interface – and interface as architecture, for that matter – including the possibility to offer a critical analysis of it:

As technology is about to take the next step and turn ubiquitous, the problems and prospects of computer interfaces will become relevant to the whole built environment. However, architecture has always had its human interface: building façades have communicated their function, their social prestige, their history, and their aesthetics. (Teräväinen 2014, 7)

I propose to follow this perspective on architecture as medium, rather than on media embedded within architecture. In the following, let us look at the way the architectural screen can work to reflect, to connect and to project.

3Reflections: The Screen as Mirror

Let me begin with the “face” of interface. I want to start with what is, perhaps, a rather unexpected example of media architecture-as-screen: a moving façade that we can consider as somewhat between a traditional screen and a moving, 3-dimensional, kinetic surface. To frame this example as a screen is already a conceptual move. So is the comparison to the screen that I aim to shed light on in its functioning as a mirror.

MegaFaces is a temporary façade designed by Asif Kahn and engineered by iArt, for the pavilion of the Russian telecom network MegaFon, set-up during the 2014 Winter Olympic and Paralympic Games in Sochi. This cross-platform work comprised a façade with 11.000 moving so-called actuators, each equipped with an LED light. Like the children’s pin point impression toy called a “pin art board” or “pinscreen” the “mirror” images are given 3D shapes on the façade. In small, individual 3D photo booths, set up on location and throughout the country, participants could upload their mirror images. Captured by camera, translated to code, and uploaded to a database comprised of thousands of faces, the self images were displayed, large-scale, via the façade/screen, creating a private-yet-public, and individual-yet-anonymous, delayed and (literally) pro-jected mirror image.

Indeed, this system demonstrates a directionality of the image as “information” that is perhaps reminiscent of, yet also somewhat exceeds the viewing model of the mirror. The mirror as interface already has two sides. It reflects but also transforms the self into an “other”. As the installation includes the capture and transmission of the mirror image – the reflection self becomes a projection of the selfie. Moreover, in his case, by its sheer size but also its public positioning in public space, it blows up the individual and private mirror image to the scale of a public monument.

Asif Kahn articulates his ambition with the work as to construct an “inclusive monument to people, regardless of their status as athletes or spectators, their age, nationality, sexuality or gender.”6This speaks to a participatory ideal of inclusion that is in tension with, even problematic in light of, the façade as large-scale monument. Inherently, due to its non-human scale and spectatorial arrangement or dispositif, in the process of making a monument, these “people” become a de-individualized part of a large data-set, that is not so much accessed as it is displayed. Moreover, the data translation and processing involved, and the delay in presentation works towards a distancing effect within the exchange between the subject and his or her image – not only in space but also in time. This delayed and extended public-ness reframes the mirror’s intimate self-portrait as a display of the self-as-other – out of reach of the subject to communicate with.

The interfacing quality of the mirror – as an exemplary spectatorial arrangement or dispositif of screen-based visuality – is the possibility to enact different versions of the self; to be in touch, so to speak, with the self as other. In this case, however, the allusion to the mirror also demonstrates the discrepancies that always-already exist in the mirror. Like a pseudo-mirror – similar but not quite the same as the model – this work can indeed be better conceived of as a mega-mirror. It demonstrates though its excess to what extent the mirror is not a reflection making a duplication but an estrangement of the self. In line with psychoanalyst Jacques Lacan’s view of the mirror, this work, then, puts a critical gloss to our common conception of the mirror as a tool for self-exploration, demonstrating that what we see in the mirror is our self image as other; as seen by others.

This gloss has particular relevance in the present cultural moment. Perhaps more than a monument for the people, the screen-façade of MegaFaces is a monument for the flawed communication of the mirror image as self-portrait in the selfie culture of today. As the designer points out, “The ‘Emoticons’, ‘Selfies’, ‘Facebook’, ‘FaceTime’ etc. have become universal tools for communicating and expressing emotion and affect, and the face persists as the prevalent shorthand in these new mediums”. This particular, but now, ironic monument to the new media of today uses the face of the mirror image as selfie to demonstrate some crucial paradoxes of today’s ideals and pitfalls of digital communication: to be disconnected in connection; to lose privacy and ownership of one’s self-image by ubiquitous forms of surveillance, yet have unprecedented access to, but little control over all kinds of data; and to have technologies and platforms for individual expression and visibility, yet be anonymous and de-individualized on the global scale of our public culture; and to lose agency in the midst of the proliferation of interactive technologies and participatory platforms.

4Connections: The Screen as Interlocutor

After considering the “face” of interface via the intimate individuality and frontality of the mirror in a shift to a public projection of self via the façade, the next case is an example that shows us the ambition of using architectural screens for dialogue, for exchange, foregrounding the “inter” of interface.

Connectivity, a hallmark of digital and networked culture, both sparks and is resulting from the ideal of communication, or the dialogic exchange of information. (De Vries 2012) While more defined in communicative terms than accessibility, as a more one-sided form of retrieval and a form of ownership, connectivity is not necessarily dialogic. Here, I wish to invoke the linguistic conception of the interlocutor – as a partner, co-present instance within a dialogic exchange. This position of the “you” that together with the “I” is included in the “we” is not so much already present as it is constructed at the interface, or via the screen, in the present (and presence) of the encounter.7

Recently I have discussed this principle of connections and encounter via interactive screens set up in public spaces. There, I discussed some instances of mobile media architecture – screens that are temporarily set up in public spaces, or temporary uses of more permanent screens. These screens, albeit in different ways, all worked for viewers in different cities to “connect” on screen, by waving, or by joining in virtual game spaces via avatars that are controlled by Kinect cameras (Verhoeff 2015). All these dispositifs or technological and spectatorial arrangements comprised a set of screens, cameras (webcam or Kinect), and internet connectivity. Whether individually controlled or larger, public set-ups, these instances constructed virtual bridges between the two locations. Inspired perhaps by earlier projects, such as the 2009–2013 project Large Screens and the Transnational Public Sphere that established a connection between large screens in Melbourne and Seoul (Papastergiadis et al. 2013), the Dutch company Dropstuff provided the case study of The Bridge – a traveling screen-based connection between different cities in Europe. On the screens of The Bridge, local publics can play different games by sharing game space on screen, or witness each other “on location” via video stream.

In my discussion of this project and some related ones that work with connected screens, I wanted to raise the question of connectivity – are people truly connecting in a dialogic exchange or rather witnessing the other when sharing screen space? Moreover, if they are in connection, what does this connection produce? What struck me, looking at the playful engagement with the games on screen, and the encounter of the spectator/participant/engager seems to allow, first and foremost, an encounter with the self on screen –whether in photographic likeness (via video stream), or in the form of a responding avatar in the case of video games. When looking at strangers looking back at us, and acknowledging our presence by waving, also, our presence within our direct environment is marked. Somewhat between mirror image and the image of another – the participants play with their presence in front of the screen. The bi-locality of the set-up – a two-way connection between locations – does not necessarily lead to a bi-directional, dialogic exchange. Or more precisely, this exchange, in its fleeting, playful form, is perhaps not so very different from the mirror of our first example in how the responsiveness of the screen invites markings of presence. The screen mediation this as a third space by triangulating these connections perhaps always absorbs connected locations.

A project that creatively exploits this space between the screen as mirror and the screen as interlocutor, between one- and two-directionality, is Occupy the Screen, developed in 2014 by Brighton-based Paul Sermon and Charlotte Gould. Curated as part of Connecting Cities, it makes use of existing, public urban screens that, as the artist himself states, people can “approach on their own terms”, without a scripted narrative or game design.8The technology of camera and chroma key technique are used for a collage of background – a colorful environment reminiscent of game aesthetics – a middle ground (audience in location A) and foreground (audience in location B). One can see people playfully engaging with the screen, within the space they occupy in front of the screen, in response to their representation and that of others, elsewhere, in the virtual space in front of them. The merging of spaces on the screen simultaneously expands and shrinks one’s direct urban surroundings, extending these in connection with elsewhere, but by merging this into a fictitious screenspace also cutting this off from continuity with the immediately connecting the urban space. After the mirror’s transformation of self into other, this is perhaps another paradox of interfacing – that in connection there is always also a loss, much as in expansion there is always also separation.

5Projections: Between Access and Display

Taking a comparative approach to the diversity that the category of media architecture comprises, perhaps the most eye-catching are those projects that work with the possibility of using existing building façades as large surfaces of display, As added screens or light-emitting installations embedded in or built on the skins of buildings, these surfaces operate by essentially visualizing all kinds of data, whether in the form of recognizable images or more abstract, colorful light effects. The following example of media architecture demonstrates a double functioning of media facades as sites for (interactive) access to, and surfaces of (spectacular) display of data.

Firstly, as our analytical object it is layered in the sense that it is both permanent and temporary. It is permanent, as far as buildings ever are permanent, because of the fact that it uses the pre-existing interactive LED façade of the Ars Electronica Center. At the same time, it is temporary in the way that it is one of several changing projects designed to make use of the façade. Secondly, on the thematic level, this project itself is in a sense also about the layering of urban space. Called Deep City, it is developed by Ursula Feuersinger, who articulates her ambition to visualize and bring to the surface hidden layers of data, about urban spaces – Linz, Vienna, Berlin and New York – their inhabitants, and re-courses – the city’s big data, if you will. These data sets were grouped in pairs (growth/diversity, green spaces/bike paths, water usage/waste and density/noise exposure). As explained on the Ars Electronica web page:

Just as a city’s history can be uncovered by an archeological dig, the collective information that defines its present and potential future can be represented as a digital cross-section, emerging from underneath its concrete, visible structures. The Ars Electronica Center façade will put these underground samples on display: Observers of the project transform into participants by physically extracting hidden artifacts from the deep, bringing them to the surface, and examining them. The resulting layers of visualized data emphasize various political, sociological, cultural, or even personal characteristics of an urban space, encouraging the inhabitants of that space to critically engage with their surroundings.9

Proposed here is the use of the building’s surfaces as site of access to data by means of visualization. The underlying premise is that visual technology can provide a form of (physical) contact with real-life experiences and “personal characteristics” of urban space, as well as the urban archive of collective information.

Indeed, the layered and hybrid interface of the work is sophisticated. The arrangement comprises a separate-but-connected terminal, or console, by means of which the participants can browse and select data sets and have some input in what is shown on the building’s façade. It invites distant observers (Crary 1990) to become more attentive (Crary 1999) and even active engagers – “participants” in the quote here – by playfully browsing and combining data sets from different cities and composing a colorful show of lights on the building’s façade. In the very act of making visible by the processing data into information, generating knowledge, and translating this into colorful lights, however, “data” becomes a spectacle – for both observers and participants. This perhaps makes the otherwise distracted observer a more attentive spectator.

In his analysis of Lozanno Hemmer’s urban installation, Body Movies, Scott Mc-Quire (2008) describes a similar shift, which he registers in the public present at the night-time event. He interprets the attention focused onto the building in its illuminated state as comparable to our usual, everyday distracted relation to architecture. This is result of the openness, or “incomprehensibility”, of the interface, that invites engaged and playful experimentation of the participants:

Here it is worth recalling Benjamin’s argument that the radical impact of cinema in the context of the modern city depended – like architecture – on the fact that it was consumed in a ‘distracted’ state. Since the film image acted at the margins of conscious perception, it was able to circumvent the habitual defence shield each city dweller erected so as to protect themselves from the excessive sensory demands of urban life. Body Movies occupies a similar liminal terrain. Passers-by aren’t sure what to make of it; the interface is striking but not immediately comprehensible. Habit is suspended in favour of experimentation. Unexpected conjunctions emerge. (McQuire 2008, 153–154)

McQuire values the affective experiences brought about by these installations/events that function as tactical urban interventions by setting-up, designing “unexpected”, interactive encounters in public space.

The unexpected and experimental nature of these works – temporary by necessity – may invite playful interactions, however, its visual spectacle also positions the public in the position of a more distant spectatorship. Access in the form of visual display, in this way, both reveals and problematizes the possibilities for, and limitations of open access to big data for exploration, examination and analysis. In this sense, the work makes a good case for the 2015 program of Visible Cities, curated by the Connecting Cities Network for which Deep Cities was developed in a Research Residency. The questions central in the Visible Cities theme advocate a role of façades and screens as “black boards” and “visualization zones” for information:

How can we make social, environmental and intercultural processes visible and use the screens as black boards and visualization zones? What is the impact on the society, when invisible structures that underlie our daily life get visualised? What is the potential to create public awareness?10

However, the impact of visualization – its participatory potential to create awareness or lead to action, or even insight – is not straightforward. At closer inspection we can see how, as experimental project, Deep City questions the impact of visibility by addressing what it also exemplifies. Indeed, what is put on display is perhaps visible but not always legible, and hence actionable.11The algorithms that order, select and process the social and personal data below the surface translate this data as input for its “visualization zone” at the surface. This process of translation is not necessarily aimed at comprehensibility. Perhaps more fundamentally, this façade as screen demonstrates the differences between input, output, observation and interpretation. As such, Deep City suggests not only the many layers of information that are hidden in cities, but also the many layers behind and beyond the surfaces of the interface.

6Architectural Consoles

Like other projects developed for interactive displays and façades that have the ambition to allow for more democratic and dynamic access to what “lies beneath” or behind the screens that surround us in the city, Deep City aims to provide an interactive point of access – however visual and spectacular in form – to information. Rater than functioning as a flat surface of display, the façade as interface – extended with a small screen and a set of controllers – is an ensemble of screens and controllers that really functions as a console for different forms of input and output. We have explored how media architecture as a bracket for different architectural interfaces, via the model of the screen as point of connection and exchange, as site of display for data, brings to the fore the diverse forms and functions that we can recognize in how building façades work to mediate. As Hank Haeusler puts it in his contribution to this collection:

When equipping building skins with screens and digital technologies one can argue that a building equals an autonomous system, able to sense and collect data, to process these data into information and lastly to communicate these information to other buildings or humans to generate new knowledge about the building itself or the urban context (Haeusler, this volume).

Digital and augmented, then, we can compare these interfacing “skins” of buildings with consoles. An object-concept in its own right, a console, as I have argued in relation to the digital, mobile screen, comprises multiple interfaces, or “skins” if you will. (Verhoeff 2012b) Like a game console that we use for very different games, making use of the technological affordances of the console in various ways, screens are also, by definition, objects that have a range of different uses. I have defined the digital screen as “a material site for interfacing, the screen can be multiplied by combining different interfaces.” (Verhoeff 2012b, 292) Rather than an interface as singular object, the digital screen is a console for multiple interfacing objects: “Theoretically, it encourages the exploration of its possibilities as console, a polymorphous ‘screenic’ platform for a variety of applications and practices.” (294) This notion of theoretical consoles puts a specific spin on my proposal to develop our thinking via the relations between object-concepts. Immediately appropriate for media architecture, the notion of “console” also opens up the status of the singular “object” itself. It demonstrates the versatility and multiplicity of interfaces as technological objects. Moreover, it underscores how, rather than focusing on the specificity of their singular use, it is in the cultural practices of interfacing – what we do with screens – that we get to their specificity. We can consider the console as the “extension” or “access-point” to the facade, allowing us to give input for their display – even if limited by algorithms and code.

To return to our point of departure, we can conclude that by exploring a comparative perspective on the interfacing potentials of media architecture as architectural screens – or perhaps, architectural consoles – we can find the diverse and often paradoxical specificities of media architecture precisely within the multiplicity of interfaces and interfacings that we can encounter on our streets.