The nuts and bolts of ASR: capacities and constraints

Automatic speech recognition, or speech-to-text (STT), refers to the technology that enables computers to recognize and convert analog human speech into digital text.¹ This conversion process consists of roughly two stages, each marked by a specific software model. The first stage uses an acoustic model to translate the analog sound waves created by a person’s articulators (lips, tongue, epiglottis, and glottis) into digital data. During this stage, the vibrating air molecules that form the sonic waves of a speech signal are captured by a microphone and converted into a digital signal consisting of ones and zeroes. Software algorithms transform this digital input into a graph showing the signal’s amplitude (dB) over time. With the help of a Fast Fourier Transform algorithm, this information is used to create a more precise sound spectrogram, which visually represents the signal’s frequency (kHz) and intensity (dB) for overlapping acoustic frames of 20–40 milliseconds. Since the 1980s, Hidden Markov Model (HMM) algorithms have been used to compare these frames to a dataset comprised of the acoustic features of phonemes. Phonemes are the smallest distinct sound units that make up a language and enable phonetic differentiation between words. As the sonic building blocks of a language, each phoneme consists of an acoustic signal with a specific frequency, intensity, and duration. HMMs, nowadays often combined with deep neural networks (DNNs), help indicate which acoustic frames correspond to what series of phonemes through statistical probability, thus converting the digital speech vectors into the most likely phoneme sequence.

In the second stage of ASR, a pronunciation model works to relate sets of phonemes to words. In this process, the acoustic features of phoneme sequences are compared to a pre-programmed phonetic dictionary constructed by linguists. ASR models created in the past two decades generally integrate this model into a statistical language model, which is used to determine the probability of word combinations through data-trained algorithms (also known as natural language processing (NLP)). Drawing on vast bodies of text, these algorithms learn to predict the likelihood of particular word pairings and patterns. While older ASR systems without NLP models can transform a speech signal into the most probable word sequence with considerable accuracy, NLP has further boosted its performance, especially when it comes to ASR’s ability to disambiguate between similar-sounding words. At the end of this stage, the output of these models appears on a computer screen within a digital interface that enables users to read and edit the text.

These complex technological models provide ASR with unique affordances as a writing technology compared to earlier writing practices. In his article, Powers compares his method to earlier dictation-based practices of authors like Henry James, James Joyce, and Fyodor Dostoevsky. These authors narrated their stories to typists, allowing them to compose stories faster and more fluently. ASR, however, eradicated the need for such a human intermediary, enabling Powers to write even more rapidly with impressive accuracy. As Powers (2007a) observes: ‘Whole phrases die and revive, as quickly as I could have hit backspace […] It won’t choke, even at bursts over 200 w.p.m.’. Released from pacing constraints, he ‘can forget the machine is even there’ and become fully immersed in his writing (Powers 2007a). He points out: ‘I don’t have to queue, stop, batch dispatch and queue up again. I spend less mental overhead on orthography and finger mechanics and more on hearing my characters speak themselves into existence’ (Powers 2007a; see also Gross 2006). This immersion is further facilitated by his Motion LE1600 tablet’s microphone array that allows him to move around, hide under the sheets, and turn off the lights, creating a sensory deprivation where he only experiences his own voice (Perell 2024: 1:17:14–1:18:40). By eliminating the need for any physical act besides enunciation in the writing process, the software affords Powers ‘the freedom to be completely disembodied when […] writing’ and reach what he considers a ‘pure compositional state’ (Michod 2007). The author can lose himself in his story, freed from some of the obvious constraints that faced his literary predecessors.

However, ASR’s greatest merit, according to Powers (2007a), lies in its foregrounding of language’s musicality—its primal dimension of meaning—that has been largely lost in an era ‘retooled for printed silence’. In an interview with Terry Gross (2006), Powers argues that people tend to subvocalize while reading because ‘much of the meaning of [a] passage comes about through the sound’. He considers subvocalization to be an evolutionary remnant of the times when storytelling was a primarily oral activity, one that shows ‘we still feel residual meaning in the wake of how things sound’ (Gross 2006). ASR technology relies heavily on this sonic dimension of language, as the software is programmed to understand the speech signal as an acoustic signal first and secondly as a lexical utterance. Powers only has to speak his words out loud rather than transform his utterances into letters on a page, making him more attuned to the acoustics of his story as opposed to its lexical form. In his own words, using ASR technology ‘allows [him] to think in terms of the music of the prose […] [to] hear the rhythm and meaning of the passage [he’s] working on’ (Gross 2006). Doing so enables him to ‘shape a sentence so that it doesn’t just describe but actually participates in the emotion under description’ (John Hope 2016). Additionally, through ASR, he can experience his own ‘sentences and paragraphs as oral phenomena’, like his subvocalizing readers, attuning himself to a dimension of meaning hidden in their sonority rather than their denotation (Gross 2006). In this compositional state, his words seem to flow directly from his mind to the screen, almost magically, creating the illusion that there are no barriers between the story in his head and the one on the (digital) page (Michod 2007). Consequently, Powers (2007a) states, ASR technology invests his text with a never-before-seen sense of fluency, immediacy, and authenticity compared to earlier writing technologies. I will return to this point later in this discussion.

Although ASR offers unique affordances, the technology also presents unique challenges—the most apparent being accuracy. Even though, as Powers (2007a) points out, the software performs with great precision even at a high speech rate, its ability to make phonetic distinctions remains inferior to that of a human being. Humans instantly interpret speech within its specific context, whereas ASR does not understand the utterance as such but merely translates it by comparing its acoustic features to pre-programmed data sets. As a result, Powers (2007a) explains, these devices abound in ‘speakos and mondegreens’ that would never have occurred with a typist. In his case, these appear even more frequently as the Motion LE1600 lacks a natural language processing model trained to model meaning based on textual probability and context (Powers 2007b). Because of this deficiency, ‘[his] tablet can turn [his] words hallucinatory without limit’ (Powers 2007b). The end of his article illustrates that, as it includes two mishearings of his closing words: ‘[…] something new is always found again, in their eager years. In Derrida’s fears. Make that: in the reader’s ears’ (Powers 2007a). When mistakes like this occur, Powers can tap on a word or phrase to respeak it (Powers 2007b). Whenever a word is not included in the software’s pre-programmed dictionary, he can add it by handwriting with a stylus, navigating the on-screen keyboard, or spelling it out loud (Powers 2007b). Still, such inaccuracies inform the writing process to such a degree that Powers felt compelled to playfully end one of his online discussions of ASR technology with the phrase “Yours in speech and speakos” - a phrase that proved to be a fitting title for this article (Powers 2007b).

While these problems appear to be relatively minor hiccups in practice, they illustrate how ASR steers—and at times even requires—users to engage with it within certain input parameters to ensure a successful man-machine collaboration. For example, while Powers is free to wander around his room, the device’s accuracy decreases rapidly when he speaks outside or in a noisy environment. As Powers (2007a) states: ‘I’ve tried dictating to my tablet while rambling; traffic and birdsong make it babble’. Additionally, in order for the microphones to pick up his speech signal, he cannot move too far away from his tablet or speak indistinctly. The Motion LE1600 even required him to go through a whole program to ‘train the system for [his] voice, speaking pace, and inflection’ and ‘define the directional angle that is to be used for speech recording’ before use (Motion 2005: 56–57). Powers (2007a) observes how this set-up called for a ‘huge cognitive readjustment’ as he ‘needed weeks to get over the oddness of auditioning [him]self in an empty room, to trust the flow of speech, to learn to hear [him]self think all over again’. In a way, one could say that while Powers was training the technology, so was the technology training him; through interacting with the device, Powers learned how to engage with the technology productively, modifying his input behavior in a cybernetic feedback loop.

Aside from matters related to position and environment, ASR’s programming also places constraints on the contents of the speech signal. Software models, after all, are not neutral but ‘programmed and engineered by [hidden] human actors’ who invest the software with biases and a specific algorithmic agency (Klinger and Svensson 2018: 4654). As the ‘author-initiator[s]’ of these systems, these programmers decide their automated workings and limits, compelling users to interact with them on specific (if oftentimes hidden) terms (Hopfinger 2020: 171). Firstly, a user can only successfully use ASR when speaking in a language that has been pre-installed on the device for speech recognition purposes. Using foreign vocabulary, slang, or newly invented words is discouraged, as such terms must be manually added to the device’s pre-programmed dictionary. While adding a few words here and there presents only a minor interruption, making numerous corrections this way would greatly hinder the writing process. Consequently, the software is unsuitable as a writing technology for stories written in a dialect or minor language, multilingual literary works, or narratives including many neologisms. ASR programs based on NLP models are also ill-suited for authors writing experimental stories in which conventional modes of storytelling are upended and any logical connection between words and sentences is overturned. In these instances, the writing challenges the program’s probability metrics, likely increasing its percentage of inaccuracies. Additionally, the software can prove problematic for authors writing in a secondary language, as accents can lead the technology to consistently misinterpret certain phonemes and words. Thus, even before Powers can attempt to write a story through ASR, both he and the story must meet these implied criteria in order to benefit from the technology’s affordances. Put differently, even before a single word leaves his lips, Powers is programmed by the ASR technology to write within certain limits pertaining to speaking environment and volume as well as language, register, and style. Consequently, rather than functioning as a ‘digital prosthetic’ where the author has full control over the apparatus, the ASR technology creates a complex interdependence between author and device where the device influences the author as much as the other way around (Powers 2007a). As Maryla Hopfinger (2020: 55) points out in her study of digitally mediated literature: ‘[a] machine dictates its conditions, enforces the behaviour it determines, and makes humans dependent. While a brush, a pen, or a flute were tools controlled by a man who was able to subordinate them to himself/herself, all mechanical or electronic technologies gain power over people’. Thus, the technology determines what can be dictated by its user and how; the user can exercise control over the machine only within the specific action parameters set by the machine itself. From this angle, the technology seems hardly as ‘forgettable’ as Powers (2007a) suggests.

Immediacy, transparency, and mediation

Further examining Powers’s claim that ASR restores a sense of unmediated immediacy and acoustic meaningfulness to writing through this lens reveals a far more complicated situation. In his article, Powers (2007a) implies that using ASR technology allows him to write more authentically than previous generations of authors, as his spoken narrative is transformed into digital text in real-time without any physical interaction with (hand)writing tools or interference from human intermediaries. The apparent seamless nature of this process gives him the impression that the words on his tablet’s screen are a rather direct and faithful representation of the story he composes within his mind. While Powers (2007a) briefly acknowledges the fact that there will always be a disconnect ‘between the story in the mind and what hits the page’ and that ‘no interface will ever be clean or invisible enough’ to achieve otherwise, his discussion of the technology’s affordances suggests to the reader that ASR is, in fact, a step in that direction. What Powers disregards here, however, is that this modern writing technology is founded on and operates across many layers of (re)mediation that radically transform his words as they move from mind to mouth to page.

As Jay David Bolter and Richard Grusin (2000: 46, 48) have pointed out in their study of mediation, modern-day computers ‘[promise] the user an unmediated experience’ by facilitating a smooth and instinctual user experience that makes the user forget the device is even there. In this strive for ‘transparency’, the technology tries to ‘erase itself’ and instill in the user ‘a feeling that the medium has disappeared and the objects are present to him, a feeling that his experience is authentic’ (Bolter and Grusin 2000: 70). From a technical point of view, this sense of ‘transparency’ relies on the computer’s hidden ‘layers of programming [that] always return control to the user’, creating an illusion of autonomy despite the software’s system of ‘automated action[s]’ (Bolter and Grusin 2000: 33). ASR is a great example of such an aspiring transparent digital medium, as the technology relies heavily on input from users, who remain largely unaware of the automated conversion processes initiated by their actions. Still, this sense of transparency does not mean that the transcribed text is therefore a(n) (more) unmediated form of expression, as Powers implies. In the first stage of ASR, for example, the expression transforms consecutively from a series of vibrating air molecules into a resonating diaphragm, then into an electrical current and eventually into a digital signal. Afterwards, this signal is broken into segments, converted into phonemes, and then combined and recombined into different words, sentences, and paragraphs using various algorithms. Finally, the digital output appears on the screen as text, which in itself is a remediation of the writing technology of pen and paper (Bolter and Grusin 2000: 45). The hardware and software thus rework Powers’s supposedly authentic speech signal dozens of times, both in analog and digital ways, even before it takes the form of a transcription. Thus, the text on the screen is a highly mediated product of a network of remediation in which each ‘act of mediation is dependent on another, indeed many other, acts of mediation’ (Bolter and Grusin 2000: 56).

Moreover, even before his words undergo this process of technological remediation, Powers forms an integral part of this network. After all, the process begins with the remediation of his not-yet-fully-articulated thoughts, feelings, and memories into coherent language. As Bolter and Grusin (2000: 57) have argued, language is not ‘a neutral, invisible conveyor of fully present meaning’ but ‘an active and visible mediator that fills up the space between signifying subjects and nature’—i.e. a man-made system of phonologies, vocabularies, and morphosyntaxes that functions as a technology in its own right (see also Mufwene 2013; Hopfinger 2020: 166–167). Powers relies on these structures and their rules to construct the story in his mind. Therefore, even before it becomes a speech signal, Powers’s story is inherently shaped by the medium of language. Additionally, as Powers expresses his story, it is mediated by his body, specifically by the articulators discussed earlier. Here, a brain-muscle connection converts the story from silent thoughts into speech produced by different parts of the mouth, lips, and throat. Bruno Latour (1998) remarks on these interlocking ‘machines’ in a conversation with Powers: ‘Even when we are alone, in the flesh, we speak through all manner of machines: phonation, vocal chords, Broca’s area, English (for me a foreign artificial tongue, a huge distributed computer, for you a mother tongue a womblike web)’ (4). While I will not delve deeper into the mechanics of speaking, considering the scope of this article, these aspects are important to mention as they show how Powers himself functions as a mediating machine operating within a complex system of mediating technologies. Furthermore, it demonstrates that Powers’s presentation of the technology as a more authentic writing technology is based more on the device’s illusory transparency than reality. The advanced technology of ASR, in fact, adds even more layers of mediation to expression than earlier writing technologies, engaging its users in a dynamic media network where the intersecting affordances and limitations of language, the body, and technology dictate what can be written and how.

Listening to Orfeo

The analogy between writer and musician is central to the construction of Orfeo, the story of an ambitious composer named Peter Els who, like a modern-day Orpheus, devotes his life to creating radically innovative music. In the following section, I will demonstrate how Powers uses Els’s pursuit of authentic musical expression to thematize the relationship between artist and (digital) media—one that closely parallels his own. Orfeo, as I will demonstrate, maps the complex networks of remediation that shape Els’s work, ultimately revealing that direct, transparent artistic expression is merely an illusion. As such, Orfeo serves as the literary counterpart to Powers’s The New York Times article, one that foregrounds the layers of (re)mediation between artist and (digital) technology that remain largely unexamined in his journalistic account.

In Orfeo, Powers presents Peter Els’s lifelong struggle with the medium of music in a way that resembles his own engagement with language and ASR. From a young age, Els is fascinated by sound, having grown up in a musical household with a piano-playing mother and music-obsessed father (14–17; Vorda 2013). As an eight-year-old boy, he starts playing the clarinet, and by age twelve, he spends his evenings immersed in his father’s extensive music collection, familiarizing himself with the classical compositions of Bach, Mozart, and Beethoven (16–17; Vorda 2013). His artistic ambitions take shape when he meets Clara, his first love, who introduces him to the great musical experimentalists of the twentieth century (31–32). Inspired to become a composer himself, Els studies music in college, focusing on post-war avant-garde composers like Olivier Messiaen, Dmitri Shostakovich, and John Cage. He becomes obsessed with revolutionizing the medium and pushing the boundaries of what music can be. As the narrator of Orfeo observes, Els ‘staked his life on finding that larger thing. Something magnificent and enduring [that] hid under music’s exhausted surface’ (10–11). This artistic quest, however, ultimately proves unsuccessful, leading him to turn to DNA as a medium for musical experimentation. DNA allows him to create the masterpieces he always dreamed of, but at great cost to himself and his art. Els’s life story and love for music parallel Powers’s own in many ways (see Vorda 2013; McCracken 2014; John Hope 2016). However, more importantly, Els’s struggle with his medium reflects Powers’s own evolving relationship with ASR: Powers also was driven by a similar hope of uncovering new creative freedoms in digital technology, leading him to gain a deeper understanding of these systems’ built-in biases, affordances, and limitations. Indeed, many of Powers’s observations regarding the role of (digital) media in literary expression discussed previously can be seen reflected in Orfeo, transposed to the context of musical composition.

The mechanics of music

The connection between literature and music is made explicit throughout the narrative, as Powers draws implicit and explicit parallels between his own work as a writer and Els’s musical compositions.² Music is repeatedly described in Orfeo as a ‘language’ (100) that—though often wordless—communicates with an audience much like a novel through ‘musical idiom[s] […] [and] [a] harmonic vocabulary’ (292). Over the course of his life, Els comes to believe that music has turned into an overdetermined language and has become exhausted as an artistic medium. He laments that music has lost its ability to serve as a ‘direct transcription of inner states’, feeling it has become ‘bogged down by meaning’ (329). The reason for that, Els observes, is the body, which is programmed to assign meaning to acoustic signals (329–330). Rather than experiencing music as mere sound, the body instinctively transforms music into a story: ‘The body had evolved to feel fear, hope, thrill, and peace in the presence of certain semi-ordered vibrations’ (330). Powers explains this further in an interview with Keenan McCracken (2014), where he describes himself as ‘the kind of listener who experiences music, if not as a language, then at least as an information-rich linguistic phenomenon’. Interweaving music and literature in his own writing, he states, feels natural to him as ‘[his] love of harmonic expectation and progression in music is not unlike [his] love of cadence and register in prose’ (McCracken 2014; see also Perell 2024: 44:38–44:43). Over centuries, musical traditions reinforced this tendency to connect music with linguistic meaning, embedding emotions and even storylines into particular combinations of notes, harmonies, and melodies. Thus, music gradually evolved from meaningless sounds into its own semiotic system.

While twentieth-century avant-garde composers sought to break with these conventions, Els discovers that even radical musical experimentation eventually solidifies into familiar patterns (326). This realization is echoed by Powers, who observes: ‘the problem with the mind’s desire to understand and embrace is that we ultimately kill the joy with overfamiliarity’ (McCracken 2014). Els saw himself as standing at the ‘moment of mash-up’ following ‘[t]en short centuries [that] had burned through all available innovations, each more fleeting than the last’ (30). As part of this new generation tasked with broaching new musical terrain, he dedicates his life to searching for ‘[f]resh, surprising music that escaped all human conventions’ (73); a type of artistic expression that relies on pure sound that would transcend the body’s built-in interpretational structures (see also Hume 2017). Els shares this goal with Powers, who likewise turned to sound, convinced that the medium would convey his internal state transparently. Yet, both author and protagonist initially neglect to explore how their media rely on a network of mediating technologies—both analog and digital—that fundamentally shape the act of artistic creation and their art. In composing his musical works, Els is no freer in bending music to his will than Powers is in crafting his narrative with ASR software. In both cases, the dream of unmediated expression is ultimately confronted by the reality of media constraints.

Indeed, Els soon discovers that music, far from being a transparent medium, is itself a technology, a structured system that both enables and restricts artistic creation. Music, after all, like language, is an artificial construct: it consists of ‘[t]welve chromatic pitches’ (354), which, comparable to phonemes, serve as its fundamental building blocks. It also relies on notation, a system that, while preserving musical ideas, simultaneously bogs down pure sound, much like written language. As Els puts it: ‘[t]hat was the curse of literacy: Once you started writing music down, the game was half over’ (29). The twelve chromatic pitches form the musician’s ‘toolkit’ (100), making up ‘the palette for everything from sultry seduction to funeral mass’ (82) as if they were the ‘meshed gears’ of a ‘magnificent timepiece’ (18). While these structures inherently inform music, shaping its existence, Els begins to consider them ‘twelve repeating black and white prison bars’ (368) that form a ‘straitjacket’ (81) and ‘harmonic jail’ (20) he cannot escape: no music exists outside of these twelve pitches and no music can ever be written that transcends them. Any musical composition will always necessarily imitate others, each being just one more variation on the same set of pitches. As Els points out: ‘A hundred thousand years of theme and variations, every composer stealing from every other’ (332). Linking this back to ASR, dictation software, at its core, is built on this principle of imitation: it is programmed to understand any utterance as a variation on the same set of phonemes, which form the foundation of the machine’s speech recognition model. As indicated before, the software even encourages its users to express themselves in a conventional or derivative manner to maintain a high level of accuracy. ASR’s programming thus reveals language to be a phonemic prison much like music. Even though variation in literature and music seems endless in practice, the finitude of these media themselves makes the number of possible variations theoretically finite as well. Consequently, as Els comes to realize, innovation is only possible within the limits set by the medium itself.

Furthermore, Els gradually recognizes that music does not exist in isolation but within an inescapable network of remediation in which different analog and digital technologies mediate the artistic process. All of these intersecting affordances and limitations dictate what Els can or cannot compose. As A. Elisabeth Reichel (2017: 81) has observed, music has become ‘encode[d] by means of historically contingent, socially and culturally constructed rules’, many of which depended on its supporting ‘materials and technologies’. For centuries, composers were restricted by the musical abilities of singers, instruments, and musicians, but over time, new tools redefined the contours of musical possibility. Musical production, for example, evolved radically over the past century with the advent of various recording and audio devices. Orfeo frequently foregrounds these technologies, referencing LPs, speakers, headphones, and MP3 players (e.g. 6, 32, 40, 72, 74, 86, 89).³ In the digital age, however, Els can wholly transcend many of the limitations imposed on his predecessors with the help of ‘a computer [programmed] to generate a string quintet using probability functions and Markov chains’—the same algorithmic principles that underlie ASR (72). He describes how ‘[t]he computer made it possible to shape any pitch, amplitude, timbre, and duration’, allowing musical fragments to be ‘combined and recombined, slowed, sped up, inverted, reversed, stacked into evolving rhythms and incanted in banks of antiphony’ (97). He witnesses how, over time, these digital tools transformed into Sibelius, a software model sophisticated enough to ‘[turn] an average tunesmith into Orpheus’ with ‘cut-and-paste harmonies, point-and-click tone painting, one-button transposition […] [that] [transform] a handful of raw building blocks [into] a new two-minute stunning tutti’ (319). The model eliminates many of the practical difficulties and extensive labor involved in composing and creating scores through analog means. Yet Els finds this freedom unsatisfying. He considers the software’s affordances in terms of speed, efficiency, and convenience to be incommensurate with the loss of traditional tools and the artist’s involvement in the composition process, leading him to ‘[yearn] for the clumsy, freighted flights of earthly instruments’ (97). Sibelius much resembles a text-generating machine as it combines and reworks pre-programmed musical phrases with minimal input from the artist, obscuring the boundary between machine-made and man-made. Els’s stance on Sibelius thus raises complex questions regarding the role of the artist in semi-automated digital creation processes that also pertain to ASR. For example, at what point does the artist lose (primary) authorship over his/her work? And is art’s worth dependent on the lingering presence of the artist’s hand and the limits of his/her abilities and materials?

Switching media: biological melodies

While the novel does not attempt to formulate an answer to these questions, it does provide a compelling thought experiment in Els’s use of DNA for musical composition. From early on in his life, Els considered music’s ‘pattern language’ (30) to be connected to other systems for representing the world, amongst which the mathematical language of chemistry: ‘To [him], music and chemistry were each other’s long-lost twins […] The structures of long polymers reminded him of Webern variations. The outlandish probability fields of atomic orbitals […] felt like the units of an avant-garde notation’ (57). Powers echoed this observation, arguing that ‘[b]oth science and art consist of endless propagating experiments and speculations of what is and what might be’; two ‘languages’ that merely use different conversion keys to map the world (McCracken 2014). As Els got older, experimental composers sought to blur the line between the two disciplines even more by making ‘music from everything. Fugues from fractals. A prelude extracted from the digits of pi. Sonatas written by the solar wind’ (142). Additionally, at this time, a school of musical thought emerged around ‘biocomposing’; music based on mathematical patterns found within the body (330). As Els observes: ‘Brain waves, skin conductivity, and heartbeats: anything could generate surprise melodies’ (330). One night, he hears compositions based on DNA sequences on the radio, which inspire him to compose his own music with biological material: ‘soundtracks extracted from DNA—strange murmurings transposed from the notorious four-letter alphabet of nucleotides into the twelve pitches of the chromatic scale. But the real art would be to reverse the process, to inscribe a piece for safekeeping into the genetic material of a bacterium’ (333). DNA appears to Els a ‘virgin territory’, ‘a newfound land’ free of ‘the twelve black and white bars in front of musical freedom’, as the medium is made up of patterns that are strange to the human ear, resisting any conventional cognitive-aural framework of understanding (334, 355). Put differently, he believes the genetic material will provide him with ‘[a] grammar but no dictionary, sense but no meaning’, a musical language outside of music’s overdetermined semiotic system (213).

Changing his medium, however, does not free him from constraints on his artistic expression. Rather, in writing music through DNA, he participates in a whole new network of remediation that radically undermines the very essence of music itself. Despite being a virgin medium for composing, DNA comes with its own set of limitations, requiring several technologies and layers of mediation. For example, in order to transform the genetic material into a ‘jukebox’, Els has to convert music ‘into a string of zeros and ones, [which] [are] then converted again into base four’ (359). Subsequently, he needs to ‘[divide] those two numbers to produce a short key’ that is embedded in the material with the help of advanced laboratory tools as if he were ‘put[ting] the tape inside the player’ (359, 333–334). Thus, he transforms analog music into a digital signal and back into an analog format, after which the musical sequences in the cell mutate, creating an ‘endless change in the musical message’ (334). Els considers this continual change to be ‘more like a feature than a bug’, thus underlining his perception of DNA as a software algorithm. He transforms himself into a ‘biohacker’, a composer-turned-programmer who writes a musical code within the genetic material that will, in turn, keep rewriting its own script until eternity (356). As such, he trades the role of author for that of an ‘author-initiator’ to borrow Hopfinger’s term (2020: 171). In fact, he has no other choice, since the limitations of his own bodily apparatus—his hands, his mind, his ears—limit his own power over the genetic medium. His creations cannot be seen, heard, or played, having become almost unrecognizable through the medial constraints of the DNA. Thus, the constraints of the medium and his own human capabilities inherently shape his artistic expression in new ways, further undermining Els’s dream of authentic artistic expression. Els, however, is drunk on the promise of a musical revolution, exclaiming: ‘What’s more beautiful than music you can’t hear?’ Still, his words leave the reader wondering: what’s the point of music you can’t hear?

In composing music through the automated self-replicating chains of DNA, Els brings his ‘merely human capacities to the sphere of the transhuman-superhuman bordering the divine’ (Fernandez-Santiago 2019: 135). But in the process, he loses music’s essence, its defining characteristic: its ability to move audiences through sound. In Els’s pursuit, superhuman artistic transcendence comes at the cost of a human recipient. While Powers does not actively frame this development in one way or another in the novel, the reader cannot but feel that somehow such a bargain defeats art’s purpose. Consequently, the narrative appears to raise the question: is art’s ability to be experienced ultimately more important than radical medial innovation? In a world where artists are continually transcending the limitations imposed on them by their bodies and minds through digital technologies, are the experiential boundaries of artist and audience still important (enough) to cater to? In an interview, Powers seems to suggest so, observing that ‘if you throw away everything of significance in the name of the new and in the name of attention and transformation, then you’re left asking people to attend to something that isn’t human, that is too alien to give any kind of solace or understanding’ (McCracken 2014). Thus, he argues that conventions and constraints in art are useful as they allow an artwork to communicate with an audience—take away all that is familiar and an audience will have no way of relating to the work. Thus, while the structural limitations imposed on the artist appear to stand in the way of innovation, he observes, they perform the vital task of anchoring the work’s connection to the audience.

Els’s DNA compositions present a far more extreme vision of digital technology’s impact on artistic creation than ASR, pushing the idea of machine-assisted authorship to a radical degree. While ASR modifies the writing process by automating transcription and, increasingly, the generation of text, it remains tied to human cognition: writers still shape their ideas, revise their sentences, and impose their own creative vision on its in- and output. By contrast, Els’s turn to DNA as a compositional medium imagines a form of artistic creation that entirely transcends human sensory experience and interpretive control. His compositions are encoded into biological material that evolves independently, making them not just difficult to access but fundamentally unstable, shifting beyond the artist’s grasp in ways that no ASR-generated output would. In this way, Orfeo envisions a future in which digital mediation does not simply assist or enhance human creativity but ultimately takes over artistic control altogether, out of human reach. Additionally, if ASR raises concerns about diminished authorial agency due to the influence of software programming, Els’s work opens up an even more pressing question: what happens when art ceases to be made for human perception at all? His DNA music is not just unhearable and unplayable by traditional means—it is conceptually alien, existing in a realm where neither the artist nor the audience can fully comprehend or engage with it. While ASR still operates within the boundaries of human language and cognition, Els’s compositions gesture toward a post-human form of art that no longer relies on human senses, intuition, or meaning-making. The novel thus sketches an image of a digital future in which art may not merely involve new tools that expand human creativity but could lead to artistic production that is entirely untethered from human experience itself. Simultaneously, however, Orfeo reminds us that art relies on meaning generated within a network of remediation that includes not just writing technologies but also authors and readers. Thus, while future literary endeavors involving digital media may revolutionize storytelling, giving rise to new writing technologies and experimental narratives, it is important to keep an eye out for its impact beyond that; how expression is restricted by new networks of remediation, how authorial agency is shared or even transferred to technologies, and how, potentially, the price of progress will be the presence of a beholder. Literature, as Powers’s work testifies to, plays a pivotal role in exploring the complexities and (philosophical) impact of these networks as they evolve to include increasingly advanced digital technologies. In tracing these speculative frontiers, literature becomes a kind of interface itself—a place where we reckon with the promise and peril of art no longer made with or accessible to us.