Expression beyond language

Grammarian Lynne Truss refers to emoticons as a “paltry substitute for expressing oneself properly (Park et al., 2015).” The problem with this claim is that it assumes that expressing oneself properly is actually possible through the means of language and furthermore it assumes language is the only apparatus capable of properly capturing one’s expression. This simply isn’t true. There are infinite subtleties and nuances we as humans cannot tie up neatly with words. It’s naïve to believe otherwise. It would be unwise to dismiss facial expressions during a conversation, so why deprive ourselves of a proxy that allows us to convey our approximate expressions. Choosing to limit ourselves to text when we are messaging puts a wider gap between real life communication and digital communication. Ideally, digital communication should aim to replicate real life communication as closely as possible. Real life is more than language; it is a full sensorial experience.

When approaching my study of the rhetoric emoticons, I immediately thought of my sister. We are very close, but see each other very rarely. That said, we rely heavily on digital communication. We use a lot of emoticons in our conversation, and I felt that our messages were clear evidence of the rhetorical power of emoticons. They always seem to enhance our conversations. Since starting my research on emoticons, I have made an effort to pay attention to whenever emoticons were used in conversations with my sister and what function they serve. I noticed that when my sister used emoticons I found myself picturing my sister’s face more, which made me feel closer to her. Typically when she sent text only I was focused more on the text and how to respond with words.

However, it acts as a good basis for my plans to do an exploratory descriptive case study on the use of emoticons between my sister and me as a promoter of affective social presence. Specifically, as I approach my work with the apparatus of the emoji keyboard, I expect to see evidence of the correlation between the modality of emoticons and perceptions of affective social presence. While this study shows a strong effect of the modality of emoticons in workplace communications, I would think this effect would be even stronger in interpersonal communications that are often more affectively complex, include more tonal nuances, and are far more common than costumer service interactions.

Absence of physical presence

Specifically they focus on adding an affective social presence to costumer service communications, as they cannot logistically create a physical social presence. Often times in life this physical social presence is impossible to have, even with the people we are closest to. The article also mentions that linguist David Crystal sees emoticons as a “crude way of capturing some of the basic features of facial expression” and he maintains that “their semantic role is limited.” Garrison et al. suggest that even a crude substitute brings the instant messaging experience closer to face-to-face communication, and urges that the emoticon is far more complex than we think. Furthermore, perceptions of intimacy may be a mediator between emoticon usage and protection against the negative effects of separation from loved ones. This fully supports my own expectation that emoticons play a role in feeling more connected in long-distance relationships (romantic and platonic). My decision to focus on conversations between my sister and me was based on the fact that we are typically far away from one another and rely more on digital communication. Emoticons are often the easiest way to communicate our feelings quickly so we feel more synchronized and perceive an affective social presence of one another.

Beyond Emotions

I would agree with this sentiment, and further the argument against Crystal’s claims by reminding readers that emoticons have extended beyond facial expressions. There is now a full keyboard including flags, foods and a variety of other objects. This allows for more varied and extensive horizons for those to choose to dive into the world of emoticons. The article also addresses the idea of emoticons as “inventional”. This is certainly something that I will be exploring in my project because my family in general has a very strange and idiosyncratic sense of humor. I have been known among my friends as “an emoticon queen”, but I would attribute this to these inventional uses of emoticons that are very common in my family’s digital dialogue.

In their 2011 article “Conventional Faces: Emoticons in Instant Messaging Discourse”, Garrison et al. discuss the rhetorical significance of using emoticons, particularly in idiosyncratic ways that exist outside the assumed shared body of knowledge employed during emoticon usage.

The Shared Body of Knowledge

The study that the article focuses on the conventions on emoticon use, and the authors maintain after their analyses that instant messaging users are accessing a shared body of knowledge. This shared body of knowledge will be a focus of my project as I look at the effect of generational gaps on emoticon use. As I compare my texting conversations with my sister with those I have had with my mom, this idea of a shared body of knowledge will be evident.

Emotional Synchronicity

Modality is defined by the authors as a “medium’s capacity to transmit multiple cues (pg. 2).” Modality can include a variety of non-verbal cues. In this case they are primarily visual modalities, such as images or emoticons. The authors hypothesized that emoticons would promote affective social presence therein enhancing customer experiences, because they function as an indicator of “socio-emotional tone” and context that would typically be provided in face-to-face communication by body language, facial expressions, and gestures.

Modality promotes immediacy. They found a significant main effect of modality on affective social presence. This modality measure was a composite of all modalities, not emoticons alone, but suggests that the modality of emoticons does have an effect on the the perception of affective social presence.

In Janssen et al.’s 2014 article “How affective technologies can influence intimate interactions and improve social connectedness”, the relationship between emoticon communication and perceived intimacy is explored and results of their study on the subject are reported. The authors emphasize the importance of social connectedness in everyday well-being and combatting loneliness. The findings overwhelmingly affirmed their hypothesis with a very large effect size. Increased emoticon use did lead to reports of higher intimacy in messaging conversations. This suggests that emoticons also may promote better social connectedness, relationship quality, and overall well-being, because perceptions of intimacy are predictors of all these things.

The Ideal for Emotional Synchronicity

Park and Shundar suggest two main apparatus that influence perception of social presence: synchronicity and modality. The authors suggest that synchronicity is the temporal immediacy of communication, so that face-to-face communication has the highest possible synchronicity and digital technologies should aim to mimic this synchronicity as much as possible in order to achieve perceptions of social presence. Affective computing refers primarily to devices that are able to recognize and process emotions. The goal of affective computing is to create an effortless and automatic way of sharing emotions with our social network. This plays into the idea of synchronicity and affective social presence that Park and Shundar stress in their article “Can synchronicity and visual modality enhance social presence in mobile messaging?”. Affective computing is a major trend in the digital industry right now, but it is merely at the precipice. Janssen et al. suggest that the theories of affective computing can also be applied to human-human interaction, both face-to-face and in particular during computer-mediated communication. The study is aimed at using these theories to explore the possibility of automated emotion communication.

Where do emoticons fit in? Human-computer connection.

Conversations with my sister, Natalie:

boy-feelings.

getting-away

interpretive dance

I-speak-American  queen-of-england  when

Conversations with my mom:

turtle-trafficmouth-fell-offwhere-the-dancing-girls-live

Affective Technologies and Social Connectedness: Annotated Bibliography #3

In Janssen et al.’s 2014 article “How affective technologies can influence intimate interactions and improve social connectedness”, the relationship between emoticon communication and perceived intimacy is explored and results of their study on the subject are reported. The authors emphasize the importance of social connectedness in everyday well-being and combatting loneliness. Emoticons are a part of a larger group of digital tools that the authors consider to be tools of “affective computing”. Affective computing refers primarily to devices that are able to recognize and process emotions. The goal of affective computing is to create an effortless and automatic way of sharing emotions with our social network. This plays into the idea of synchronicity and affective social presence that Park and Shundar stress in their article “Can synchronicity and visual modality enhance social presence in mobile messaging?”. Affective computing is a major trend in the digital industry right now, but it is merely at the precipice. Janssen et al. point out a variety of ambitious projects that have emerged in the past few years that have embodied the affective computing idea. They mention the Hug Over a Distance project, in which an inflatable vest is worn and remotely triggered to simulate the sensation of a hug. The primary issue with this item and a variety of other affective technologies is obviously practicality. Janssen et al. ran this study primarily in order to look at the feasibility and effects on perceived intimacy of automated affective computing (i.e. emoticon use), but to compare the success of the automated affective computing, they compared this to non-automated messaging with emoticons. It is these findings that are most relevant to my thesis.

Janssen et al. suggest that the theories of affective computing can also be applied to human-human interaction, both face-to-face and in particular during computer-mediated communication. The study is aimed at using these theories to explore the possibility of automated emotion communication. While the study focuses on a variety of hypotheses, I am focused primarily on their findings on whether or not increased emoticon usage leads to increased perceived intimacy. The findings overwhelmingly affirmed their hypothesis with a very large effect size. Increased emoticon use did lead to reports of higher intimacy in messaging conversations. This suggests that emoticons also may promote better social connectedness, relationship quality, and overall well-being, because perceptions of intimacy are predictors of all these things. Furthermore, perceptions of intimacy may be a mediator between emoticon usage and protection against the negative effects of separation from loved ones. This fully supports my own expectation that emoticons play a role in feeling more connected in long-distance relationships (romantic and platonic). My decision to focus on conversations between my sister and me was based on the fact that we are typically far away from one another and rely more on digital communication. Emoticons are often the easiest way to communicate our feelings quickly so we feel more synchronized and perceive an affective social presence of one another.

Janssen, Joris H., Wijnand A. IJsselsteijn, and Joyce H.D.M. Westerink. “How Affective Technologies Can Influence Intimate Interactions And Improve Social Connectedness.” International Journal Of Human-Computer Studies 72.1 (2014): 33-43. Academic Search Complete. Web. 28 Apr. 2015.

Conventional Faces: Annotated Bibliography #2

In their 2011 article “Conventional Faces: Emoticons in Instant Messaging Discourse”, Garrison et al. discuss the rhetorical significance of using emoticons, particularly in idiosyncratic ways that exist outside the assumed shared body of knowledge employed during emoticon usage. The article does a good job of reviewing some of the literature on emoticons, particularly those who have negative opinions on the use of emoticons. They include a quote from grammarian Lynne Truss who calls emoticons a “paltry substitute for expressing oneself properly.” The problem with this claim is that it assumes that expressing oneself properly is actually possible through the means of language and furthermore only the means of language can properly capture one’s expression. This simply isn’t true. There are infinite subtleties and nuances we as humans cannot tie up neatly with words. It’s naïve to believe otherwise. Choosing to limit ourselves to text when we are messaging puts a wider gap between real life communication and digital communication. Ideally, digital communication should aim to replicate real life communication as closely as possible.

The article also mentions that linguist David Crystal sees emoticons as a “crude way of capturing some of the basic features of facial expression” and he maintains that “their semantic role is limited.” Garrison et al. suggest that even a crude substitute brings the instant messaging experience closer to face-to-face communication, and urges that the emoticon is far more complex than we think. I would agree with this sentiment, and further the argument against Crystal’s claims by reminding readers that emoticons have extended beyond facial expressions. There is now a full keyboard including flags, foods and a variety of other objects. This allows for more varied and extensive horizons for those to choose to dive into the world of emoticons.

The study that the article focuses on the conventions on emoticon use, and the authors maintain after their analyses that instant messaging users are accessing a shared body of knowledge. This shared body of knowledge will be a focus of my project as I look at the effect of generational gaps on emoticon use. As I compare my texting conversations with my sister with those I have had with my mom, this idea of a shared body of knowledge will be evident. The article also addresses the idea of emoticons as “inventional”. This is certainly something that I will be exploring in my project because my family in general has a very strange and idiosyncratic sense of humor. I have been known among my friends as “an emoticon queen”, but I would attribute this to these inventional uses of emoticons that are very common in my family’s digital dialogue.

Garrison, Anthony, et al. “Conventional Faces: Emoticons In Instant Messaging Discourse.” Computers & Composition28.2 (2011): 112-125. Academic Search Complete. Web. 28 Apr. 2015.

Synchronicity and Visual Modality in Mobile Messaging: Annotated Bibliography #1

In Park and Shundar’s 2015 article “Can synchronicity and visual modality enhance social presence in mobile messaging?”, they discuss the ways that a sense of social presence can enhance customer service in the age of digitalizing these services. Although this article specifically looks at creating this social presence in the work place, many of the theories they put forward are also applicable to personal everyday social interactions. Specifically they focus on adding an affective social presence to costumer service communications, as they cannot logistically create a physical social presence. Often times in life this physical social presence is impossible to have, even with the people we are closest to.

Park and Shundar suggest two main apparatus that influence a customer’s perception of social presence: synchronicity and modality. The authors suggest that synchronicity is the temporal immediacy of communication, so that face-to-face communication has the highest possible synchronicity and digital technologies should aim to mimic this synchronicity as much as possible in order to achieve perceptions of social presence. Modality is defined by the authors as a “medium’s capacity to transmit multiple cues (pg. 2).” Modality can include a variety of non-verbal cues. In this case they are primarily visual modalities, such as images or emoticons. The authors hypothesized that emoticons would blank blank blank, because they function as an indicator of “socio-emotional tone” and context that would typically be provided in face-to-face communication by body language, facial expressions, and gestures.

In Park and Shundar’s study, synchronicity and modality are both treated as independent variables that influence the outcome variables related to costumer experience, mediated by social presence. They specifically hypothesize that modality will have a main effect on affective social presence. Their results were very substantial. They found a significant main effect of modality on affective social presence. This modality measure was a composite of all modalities, not emoticons alone, but suggests that the modality of emoticons does have an effect on the the perception of affective social presence.

This study of synchronicity and modality is very structured and empirical, using 108 participants and a number of measures to evaluate the relationships between key constructs. However, it acts as a good basis for my plans to do an exploratory descriptive case study on the use of emoticons between my sister and me as a promoter of affective social presence. Specifically, as I approach my work with the apparatus of the emoji keyboard, I expect to see evidence of the correlation between the modality of emoticons and perceptions of affective social presence. While this study shows a strong effect of the modality of emoticons in workplace communications, I would think this effect would be even stronger in interpersonal communications that are often more affectively complex, include more tonal nuances, and are far more common than costumer service interactions.

Park, Eun Kyung; Sundar, S. Shyam. “Can synchronicity and visual modality enhance social presence in mobile messaging?” Computers in Human Behavior 45.1 (2015): 121-128.

Hearing Hands

via Hearing Hands – Touching Ad By Samsung – YouTube.

This video is one of many emotional ads that have become increasingly popular in the media over the past several years. And let me tell you, they get me every single time. I am curious about what might be causing the rise of motivational/inspirational marketing. If you haven’t noticed the trend, check out this ad released by Dove celebrating curly hair, one of many ads released as part of their Campaign for Real Beauty. Or some of you may remember this ad released by Google Chrome in 2011 in which a father uses the internet to speak to his daughter and share memories with her as she grows up.

The service that Samsung is offering in their Hearing Hands ad definitely makes us think about the digital world we are living in. Technology is a subject of never-ending contention. Good or evil? Helping or hurting? Making us smarter or dumbing us down? In this ad, we are presented a case in which technology helps to create “a world without barriers” for someone whose impairment creates barriers that extend into nearly every aspect of their life. And personally, despite my instinct that somewhere an ad agent is buckled over with evil laughter, I’m a total sucker for it.

Singing Chords

In honor of finishing our sound unit, check out this amazing video of jazz singer Lalah Hathaway and the group Snarky Puppy. Prepare to drop your jaw when Lalah sings chords. I recommend listening to the whole video but you can skip to 6:10 if you are feeling impatient and want to hear something you might not have realized was even possible.

Mountains Beyond Mountains

When approaching my soundscape, I wanted to convey a space that is meaningful to me, and so I decided on a campsite loosely set in the blue ridge mountains. Having grown up in Asheville and reaching a time in my life when I will be saying goodbye to it in a more semi-permanent way, the mountains are something I have thought about a lot recently. As a group of good friends and I have been planning our camping trip over spring break, I thought a campsite in the mountains was a very fitting choice of soundscape for me at the moment.

Focusing in on the sounds that surround and penetrate a space is a fairly untouched territory for me. Much like most people, and in some ways even more so, I am highly reliant on and often consumed with my visual field. Sounds typically fall into one of three categories in my mind: music, voices, and utilitarian. Music is a substantial part of my life. My father is a music producer, and has always immersed my siblings and me in good music and knowledge about the world of music. With music (at least that on my iPod), as Michael Bull suggested, I am choosing an aspect of my atmosphere, juxtaposing it against my visual and physical world. I am able to do this because sound, in a sense, is an optional part of functionality in our daily lives. For able-bodied persons, we cannot really choose to do without our eyes actively involved in our goals as we go about our lives, though sometimes we give it our very best when checking a text while walking. Our ears, on the other hand, have some flexibility. While sounds may be helpful cues, we can afford to drown them out with music and create our own space within our auditory sphere. Of course, this does not always hold up, as voices play an extremely important role in our functionality as humans.

It is the sounds I describe as ‘utilitarian’ that are, to some extent, optional. Utilitarian may be a narrow word to describe them, but I merely mean to say they are related to the operational quality of things as opposed to an intentional aesthetic quality. For me, these sounds go unappreciated the majority of the time. Unappreciated in the sense that I don’t truly listen to them, they are simply a constant part of my environment. When reading Karin Bijsterveld’s article on industrial noise, I was at first surprised by the workers’ discomfort with the lack of sound, but upon thinking about it, I became uneasy when I thought of sound being stripped away from my functional environment. These sounds that I have typically ignored have a profound effect on how well I function as a human, but also, as Bijsterveld suggests, may have an aesthetic value in my interactions with the surrounding world.

This longwinded introduction was my mindset as I sat down to search for sound clippings to use in my soundscape. When thinking of the fundamental sounds of a campsite, I found that two things automatically came to mind: fire and insects. These are two sounds that represent the continuous background of camping at night. They gradually become a static default, almost like when you re-calibrate a scale to zero after putting something on it. I then tried to think of sounds that conveyed humans interacting with the space. The sounds of tent fabric and zippers popped in my head as something that is highly recognizable. At times I found that the impact of sounds I included unintentionally suggested something about the space. For instance, I included the sound of leaves crunching and a branch snapping without thinking about it much, but then I realized this conveys a campsite likely in late fall or early winter when the leaves and branches are dried up and on the ground. I then took note of the fact that I had been visualizing the woods in the fall with leaves changing color despite the fact that I had my spring camping trip in mind. Even in a space that has certain sounds associated with it, I found telling variation of possibilities.

Images: Series Three

porto-on-chicagoNat-lo-chicago Alana&Annie

“Be patient towards all that is unsolved in your heart and try to love the questions themselves.”
Rainer Maria Rilke


In this final set of images, I focused more on places than I did on people. I feel this is another very strong part of the way I want to represent myself. It is not only that I see traveling as an influential very part of my life, but in some ways I must admit, it makes a positive impression with others. Traveling is something that people respect, and I tend to put it at the forefront of my social media, even if it is not the most present thing in my life at the moment.

Images: Series Two

Nat-paintburren+portoLaugh

“I am trying to respect my own complexity”
Jeanette Winterson


I find it very difficult to talk about the process of choosing and editing these images because I feel that I am most often driven by what Barthes would refer to as the punctum. Describing a process that is felt more than thought is challenging, but thinking more about Vivian’s ‘non-representational’ concept I believe that I did focus on creating a ‘virtual’ experience within the images. At first I worked this virtuality to feel right for myself. After all, the assignment is about identity and if I don’t feel a strong virtual experience, it is likely the images do not portray my identity accurately. When finalizing the images however, I worked on creating small visual ties throughout that pulled the viewer through their experience. This was mainly accomplished using the window theme that I discussed during my first post, but also using color manipulation.