SOLO STUDY

Since we perceive the world using multiple senses, immersion is often associated with the high pervasiveness and fidelity of multisensory inputs, such as visual, auditory, and tactile cues (Striner, Azad & Martens 2018). A good interactive project needs to attract the audience and mobilize the multi-faceted senses of the audience as much as possible, so I tried to use body movement, image, filters of camera live recording and sound effects. To attract viewers and resonate more easily, I chose Pikachu, which is the representative of Pokemon. The voice of Pikachu is also audiences might familiar with and its cute appearance might attract attention at first sight as well.

The camera live recording is divided into two parts, which are black-and-white and colorful. In the part with filters, the color and sharpness would change in random. Pikachu would sit on your shoulder (as in the animation of PokemonGo), but it would move followed by your ears or head. In other words, whatever how you move, Pikachu always sits on your shoulder! When you move, the square where your image appears will play a sound of Pikachu, so you can choose any sound effects by changing your position. It seems like the audience can choose sounds by their movements and dub Pikachu.

I really developed lots of skills in the project, the most typical one is to learn to make moveable image overlays. However, connecting the image with the exact part of the body and find out the accurate data for the movement is the hardest thing. Thanks to Camille’s tutorial, jit.scissor and link sounds to each part are easy to understand. Fortunately, the product is almost the same as what I expected, although I changed ideas many times during the process.

The expected challenges I am considering is how to get more interactions, for example, what about using audio? Moreover, to make the project more attractive is another necessary point. A great interaction project needs to attract the audience and focus on it for a long time. How can I make it become more interesting, attractive and long life span?

 

GROUP STUDY

Our inspiration comes from my assignment 1 solo study and the dancing machine that normally appears in the game center, which is to combine video and audio is a good point to grab attention because the more senses we involved, there will be more feelings of participation.  We divided the project into two parts which are video and camera live recording and each part consists of several squares. Moving your whole body or just upper body, and follow the dance video we provided, different sound effects(when the users correctly move their limbs, there will be a sound of “perfect”) can be operated by users’ movements. Combining both technical operation and inspiration, we hope to create something that viewers/users can play with by simply moving their bodies We would like to explore the idea of altering the reflection of reality, moreover, build the connection between viewers/users and the project by MAX software.

It is important to find a suitable and interesting video which can let audiences add sound effects because it will determine whether the audience is interested in continuing this project. Moreover, the non-tactile project is more difficult, the technique we develop is to combine the camera living recording, video and sounds into one project and play at the same time. Each square connects to each kind of sound, by recognizing the movement and location of the viewer’s hands, the sound will be played and matched to the video.

The hardest part is to consider how to make the project interesting by using multiple senses and how to grab the audience’s attention at first sight. However, it is still not an excellent work because we expect to make one action to control one sound, for instance, when you wave hands, ‘claps’ will be played; when you kick the leg, another sound will appear. It is more like ‘dance machine’, you dance followed by the video and the multiple sounds or music will be played to match. Thus, it will cover more sense of movement and this is what we want to achieve in further study.

 

REFLECTION

To be honest, the non-tactile project seems harder for me. Although I came up with many ideas, due to the limitation of knowledge of Max, some ideas are difficult to achieve. The solo project plan could not be determined until the last week, but fortunately, I got what I want finally! For the assignments, the knowledge learned on the studio is not enough, I must spend more time on extracurricular searches, such as on forums, cycling 74 and Max help tools. This really helped me a lot on the technical issues and enjoyed the process of practice, especially after the success of the project. Moreover, thanks to Camille’s patient, every student has as many questions as I do, but Camille still spent extra time to solve for us! Thank you so much, Dr. Camille!

Due to the difficulty of the non-tactile project and the break of Easter holiday, the workload seems to be much bigger than before because we need to reasonably arrange the work of the team, even if there are two members who will be absent for two weeks. Moreover, due to the Easter holiday, two lessons will be missed, we did not have enough time to chat with Camille if there were any problems. Fortunately, brainstorm always happens in our group, we got two plans in case one of them cannot achieve. We are divided into two groups and work together in the last week. I learned the importance of teamwork, and it is most important to say what you think and trust each other in teamwork. We believe that the other party can take a good job in the field that they are good at and complete the task on time, even if we can’t get meetings. Fair work and trust are really important in teamwork! Of course, arranging time is another necessary aspect, because the solo project has almost occupied a place as well.

In the first assignment, ‘how to get interactions with audiences’ is the primary problem I thought about, but now, I need to think further, which is ‘how can I grab attention and make audiences focus on’. Since we perceive the world using multiple senses, immersion is often associated with the high pervasiveness and fidelity of multisensory inputs, such as visual, auditory, and tactile cues. (Striner, Azad & Martens 2018) Therefore, try to fully use transmedia is what I expected to achieve, such as images, sounds, movements, and camera. I combined as many media elements as possible in my work.

I exactly learned more techniques of interactive art making

1. Finding inspiration from life is another skill I have learned. Through the first assignment, others’ feedback is ‘This project is very interesting; the video is very attractive because the anime is cute; and the idea of dubbing the video is cool because this process is familiar to everyone, so it is easy to understand and operate ‘ Thus, I still chose the element that everyone is familiar with this time, which is Pikachu. For the group project, dance machine is the game what we familiar with, and it is the key point that inspired us.
2. ‘Linking is a crucial agent in the intermodal composition. Its function is analogous to superimposition in stereoscopic vision. It causes the sounds and images to interact and create the added dimension. In the process, two elements are unified within this new dimension.’ (Basic Theory of Intermedia Composing with Sounds and Images, 1997) I hope to use a variety of media elements to make the audience more engaged and contextual, so the fusion of images and sounds is the key to projects.
3. ‘The intermodal experience can be highly immersive. The multi-sensory content can make it extremely rich and intense. The intermodal gestures act as very strong stimuli. They can have a very direct, almost physical impact.’ (Basic Theory of Intermedia Composing with Sounds and Images, 1997)  This paragraph really resonated with me! When audiences see Pikachu, they might think of its sound, so I connect the sound, movement, and image to the work. This theory also applies to the group project, each interactive multimedia offers a different approach, the sound effects also works on the video.
4. ‘Creation is the pleasure participants get from having the power to create something while interacting with a work. It is also the pleasure participants get from being able to express themselves creatively. Exploration is the pleasure participants get from exploring a situation. ‘ (Costello & Edmonds, 2007) In the solo project, the content of the image is the first step in attracting audiences. When audiences move, Pikachu moves with it, reminding audiences of the scenes in the Pokémon they are familiar with. When using audio in non-tactile interactions, which is my favorite skill in this project, audiences actively participate in the activities because their actions can change the sound of Pikachu. In other words, they can create sounds for Pikachu. For the group project, the sound effects are controlled by audiences, they can also dance followed by the video so that create their own ‘sound video’.
5. The saturation, brightness, and contrast of the webcamera are what I learned as well. By altering these elements, the changes in colors and shapes are really funny. Thus, I asked myself, ‘What effect does color, shape, tool, and image change have on the senses of artists and audiences? What is their role? Can they bring more interactive experiences to the audience?‘

I think the most successful part of both solo and group works are to create a project based on the elements that audiences are familiar with, and make full use of the elements of sound, camera filters, images, and movements to give the audience a sense of participation in as many aspects as possible and try to make the project look less boring. The movement of hands to change sounds can increase the audiences’ experience. Non-tactile interactions are special and more interesting because the participants’ behavior does not require special performances, which makes everything simple.

To improve my solo project, the feature changes of the webcamera is what I hope to improve most. The filters, saturation, brightness, and contrast of the webcamera exactly works a lot and obvious to cause visual effects. I hope that I can develop more skills in this function in future work. Moreover, the live recording of the sound is also lacking. This would be another good feature to get interaction with audiences, so I would like to think about how to combine it into my next project. The last weakness is to make the project more attractive so that it can grab audiences’ attention for a long time. Maybe the content needs to be deeper.

For group work, the matching of sound, movement, and video are useful but if movements can control more elements would be better. For example, it might be a good idea if using 3D models in the project. Although this project is successful, due to time and skill constraints, I believe that our project has more room for development to improve interactive art.

I really enjoy the process of learning to use Max, even if it’s really hard and the related resources are limited, but it forces me to study and develop on my own. I exactly enjoy the process of doing projects researching references because I can get the sense of accomplishment after completion. At the same time, I really appreciate the help and patient guidance of Carmel! I am confident to welcome the next project.

 

References:

‘Basic Theory of Intermedia Composing with Sounds and Images’ 1997, Adam Marszelek, vol. XIX, pp. 43-50.

Costello, B & Edmonds, A. E 2007, ‘A study in play, pleasure and interaction design’, Designing Pleasurable Products and Interfaces.

Striner, A, Azad, S & Martens, C 2018, ‘A Common Framework for Audience Interactivity’, viewed 18 March 2019, <https://arxiv.org/pdf/1710.03320.pdf>