Final Project Presentation1: Liang

LookingOut 9: Liang

MusicInk (2013)
by Riccardo Vendramin & Gilda Negrini

MusicInk is a project aiming at music teaching. It creates a new way to understand music and interact with musical instruments. It is a toy that provides an innovative approach to music, allowing kids not only to draw musical properties, but also to compose a real symphony. The major technology covers a conductive ink by Bare Conductive, MPR121 controller, Arduino, LiPo shield, Bluetooth shield by Seedstudio, Android, Pure Data for Android, Pure Data patch. Children are able to complete a unique electrically conductive paint and then play their drawings with music.

LookingOut 8: Liang

Project #1 Bubble Cellar
BY Joelle Aeschlimann, Pauline Saglio, and Mathieu Rivier (2013)

This is an interactive installation composed of a bubble blower, a projector, wind sensors. It invites users to come near and blow into the blower to create bubbles just as we always do in our daily experience. Instead of creating real bubbles, the installation projects virtual soap bubbles on the wall, which rises with lightness and fragility in the air, leaving us in the expectation of its future explosion. Since it is projected shadows, it allows much more imaginations and wonders, which are turned out to be some animations when the bubbles are exploded.

Project #2 MoleBot
BY Design Media Laboratory (2012)

It is a mole robot built under a transformable surface that allows people to enjoy playful interactions with everyday objects and props, and experience what it feels like to have a mole live under your table. It can undergo many tasks, like pick fruits, lead objects... users are able to interact with it via gestures.

Final Project Proposal: Liang

Project: Inkative

INTRODUCTION

Project Intent

What does the interaction between humans and living creatures looks like? I hope Inkative be such a platform that records and depicts the dynamic interactive process in a tangible way. It is able to convert little creatures' (like gold fishes, bees, flies) reaction to surrounding stimulus into something that can be recognised and described. To some extent, it will provide a novel way for people to observe and learn how to communicate with these little lives.

PROJECT BACKGROUND

The original idea of this project came from a episode of film "Sherlock Holmes" (here). Out of curiosity about how people's behaviours or environmental changes affect little live creatures' behaviours, I started to read and review literatures and works of other researchers and artists. Here lists some of them:

David Bowen's fly works

fly revolver (2013) [link]: A revolver is controlled by the activities of a collection of houseflies, which are monitored via video  in an acrylic sphere. The revolver aims in real-time according to files' relative positions on the target. If a fly is detected in the center of the target the trigger of the revolver is pulled.

fly drawing device (2011) [link]: Based on the subtle movements of several houseflies, a artificial device is made to produce drawing on a wall. The drawing is composed of random and discrete strokes. There is a big sphere with a chamber on the top. When flies enter the chamber the device is triggered to begin the drawing.

Tangible Design's Picidae Chorus [link]

It is a kinetic, light and sound installation featuring 7 geometric birds, installed in the woods. It is lit up as the birds pecked a percussion box and slowly fade as they come to a stop. It provides a playful and engaging environment for visitors in the park.

Social Firefly by Jason mcDermott, Liam Ryon, and Frank Maguire (2011) [link]
It is innovative light artwork and also a community of friendly intelligent lights that influence one another. The fireflies are programmed to respond to light from their neighbours. In this community, popular fireflies become highly influential, whilst isolated ones must work harder to reach their friends. People can speak the same language and influence the interaction between community members by shining lights on these fireflies.

In conclusion, these works can be placed into two categories: bio controller and interactive installation. The former only focuses on what kind of outputs can be produced by animals' behaviours, lacking the ability to observe how environment can affect their behaviours. Therefore, it is more like a one-way system which addresses the variations of outputs but ignores the interactive input from ambient stimulus. Contrarily, the latter category focuses on how to simulate creatures with modern technology and fabrication method. It presents an interesting way to explore how animals' behaviours and habits affect people's social activities. So in most cases, installation art is the best form for this kind of system which invites audience to participate in. I hope to create a device based on the ideas of above works, enabling users to interact with animals, recording the dynamic interactive process, producing something can be perceived and studied by humans, and finally helping people better understand the communications with the other living friends on the planet.  

INITIAL DESIGN & IMPLEMENTATION

Prototype

Initial prototype is pretty simple. It consists of a large transparent sphere, 3 cameras and a swarm of houseflies. The cameras capture the movements of flies and map them to some digital outputs. The sketch is shown here. However, it fails to find the interaction between flies and people in some prior experiments. The possible reason is that the sphere is too large for flies to recognise the surrounding changes. So this design is claimed to death.

The project currently aims to observe the interactions between people and little animals. So here the application is about gold fish. The system looks like a stage, the fish tank can be placed on the stage. Two cameras are used to track the positions of gold fish in the fish tank and deliver data to the computer. Beneath the stage there is a chamber which looks like a CNC. In this chamber it will control a brush moving in 3D space to draw on a ink paper. Unlike CNC, it is has circular structure. A stepper motor is set at the center position enable the upper circular panel to spin. A steel rod is used as a bracket to support another stepper motor to drive the brush. With these two motors and such structure, the brush can go anywhere on a circular plane theoretically. The brush is mounted on a device which is driven by a servo, so that it can be lifted up and down to indicate the animal's vertical position. The form of drawing is Chinese traditional painting because it can reflect the animal's 3D position through the pressure and the amount of water of the brush. In addition, to address the interaction, proximity sensors are used to detect if there are people standing around the platform.

This is a initial sketch for this prototype:

The diagram of the system is:


Materials
Required: masonite wood, steel rod, ink paper, Chinese calligraph brush, acrylic board, felt
Maybe: PLA

Hardware
Required: Webcam, proximity sensor, stepper motor, servo, Arduino, computer, resistor, LED, jumper wire
Maybe: Raspberry Pi

Software
Required: Arduino, openFrameworks, openCV

Fabrication Processes
Laser cutting all plates and parts that the stage needs, including the brackets, the base, the circular plane, gears and any enclosures. For those laser cutting cannot satisfy 3D printing may help.

Budget
Name                                 Price                                  Number
Webcam                            $5.09                                     2
Masonite wood                   $39.95(12''*12'')                     1
Proximity sensor                 $13.95                                   4+
Arduino Uno                        $29.95                                   1
Brush                                 $4.38                                     1
Ink paper                            $2.00                                     n
Stepper motor                     $16.95                                   2
Servo                                  $8.95                                    1
Acrylic                                $15.00                                   1
Felt                                     $14.30                                  1

Timeline
10/14 - 10/20: schematic design, prototype scope, sketches, technology and material exploration, diagrams.
10/21 - 10/27: components purchase, openCV test in openFrameworks, finish object track and location in 3D environment.
10/28 - 11/03: physical structure design and fabricate all parts, mount circuit, test on stepper motor.
11/04 -  11/10: mount all part and test the mechanism, test on servo.
11/11 - 11/17: complete circuit logic, including the combination of stepper motor and servo, mesh up CV data and circuit program attempt 1.
11/18 - 11/24: mesh up CV data and circuit program attempt 2, andtest the entire system.
11/25 - 12/01: refine design, development and mechanism, test and prepare for final presentation (shoot video demo and final documentation).

Documentations go through the entire design and development process.

Challenges
1. Track fish's position with openCV and openFrameworks.
2. Data processing. Convert location(x,y,z) into the angles applied for stepper motors and the servo.
3. CNC-like mechanism design and fabrication.  

CONCLUSION
Inkative is a platform that records the dynamic interaction between people and fish and converts it into Chinese traditional painting. People are able to study the fish's reaction to ambient stimulus from these paintings. In the future, it can also be applied for other animal, like bees, flies and etc. It has large potential value in assisting people to understand how animals behaviour changes as the environment alters.

Whisker V1, V2 & V3: Liang, Wanfang and Ayo

INTRODUCTION

Tasked with the project of making a whisker for a 'chinchilla/human hybrid', we set off on an epic journey of discovering what it means to be a 'whisker'. The only constraint that guided our exploration was the requirement that the whisker retain sensitivity to a physical input.

VERSION 1

Our initial prototypes were created by exploring the materials within the immediate vicinity of the Code Lab. We sought to give form to a whisker that would respond to touch input. The image on the left was created from a wooden stick, zip ties, gel beads, and a modified plastic bottle. The idea behind its operation was to capture touch by bending the structure one way or the other.  

The image in the middle was created using a gear recycled from a rabbit, plywood, a metal rod, and three springs. The idea behind its operation was to capture touch input in the springs and use this energy to rotate the structure. One idea that was bandied around was to attach a pencil to the end of the whisker and have it record the touches imparted to it on paper. The advantage of this whisker was its pleasant one-to-one mapping between input(touch) and output (shape deformation). This property served as fodder for the third prototype.   

The third prototype embodied the form we imagined for a whisker that would respond to touch input. It was created using five springs, paper, and a metal rod. Because of its use of lightweight construction materials, it deformed quite pleasantly when touched and evoked a 'jello' metaphor. However, our desire for more interactivity inspired the future iterations of the whisker.

VERSION 2

After feedback from our peers, that our whisker reminded them of lunar modules, we shifted our design focus to explore this concept a little bit more. This design utilized acrylic and electronics to create a whisker that would light up in response to touch input. The shape of the whisker was designed to be hexagonal in order to allow the modular connection of an arbitrary number of these whisker modules.

VERSION 3

Further feedback from our peers revealed that while the second iteration was enjoyable, it lost the interactive capabilities of the initial prototype. Thus, we sought to integrate these capabilities into this final version. We utilized 3d-printed parts (to make the structure more flexible than acrylic), a ping-pong ball to (diffuse the integrated LED and add a level of interactivity), a coil of wire, and four springs. While retaining the lunar module feel of the previous iteration, interactivity was added by programming behaviors into the whisker. Upon initial touch, the whisker emits a cool blue light, a sort of welcome to the user. If the whisker is still held for a period longer than approximately 2 secs, the whisker proceeds to emit a pulsing green light that signifies it is communicating with either the user, or another module. Now, if the whisker is bent or deformed from its original straight orientation, it emits a red light and begins to vibrate signifying that it is uncomfortable and angry. These additional behaviours breathed new life into the whisker, transforming it from a simple input-output sensor, to a seemingly complex thing with emotions and motivations.

Hardware: Teensy 2.0, Whisker(Wires), RGBLED, Motor, CapacitiveSensor is a library in Arduino, just using a wire and a large resister (10MOhm) can let the Teensy (Can write code in Ardunio) to get

analog signal of touch.

Design process video:

Final version demo:

CONCLUSION
We present a series of iteration processes for designing a whisker. Along with the process and calendar addressed in MTI, we practice sketch design, prototyping, laser cutting and 3D printing. To make whisker interactive, we also put some sensor technology there and produce digital outputs. The next step for this prototype focuses on two aspects:
1. Shape redesign.
2. Multiple whiskers communication.

Links:
Fabrication Package: https://github.com/EdigaHe/Whisker-Fabrication-Source-Files/tree/master
Capacitive Sensor: http://mtif13.blogspot.com/2013/10/about-capacitivesensor-in-ardunio.html 
Teensy: https://www.sparkfun.com/products/11780
Teensy Source Code: https://github.com/wfdiao/CapacitiveSensorForArduinio/blob/master/CapacitiveSensorSketch/CapacitiveSensorSketch.ino

LookingOut 6: Liang

Not Yogurt
by Gabriela Gordillo

This is a smart setup for live music performance. Artists can play music from a customized disc which turns different sources into a loop pattern. The hardware configuration is composed of vynil, copper plaque, and electronic circuit. When the disc spins, the plaque touches the vynil and thereby connects the circuit to make it sound. Then the whole device would rebuild and adapt for live performance. For a better output, the cooper plaques are distributed with patterns to represent different music structures (a regular tempo, counterpoint, 3/2, etc), so that they could build a recognizable rhythm.

In addition, they use FeCl3 solution to print circuit.

The value behind is to enable live music performer to have the freedom to create music in a new way. Live performance is an activity that integrates artist's talent, emotion, creativity and improvisational abilities. The printed disc helps them achieve more creative and personal music outputs.

Picidae Chorus
by Tangible Interaction (Alex Beim, Mike Manning, Pablo Gindel, Kenji Rodriguez, Andy Meakin)

It is a pubic artwork for Stanley Park's 125th birthday. It is a kinetic, light and sound installation featuring 7 geometric birds, installed in the woods. It is lit up as the birds pecked a percussion box and slowly fade as they come to a stop. The total height for each bird is 12 inches. They sometimes peck in union, sometimes peck alone. Not similar to the real woodpeckers, they embrace simple principles - creation, imagination, repetition, and propagation. They are made from acrylic and each one has a RGB LED module inside and a solenoid to control movement. The basic electronic part is about Arduino. It provides a playful and engaging environment for visitors in the park.

Bonus: M-blocks by MIT (aha)
Before I shut down my laptop I found this from friends, similar to cublets.

LookingOut 5: Liang

Project #1: Denki Puzzle
by Yuri Suzuki and Technology Will Save Us

This project uses a kit of bespoke PCB components to create a puzzle device which enables users to get a better understanding of how things work by assembling these pieces. Users can create fully working appliances such as a radio or light system. Through customer electronics people have the ability to make things work by simple manipulations. At this point, they also have the chance to know how a digital device or a electronic system exactly works. It exposes the basic principles of things to users, which helps people more easily and more intuitively to access technology and things.

Project #2: The Sound of the Earth
by Yuri Suzuki

This project maps recorded music from those countries which Yuri has traveled through to a globe. The grooves represent the outlines of the geographic land mass. Each country on the disc is engraved with a different sound, as the needle passes over it plays field recordings. The recorded music ranges from traditional folk music, national anthems, popular music to spoken word broadcasts. It applies the metaphor of a globe in the sound recorder, establishing a strong connection between personal travel and music recordings. It is a simple but reasonable design.

Whisker Evaluation: Liang, Wanfang, Ayo

Evaluation of Jake's whisker project

Overall, this device works like a detector, a sound maker, a touch sensitive convertor, or (according to Jake) an instrument. The whole installation uses Arduino board and piezo to map audio frequency to users' interaction with the whiskers. The users are engaged in playing with these whiskers to create diverse sounds. So it is a playable and interactive instrument.

The latest version adds audio feedback to the whisker system and redesigns the structure that they are no longer fixed on a base but suspended in the air. In addition, whiskers are mounted toward different directions which somehow makes the entire system dynamic. The audio mapping is brilliant but it will be better if these whiskers have different mappings with sound (like volume...). Compared with previous version, it lacks resonate which may make whisker more interactive and interesting. Although audio feedback is added in new system, the interactions between users and the whiskers stay consistent. 

The technology (Arduino, piezo, and speaker) is appropriate for this project. It is very simple, clean and smart. The first version's LED and other proper sensors can be added in this project. Alternatively, more tactile components like touch sensor or vibration sensor can be haptic input and output. As for the design of the shape or the look, it depends on the application and purpose of this installation. For example, whiskers can be earrings which is mobile and delicate, whilst they also can be huge, complicated and static exhibition device standing in the corner of a museum. We recommend that the design of next version should be more fancy and imaginative, may be more mobile or distributed.

Evaluation of our whisker project

We redesign the main body of the project with acrylic laser cutting method. We also add electronic part which enables the whisker to sense the touch via capacitive sensing. With a LED inside a pingpong ball, it lights up diffused blue light when someone touches the whisker, just like the heart of this creature. It lies a foundation for future communication among several devices. We hope to create little creatures that have the ability to communication, exchange information and interact with others.

The first version uses soft wood and springs and other plastic material to build three types of flexible prototypes. The simple and light configuration encourages users to touch it, move it, and interact with it, even though they do not have the digital feedback as that in the second version. The fixed and static shape and structure keeps users from playing with the whisker. On the other hand, the whisker cannot rotate as it performs in the first version, and this makes it less functional. In the future development, we should step back to empower more flexible attributes to the creature and make something more imaginative and interactive. Since the major task of the whisker we create is to establish communication and connection with multiple whiskers, we should keep both natural and simple design and various digital outputs. With the characteristics of materials and physical construction, we believe the design itself will offer a lot of affordances for users and other whiskers.

Based on the communication between multiple whiskers, it has a wide applications in the future. One example is to educate kids some simple concepts, like transitivity and combination. It is also a good kit for young children to explore the culture of communication through playing with these whiskers. In conclusion, it has a positive impact on children's recognition development and constructivism learning.

LookingOut 4: Liang

Project #1: Social Firefly
by Jason mcDermott, Liam Ryon, and Frank Maguire for the Vivid 2011 Festival

This project is an innovative light artwork and also a community of friendly intelligent lights that influence one another. The fireflies are programmed to respond to light from their neighbours. In this community, popular fireflies become highly influential, whilst isolated ones must work harder to reach their friends. People can speak the same language and influence the interaction between community members by shining lights on these fireflies.

Technology: This project uses a bunch of fabrication methods including SLA 3D printing, plywood, acrylic laser cutting, polystyrene vacuum forming. It contains 95 components in each unit and eventually there are over 4800 parts for one tree decoration. The electronic part is Arduino Pro Mini.

Interaction: These devices are set up on any physical trees and perform as real fireflies. The people who walk around them have some influence on them and they shine lights according to the surround variants.

Value: This project proposes a more physical way to simulate other creatures' behaviours and explores the beauty of our interaction with our friends who also live on our planet. It gives us some ideas on how to apply fabrication methods to record human's dynamic and casual interaction with creatures which may be helpful for human being to better understand biology. This will be the perspective my final project will stand on and dive into much deeper and further.



Project #2: Firefly
by Jason Krugman for the Flashlight Nuit Blanche event in New York 2011.

The Firefly wind light system constructs a lot of arrays of small LED/switch pairs which sense the wind to light up when wind blows through. 

Technology: Adopting LED/switch that sense wind on a small circuit unit. It's principle is that as the wind blows to an electrified wind chime it will contact with the metal gong and close a circuit. 

Interaction: It is an huge art installation which is supposed to sense the immediate environment.  The wind blows it will change the states of those LED/switch. 

   

Value: This is another project related to fireflies. It shows how to make something belonging to art by using simple sensing technology to react to environment. My reflection to this art installation is that some stimulus can make a tangible design different.

Whiskers V2: Liang, Wanfang, Ayo

Our project focuses on the connection among whiskers. We use laser cutting to assemble this little creature's shape. Capacitive sensing technology enables it to detect any touch from any conductive stuff. So multiple whiskers can communicate with each other by touching. The LED and the pingpong ball compose the heart of this creature. When you touch it, it will blink which means it knows the touch, thereby the connection.

LookingOut 3: Liang

Project #1: AnyTouch 
by Digital Labs Paris & Ayotle

AnyTouch is a prototype that can make any object tactile. Based on computer vision software Ayotle provides, designers and engineers from Digital Labs Paris build this project. The original idea comes from the popularity of tactile used in screen interaction. Their goal is to make everything tactile for human's interactions.

Technology: based on the association of a 3D camera and a vision software designed by Ayotle, AnyTouch turns any attributes of an object (like shape, color, volume, etc.) into interactive elements. In addition, it even can detect the user's gestures.

Interaction: the user places any object on a white paper which is within the range of the 3D camera. Then the user sets the control for the object by drawing sketches around it (e.g. drawing a slider for a block, so moving the block can be as an input of the slider). Then the camera captures the objects and sketches and processes the data collected. As a result, user's control and interaction with the object would be interpreted as input to digital content. So the human's manipulations on daily objects are effect operations on virtual bits.

Value: AnyTouch tells the story that anything could be a trigger for users to access digital world. With certain modern technology, we have the ability to make things tactile and interactive, maybe in an augmented way or maybe in a more mechanical way.




Project #2: Super Angry Birds

by CIID

Super Angry Birds is a SUPER tangible version of "Angry Birds".

Technology: basically, this project adopts the theory of a motorised fader and records the value of the curve the user draws by MAX/MSP and Arduino. 

Interaction: Firstly, the user draws a force line and the datas of this line are stored in the table. Then the project sends the current position of the slider through the table and the extracts the value to send to the motor that applies an opposing force.

Value: It is definitely interesting! 

LookingOut 2: Liang

Project #1: Inshin-Denshin
by Yuri Suzuki, Olivier Bau and Ivan Poupyrev at Disney Research.
Display and honorary mention at ARS Electronica 2013.



1. Fields (Disciplines):
Sound, HCI

2. Technology

Digital audio processing, biology

3. Interaction

Ishin-Denshin proposes that sound and voice could be transmitted in interpersonal communication through touch. One person record sounds with a microphone and transmits them to another person by touching other's ear. The sound can be transmitted from person to person.

4. Design

The installation consists of a microphone that can record sounds and transmit them through touch. The microphone only records sounds of amplitude higher than a set threshold. A computer creates a loop with the recording and sent it back to an amplification driver. This amplification driver converts the recorded sound signal into a high voltage, low current inaudible signal. The output of the amplification hardware is connected to the conductive metallic casing of the microphone via a very thin, almost invisible wire wrapped around the microphone audio cable. When holding the microphone, the visitor comes in contact with the inaudible, high voltage, low power version of the recorded sound. This creates a modulated electrostatic filed around the visitors’ skin. When touching another person’s ear, this modulated electrostatic field creates a very small vibration of the ear lobe. As a result, both the finger and the ear together form a speaker make the signal audible for the person touched. The inaudible signal can be transmitted from body to body, using any sort of physical contact.

5. Value

This work convert digital audio to physical touch by adopting tricky but smart method that takes advantage of human body's characteristics. It encourages us to develop human's potential capabilities in interaction with digital information.

Reference: http://www.disneyresearch.com/project/ishin-den-shin/



Project #2: Glassified
by Anirudh Sharma, Lirong Liu and Pattie Maes at MIT Media Lab.
UIST 2013 Demo




1. Fields (Disciplines):
TUI, AR, HCI

2. Technology

Algorithms, openFrameworks, physics simulator, Wacom digitizer

3. Interaction

Glassified is a modified ruler with a transparent display to recognise strokes drawed on paper and present virtual graphics with physical strokes. It entitles both physical and digital objects to work together, enriching users to complete more tasks with a regular rules.

4. Design

Glassified prototype consists of a transparent OLED combined with a see-through ruler. A computer is connected to the ruler to do gesture/stroke recognition and sends commands to the display via RS232 communication. By using Wacom digitizer and certain API, it enables to distinguish the difference between the pixel density of the tracking system and the one of the display. On the computer, adopting several openFrameworks applications to solve various problems.

5. Value

Augmented Reality has been applied within Human-Computer Interaction since 1980s. This project shed lights on AR in daily tools, which inspires us to dig more about the inherent attributes and combine them properly with specific modern technology to serve people's daily work and life.

Reference: http://fluid.media.mit.edu/node/226