Intro Physical Computing - Final Project - "Garden"


The final project for my class Introduction to Physical Computing is this piece simply called “Garden”. It is a tactile display full of other-worldly textures, plants and animals meant to be explored and discovered using your hands.  Hidden amongst the beautiful elements are four subtle interactions which when triggered bring life to the piece. Interacting with our project is meant to be a joyful, age-agnostic experience as users feel wonderful textures, discover cute creatures, and light up the garden environment. In a way, our project “Garden” was envisioned and designed to be the antithesis of much of the other work being done at the ITP program. While often the focus is on projects built around a very visible technology, what my partner Julia and I were interested in doing was making an interactive experience where the technology itself was all but invisible and the pleasure comes instead from having an experience based on human’s inherent intuition of nature.

The four technical interactions are based around the cave, large rock, mushrooms and tall grass patch present in our exhibit. There is a small ‘fluff’ creature who lives inside of the cave, but he is very shy and if a user gets too close or reaches into the cave he will disappear into his hole. However if the user lifts up the large rock on the other side of the piece, two glow worms underneath light up and then turn off one by one, counting down until the ‘fluff’ leaves his cave and pops up under the rock where he is much more receptive to pats. Near the front of the project, there is a squishy purple fungus, and poking it many times in a row will light up a string of bioluminescent mushrooms which lead all the way around the piece to a grove of flowers in the back which twinkle with light. And finally, if a user brushes their hands through the tall patch of grass the branches in the back of the display wave back in forth in union to the motion of their hand.

When the planning first began for this project, Julia and I both brought with us an interest in making something inspired by nature. Essentially from the very beginning, we knew we wanted to make something with a series of subtle but intuitive interactions hidden amongst a natural environment. When we let our imaginations run wild, we briefly conceived of something like a forest room full of human scale interactions that users could explore at will. Of course within the limitations of budget and space we did immediately have to scale our idea down, but we have not let go of that possibility for the eventual future. 

When our plans transitioned to a more table-top scale, we began talking more specifically about the visual inspirations for this project. We felt that aiming for an alien-looking world unlike anything on earth would be ideal, considering any attempt to replicate or ‘improve on’ an existing environment felt like a slight to the natural world we were instead trying to celebrate. A major way we imagined we would create this otherworldly feeling was through the use of glowing and phosphorescent plant life, and we found major inspiration for this in the world building of the Avatar movies and the beautiful projection mapping videos made by Friedrich van Schoor and Tarek Mawad. I was also initially interested in bringing in some harder science fiction elements and for a while during the initial planning of the project we imagined including a combination of plants and deteriorating futuristic technology to project a very abstract post apocalyptic narrative. This imagery was heavily inspired by the video game Horizon Zero Dawn and similar ‘lived in’ science fiction worlds.


Once we had settled on a general design aesthetic we did a thorough brainstorm collecting as many ideas as possible for the potential interactions that would be integrated into the piece. Our final list included: bending a leaf, pulling a flower petal, planting a flower, lifting a rock, blowing a dandelion, turning a knob or button on a sci-fi object, stepping on a footprint, poking something, brushing a hand through grass, touching water, reaching into a cave or opening a secret compartment. We then took these ideas and created analog imitations of as many as possible to bring to a user test and gauge reaction. We were looking both for what was intuitive as well as to what people were most drawn. 

The feedback we received from these tests ended up being crucial for the final design. In the literal sense, it helped us to eliminate some interactions that ended up being hard for users to understand. But more conceptually these tests were key to us realizing that in order for the piece to feel unified we should focus specifically on interactions that were tactile. This realization also cemented the idea that the interactions were just one part of the larger project, which also would include purely analog textures and objects to touch. We also realized that including a science fiction artifact of some kind detracted from the purity of a nature-based experience. Instead we only wanted to include touch-based interactions that were intuitive based on known interactions with the natural world rather than with technological devices.

The final stage of refinement as we solidified the specifics of the concept involved the necessary trimming of some of the complexity of the project outputs. We wanted there to be things that were revealed only through patience or combined interactions. This concept certainly lived on in the final project through the ‘fluff connection’ between the cave and the rock as well as the requirement for continued input to activate the whole mushroom colony. However our initial ideas were much more fantastical (and hopefully possible in another project in the future). Things we had imagined included: an earthquake effect that caused a crack in the surface of the project to glow red and emit fog, seedlings that grew up from the ground and an object that would whisper secrets to you if you were close enough to it.

Because we had committed to the idea of a varied tactile experience and were interested in making as much of the project by hand, our list of materials was very long and the fabrication of the project took up a large chunk of our six weeks of work. The base of the whole project was made of plywood salvaged from the ITP workshop cut and screwed into a solid rectangular base. Then using roughly carved foam we created the basic topography before covering it all with plastic turf. Julia then cut away at the surface material in certain places before adding dust and fake moss to create the varied look of a natural landscape. They then made use of the most unusual, alien-looking parts of a large collection of fake flowers to decorate the landscape with details and make it feel somewhat familiar, but unidentifiable as a particular earthly place.

Besides the pink noodle-y worms we used as indicators for the various interactions, every remaining aspect of our project was hand made. The yellow mushrooms were made out of cast resin made in a silicone mold based on a series of individual clay models. They were then painted with silicone to give them a slightly springy texture. The glow worms under the rock and the purple input fungus were all made using a similar process, but instead with silicone cast in acetate molds. 


Julia made multiple kinds of flowers and plants as well. The light up ones were made by them using stretchy, colorful fabric cut, arranged and sown to give them their full, fluffy texture. Julia also made a series of yellow flowers with petals cut from a stiff yellow fabric. These were specifically designed to have two circular ‘faces’ made out of air-dry clay so that they would blend in with the flower of a similar design which instead had the two circular grills of an ultra-sonic distance sensor built into it. And Julia also hand-sowed each blade of grass into the grass patch which users would stroke to activate the waving branches in the back of the display.

The final component which really felt like the aspect that pulled it all together was the hand made, recycled paper ‘cement’ we used to make the log, cave and various small rocks throughout the display. Using an adapted recipe from fellow ITP student Katya Rozanova, we boiled shredded paper for about an hour to turn it into a fairly glutinous substance, before blending it with glue and corn starch to create a fairly thick paste. We then shaped this paste onto skeletons made of sculpting wire and aluminum foil before baking then in the oven. The final product came out in a lovely gray color which we left exposed on the small rocks throughout, and simply applied a coat of paint to the log and cave to make them stand out individually.

Bringing together the code and electronic components of the project was largely my task. The entire project is operated by an Arduino Nano and an Arduino Uno. The trail of mushrooms are the only components connected to the Nano. Under the purple fungus is an arcade-style button, and every time it changes state from off to on, a ‘mushroom counter’ is increased. Every time that counter increases it triggers a state in which an additional mushroom LED is lit. Every time the button is pressed, a timer begins counting the time between presses, and if that timer counts high enough, then the ‘mushroom counter’ is decreased by one. This serves to both force users to press the button fast enough, as well as turning the mushrooms off one by one once they decide to stop pressing. And when the mushroom counter reaches its maximum value a function is triggered which uses a cycling timer to light each of the three flowers one by one.


The cave interaction uses an ultrasonic distance sensor and the New Ping library by Tim Eckel to measure how close a user is to the cave. The code filters out the unavoidable errant values and using the clean input data to simply move the servo to hide the fluff if the sensor records the user within a certain distance.


The rock lift interaction is activated based on a light sensor underneath it, recognizing if it is covered by the rock or not. When the program is first powered up, an initial light value is recorded, and this is used as the baseline value against which future light amounts are measured to decide if the rock is lifted. When the light threshold is reached, a timer begins counting up and the LEDs in the glow worms turn off one by one according to that timer. The ‘fluff’ servo located in the cave also moves to hide that creature as it ‘makes its way over to the rock’. Finally once the timer reaches a certain value the servo under the rock moves, lifting the ‘fluff’ into view.


The grass patch interaction is powered by a short flex sensor hidden in one of the blades of grass. Using a modified version of the smoothing code example from the Arduino examples library, the smoothed value is recorded at the beginning of the loop, and then subsequently set as the ‘’old’ value at the end of the loop. If the next time through the loop the old value and new value are different enough (because a user has brushed their hand through the grass, thus flexing the sensor) then the three servos in the back of the piece are activated, moving them either to the left or the right, depending on the direction the grass was brushed.


Our project first showed at the ITP Winter Show and the reaction to it was everything Julia and I had hoped for. In most cases it took a minute to acclimate new visitors to the sturdy nature of our project, but as soon as users realized they were free to touch everything, the vast majority embraced the tactile exploration of the world we had created. There seemed to be a particular fondness for the liveliness of the fluff that ‘lives’ in our display as well as the pleasantly springy feeling of lighting up the mushrooms one by one. We did end up offering some guidance to most users about the locations of each interaction, but that did not seem to detract from the joy that came from realizing that by simply interacting with our display as they would with elements of nature that they could instigate playful, magical reactions from the piece.

All that being said, undoubtedly our proudest success was the overwhelmingly positive reaction from children. In a way they were the best users because they were so willing to disregard any aspects of the technology and simply experience it as a magical garden. Throughout the show, we had multiple children come back again and again to our display, often each time bringing with them a different family member to show them the discoveries they had made within our display. Fittingly because this project works so well for a younger audience, Julia and I are currently in preliminary discussions to show our piece at the Childrens Museum of the Arts in New York City in January or February 2019. This will hopefully just be the next of many steps for showing and improving this Garden and any similar works to come.

pcompStefan SkripakComment