Spatial Computing Course Syllabus

Spatial Computing Course → Syllabus / Materials

Current Semester: IXG-6185-A SVA MFA ixD
Professor: William Martin (amartin1@sva.edu)
Course Materials

Description

“Spatial computing” is a field of human-computer interaction that enables real objects in physical environments to act as the means of interactivity with machines, in particular, by referring to objects that have significance to humans.

Spatial computing has a number of related concepts, including tangible interfaces, ambient computing, responsive environments, augmented reality, etc. In this course, we will attempt to create our own experiments in spatial computing given:

1. concepts that define “space” (location, occupancy, etc), and
2. interactive ideas afforded by everyday physical objects.

Recently, the term “spatial computing” has been revivified by Apple with its release of the Apple Vision Pro product. While interesting as a device, we will focus more on the rudiments of spatial computing rather than the specific implementation from any given software vendor.

The experiments we will conduct will use modern artificial intelligence tools to abstract away the technical complexities inherent to spatial problems. We will, however, approach this carefully.

In 2022, OpenAI’s ChatGPT and DALL-E popularized the latest milestones in computational techniques called “generative artificial intelligence,” which have since captivated the world with remarkable capabilities. Simple text prompts can now appear to write essays, perform deep research, reason deductively, generate striking imagery and video, and even author and debug code.

Today, AI has many more flavors than just chatbots, including projects like AlphaFold that aim to provide a key to curing the most pernicious diseases.

So-called “AI agents” are performing even more complex tasks – often with questionable quality – that are disrupting the nature of work, upending our notions of workplace roles, and forcing us to refine our understanding of the value of human labor and authorship.

At the same time, AI already pervades the built environment, the everyday “space” around us. Government agencies partner with companies like Flock.ai to track and identify cars on public roadways; autonomous vehicles and dog-like robots can continuously map environments in real time; environmental sensors record changes in air quality and climate; tech companies listen to and monitor our homes; and satellite data feed increasingly sophisticated predictive models of urban and natural growth and destruction.

The infrastructure that powers hosted generative AI services (and the training the supporting machine learning models require) is having unquestionable impacts on the environment at large and the communities in which those data centers exist, including threats to drinking water supply, inescapable noise pollution, power consumption that threatens legacy electric infrastructure, and political fallout.

At the frontier of AI’s capabilities is spatial design. Roboticists in particular are making strides in advancing computational means to navigate physical environments. Spatial designers (architects, urban planners, etc.) face particularly complex problems when proposing changes to the built environment, a task which at best requires anticipating the consequences of such decisions, for the inhabitants of our buildings and cities in its immediacy and, longer term, economically and socially.

Therein lies an opportunity for interaction designers as well. Regardless of the overwhelming trend to digitize our lives and centralize our attentions, humans still are inherently spatial beings and have persistent intuitions of how physical objects behave in physical space. The richness of experience in the physical environment – with its smells, sounds, textures, materiality, gravity, sights – still outstrips any digital one, and interaction designers are well poised to expand their expertise by reclaiming and redefining what tangible interactivity should be.

This instance of Spatial Computing asks, How might we, interaction designers, leverage AI agents as a means of prototyping spatial design ideas?

In this course, students will:

• explore the definitions, affordances, and inner workings of generative and discriminative artificial intelligence,
• scrutinize canonical writings from relevant technological, architectural, and computational theories regarding notions of “space,”
• experiment with the rapidly evolving landscape of AI methods, including spatial / vision language models, computer vision algorithms, AI agents, robotics simulations, and more,
• develop a critical and technical understanding of the technologies, and
• speculate on interactive spatial ideas at the human scale, in particular, a desktop experiences involving multiple users in physical space.

This course will leverage AI agents frameworks (Hermes, Claude Code, OpenCode, etc.) to create a spatial simulation. Each week, new spatial concepts will be introduced to enrichen this simulation.

Class sessions each involve a brief lecture, an intensive technical workshop, and student presentations, supplemented with readings and media in-between sessions.

Setting Expectations

This course supports an environment to experiment, invent, and develop a series of ideas together. It’s less about becoming an “expert” in software skills.

Expect more questions than answers. The course uses several frameworks to structure our experiments. We will ground them with a conceptual and technical understanding of today’s AI technologies alongside various notions of “space,” using them as creative inspiration. We will learn by doing.

Expect that AI will be inconsistent and nondeterministic. If your projects aren’t producing consistent results, this is expected. We are researching and inventing, meaning we’re investing our time and effort, but we don’t quite know what might result.

This course is quite technical, so keep in mind that you’ll need to invest the proper effort to keep up. We do have generative coding tools at our disposal that will help, but they aren’t perfect.

The hope for the course is that you will understand how AI really works and what it really is, and that you will gain a fundamental perspective on spatial design relevant to architecture and urbanism, one that presents an alternative to the tool-based approaches that are perhaps more akin to data journalism or data science.

Learning Objectives

By the end of the course, you will:

• be familiar with spatial computing and its concepts,
• be able to propose the appropriate spatial computing methods for a given design problem,
• innovate on interactive spatial design problems as a unique addition to your skillset,
• program an AI agent to prototype interactive spatial ideas.

Sessions

† Weekly topics subject to change

Course Policies

Sessions

Every session involves a short lecture, but most of the sessions will be workshops. Students should come prepared with a laptop and all “Tech Prep” work completed before class.

For these workshops, please do the following:

• Do your best to remove distractions. Close email and messaging clients (except the course’s Discord server).
• Work on Spatial Computing and not other courses. Why would you just displace the time spent? Why not learn Spatial Computing during Spatial Computing? 🤯
• Do your best to keep up. Workshops will move quickly, and it’s quite important to participate and become familiar with these tools.

If I perceive that you are not participating in course sessions due to distraction or working on other courses, I will ask you to leave the session and you will be counted absent.

Auditing

I allow auditing as long as auditing students do not interfere with the learning of enrolled students. This means in any projects, you should first look to other auditors to form a team. For workshops when there are no other auditors, please join a pair of enrolled students to make a team of three.

(In my experience, auditing students rarely actually audit past week three of any course, so do think seriously about it. In universities, time is still money.)

Grading

A note on the difference between a P and HP grade in my courses:

• If you complete all the assignments as written, the submissions make sense, and the submissions represent genuine original thoughtful work (e.g. no unexplained ChatGPT screenshots), that’s a P.
• If the quality of your work demonstrates exceptional mastery, novel discoveries, or contributions that exceed expectations, that’s a candidate for an HP grade.