User Interface Engineering - SS '17

An in-depth introduction to the core concepts of intelligent user-interfaces. The course primarily deals with machine analysis of human non-verbal behavior and its applications to human-computer, human-robot, and computer-mediated human-human interaction. Methods involve machine learning, deep learning and model based optimization.


eDoz Course Nr.
O. Hilliges, F. Pece
E. Aksan and C. Gebhardt
Thu 10:00 - 12:00, NO C 6
Thu 13:00 - 15:00, NO C 6
Office hours
coordinate via e-mail:


Course website online
Azure platform for the exercises is finally online. Check your email for instructions

Learning Objectives

Students will learn about fundamental aspects of modern intelligent user interfaces. After completing the course students will have acquired theoretical and practical knowledge about the most important problems in machine understanding of human behavior and how to leverage such understanding in the design of intelligent user-facing technologies.

The core competency acquired through this course is a solid foundation in machine learning and deep-learning algorithms to process and interpret human input into computing systems. In particular, students should be able to develop systems that deal with the problem of recognizing people in images, detecting and describing body parts, inferring their spatial configuration, performing action/gesture recognition from still images or image sequences, also considering multi-modal data, among others. Furthermore, students will be able to leverage models of human behavior in optimization based (algorithmic) design of user interfaces.


Wk.Date ContentSlides Extra Material
1 24.02.

Introduction to class contents & admin

2 02.03.
ML for HCI Pt. I

Linear Models

slides slides (annotated)
ipynb ipynb (Azure)
3 09.03.
ML for HCI Pt. II

Non-linear SVM & Decision Trees

slides slides (annotated)
4 16.03.
ML for HCI Pt. III

Ensemble Methods

slides slides (annotated) pptx (wth videos)
5 23.03.
Dynamic input


slides slides (annotated) hmm-solution hmm-ipynb (Azure)


There will be 3 exercises (2 homework assignments and 1 case study) and one multi-week project. The exercises will constitute 40 % of the final grade. Assignments have to be completed individually. It is ok to discuss with your team members but you have to write your own code.

Exercise sheets and solutions will only be accessible from within the ETH network.

Exercise Assignment Solution Due date
Exercise 1 Slides Jupyter Notebooks Assignment Page

Register at Kaggle:


Case Study

We will do one in class case study, simulating a program committee meeting. This is a mandatory and graded part of the course requirements.


The performance assessment is an oral exam conducted during the examiniation session (Jul-Aug). It will constitute 60 % of the final grade.