The Curse of Dimensionality

August 17, 2018

Dov KatzYou can find more information about the following topic on my website

The curse of dimensionality refers to various phenomena that arise when analyzing and organizing data in high-dimensional spaces (often with hundreds or thousands of dimensions) that do not occur in low-dimensional settings such as the three-dimensional physical space of everyday experience.” (

This is a funny definition if I’ve ever seen one. It implies that our everyday experience is low dimensional and therefore the data associated with it is easy to analyze and organize. But if the 3D world we live in was so simple, why is AI struggling to make sense of it?

The answer is that the above definition is both right and wrong. It’s wrong because the world we live in is very very very high dimensional. Sure, our space is three dimensional. But our visual perception of it is composed of millions of pixels refreshed multiple times per second. Therefore our visual data of the simple 3D space around us actually has millions of dimensions.

At the same time, this definition is exactly right. Or at least, it exposes a brilliant truth: it must be possible to organize the data pertaining to the world around us in a low dimensional form. Otherwise, how can any intelligent creature with finite resources (read: humans) make sense of it?

One of my favorite example is the dimensionality of a line. We all remember from school that a line can be described using the equation y=ax+b. In short, a line can be described using two parameters: a & b. Now, imagine tilting your head such that the line rotates to be parallel with the ground (or: x axis). Now, you can describe the line using a single number: y=c (or: the height of the line parallel to the x axis).

This simple example illustrates that perspective or representation is key. If you look at the data the right way, a high dimensional state space becomes lower dimensional. And, in the lower dimension representation, problems are easier to solve, and it becomes possible to make sense of a high dimensional world.

This article by Dov Katz found here:

Share this :

The Scientist in the Crib

August 09, 2018

Humanity’s advantage over the rest of the biosphere is out amazing abaility to learn and adapt. Alison Gopnik, Andrew N. Meltzoff and Patricia K. Kuhl. wrote a book called “The Scientist in the Crib” that discusses how babies learn.

Scientists often start with a crude, intuitive understanding of a phenomena. They then explore it by conducting some experiments. But, to conduct insightful experiments, scientists spend considerable amount of time learning what others have done. Typically, a scientific experiment is the last floor in a very tall building, sometimes extending decades or even centuries back.

So how do children learn? They begin with some crude knowledge provided by our genes. For instance, researchers have demonstrated that newborns already understand the concept of momentum. If you show a newborn a moving object that disappears behind a blanket, they expected it to reappear on the other based on its velocity and direction.

Of course, there’s only so much knowledge that can be hardcoded. Most things have to be learnt. Children appear to be spending much time playing. But, really, they are conducting serious experiments. What happens when I paint the wall? Will the glass break if I drop it? Is twisting the doorknob the right way to move the door?

Finally, there are teacher, also known as “parents”. The world is pretty complex. Making sense of it by running experiments alone isn’t practical. A parent, however, can give us a shove in the right direction to maximize our learning. This is what researchers refer to as “structure”. When you have a sense of how to organize things, discovery the logic behind a phenomena becomes easier.

The “Scientist in the Crib” does a good job describing these three components of learning: what we’re born with, what we learn from experimenting, and the role of teachers. It includes some fascinating examples demonstrating we are born with quite a bit of knowledge.

An interesting topic in the book is the brain’s plasticity. This is the notion that our brain is extremely adaptive. For instance, Japanese and English speakers are sensitive to different sounds. An adult speaker might not be able to hear a certain sound because of their cultural background. Babies, however, are able to hear all sounds regardless of their culture. This demonstrates brain plasticity — because culturally we don’t need to distinguish between two sounds, the adult brain adapts and can no more hear the unnecessary sound,

If you find this topic interesting, I highly recommend reading the book!

Check out my social media profile: for more information. Also, my personal webpage:

Dov KatzSource:

Share this :

Dov Katz – A Former Oculus’ Computer Vision Head

August 08, 2018
Dov Katz, having a stellar academic credential was hired as one of the senior computer vision engineers for Oculus. It is a large engineering office in Seattle and in fact, one of the leaders in virtual reality technology. Oculus was purchased by Facebook for about $2 billion in cash and stock in 2014. Being a senior computer vision engineer, Katz played a huge role in the development of the Oculus Rift headset. Because of his work and expertise, he was profiled in the Times of Israel story in 2014.  Publications by Dov Katz can be read here on his website.

Katz led the team that developed the Oculus Rift headset, a 3D virtual reality device that provides a 100-degree field of view when worn. It stretches the virtual world beyond the peripheral vision providing an immersive experience. It is a breakthrough in the virtual reality world. From senior engineer, Dov Katz was promoted to computer vision head. Today, Katz is no longer connected to Oculus but continues to further enrich his expertise in such industry.
Dov Katz graduated from Tel Aviv University with a degree in computer science and electrical engineering. He earned his masters and Ph.D. from the University of Massachusetts Amherst and had postdoctoral fellowship at the Robotics Institute of Carnegie Mellon University in Pittsburgh.

Additional News About Dov Katz
Share this :