06  /  The Labs

Try what websites do next.

The Labs is our research studio. Four experiments run right here in your browser. Play with your hand in front of your camera, or with a mouse or a finger.

Play without a camera
  • Camera optional. Every experiment also works with a mouse or a finger.
  • Stays on your device. The video never leaves it. Nothing is uploaded or recorded.
  • Made for business. Each one is a real technique we can build into your website.

Experiment 00

Wave at TAM.

  • CameraHold up your open hand and wave it side to side. TAM spots the wave and waves back.
  • Mouse or fingerPress the big button to say hi and he waves back. He follows your pointer, too.

Press the button to say hi, or turn on the camera and wave.

Waves back 0

Experiment 01

Made of light.

  • CameraSweep your open hand through the headphones to scatter the light. Make a fist to pull it all into your hand, then open it to throw it back.
  • Mouse or fingerMove through them to scatter the light. Press and hold to pull it in, then let go.
A pair of white headphones with orange ear cushions on black

Move through the headphones, or turn on the camera and use your hand.

Experiment 02

Make waves.

  • CameraSkim the water with your fingertips to make ripples. Pinch for a big drop. Close your hand to calm it.
  • Mouse or fingerMove across the water to make ripples. Click or tap for a drop.

Move across the water, tap it, or turn on the camera and skim it with your fingertips.

Experiment 03

Take it apart by hand.

An air purifier in real-time 3D, run by four hand moves. Each move does one thing, and the panel shows what the camera reads. No camera? Drag the unit to turn it, slide Apart, or pick a part.

  1. 01 Open your hand The five parts spread apart and their names appear.
  2. 02 Close it The parts slide back together into one unit.
  3. 03 Pinch and move Thumb and first finger together, then move sideways: the unit turns.
  4. 04 Point up or down Point high for the top parts, low for the bottom. That part lights up.

Try thisOpen your hand slowly and watch the names arrive, then point at the fan.

An air purifier taken apart: pre-filter, particle charger, collector plates, carbon tray and fan

Autopilot

Drag the unit to turn it, or turn on the camera.

How it works

From a camera frame to a moving part, in the browser.

  1. On your device

    The camera frames stay in your browser. Nothing is uploaded, saved or recorded.

  2. 21 points

    A hand model finds the wrist, knuckles and fingertips, many times a second.

  3. Read as moves

    Distances and angles between the points become open, closed, pinch, point and wave.

  4. The page answers

    Each move drives the robot, the light, the water or the 3D model the moment your hand moves.

For your business

The same research, working for you.

A camera and a browser are all it takes. A few things we can build with them.

Try before you buy

Glasses, hats or a paint color, seen on your own face or your own wall through the camera.

Touch-free screens

Displays at events and in stores that people control with a wave, no touching required.

Product demos people remember

Your product explained through something people do with their own hands, so it sticks and gets shared.

Questions

About the camera.

Is my video sent anywhere?

No. The hand tracking runs inside your browser on your own device. The video is never uploaded, saved or recorded, and the camera turns off when you press the button or leave the page.

Do I need a camera?

No. Every experiment also works with a mouse or a finger. The camera is there for the fun of doing it with your hand.

What does it download?

The first time you turn the camera on, your browser downloads the hand tracking model, about 8 MB, from Google's public MediaPipe library. After that it is usually cached.

Does it work on phones?

Yes, on recent phones, with the front camera. Prop the phone up so your hand is free, or use your finger on the screen instead.

It does not see my hand. What should I do?

Turn your palm to face the camera, hold it about an arm's length away, and make sure the room is well lit. One hand at a time works best.

Could this run on my website?

Yes. Everything on this page is standard web technology that runs in any modern browser, so it can go on your site as a section or a whole page.

Next step

Build what's next.

Have an idea that needs a camera, a gesture or a product people can take apart? Tell us about it.