How Machines Learn

On the internet, the algorithms are all around you. You are watching this video because an algorithm brought it to you (among others) to click, which you did, and the algorithm took note. When you open the TweetBook, A the algorithm decides what you see. When you search through your photos, A the algorithm does the finding. Maybe even makes a little movie for you. When you buy something, A the algorithm sets the price and A the algorithm is at your bank watching transactions for fraud. The stock market is full of algorithms trading with algorithms. Given this, you might want to know how these little algorithmic bots shaping your world work, especially when they don’t. In Ye Olden Days, humans built algorithmic bots by giving them instructions the humans could explain. “If this, then that.” But many problems are just too big and hard for a human to write simple instructions for. There’s a gazillion financial transactions a second, which ones are fraudulent? There’s octillion videos on NetMeTube. Which eight should the user see as recommendations? Which shouldn’t be allowed on the site at all? For this airline seat, what is the maximum price this user will pay right now? Algorithmic bots give answers to these questions. Not perfect answers,
but much better than a human could do. But how these bots work exactly,
more and more, no one knows. Not even the humans who built them, or “built them”, as we will see… Now companies that use these bots
don’t want to talk about how they work because the bots are valuable employees. Very, VERY valuable. And how their brains are built is a fiercely guarded trade secret. Right now the cutting edge is most likely very ‘I hope you like linear algebra’, but what the current hotness is on any particular site and how the bots work,
is a bit “I dunno”, and always will be. So let’s talk about one of the more quaint but understandable ways bots CAN be “built” without understanding how their brains work. Say you want a bot that can recognize
what is in a picture. Is it a bee, or is it a three? It’s easy for humans (even little humans), but it’s impossible to just tell a bot
in bot language how to do it, because really we just know
that’s a bee and that’s a three. We can say in words what makes them different,
but bots don’t understand words. And it’s the wiring in our brains
that makes it happen anyway. While an individual neuron may be understood, and clusters of neurons’ general purpose vaguely grasped, the whole is beyond. Nonetheless, it works. So to get a bot that can do this sorting, you don’t build it yourself. You build a bot that builds bots,
and a bot that teaches bots. These bots’ brains are simpler,
something a smart human programmer can make. The builder bot builds bots,
though it’s not very good at it. At first it connects the wires and modules in the bot brains almost at random. This leads to some very… “special” student bots sent to teacher bot to teach. Of course, teacher bot can’t
tell a bee from a three either; if the human could build teacher bot to do that,
well, then, problem solved. Instead the human gives teacher bot a bunch of “bee” photos, and “three” photos, and an answer key to which is what. Teacher bot can’t teach, but teacher bot can TEST. The adorkable student bots stick out their tongues, try very hard, but they are bad at what they do. Very, VERY, bad. And it’s not their fault, really,
they were built that way. Grades in hand, the student bots take a march of shame back to builder bot. those that did best are put to one side, the others recycled. Builder bot still isn’t good at building bots, but now it takes those left
and makes copies with changes in new combinations. Back to school they go. Teacher bot teaches – er, tests again, and builder bot builds again. And again, and again. Now a builder that builds at random,
and a teacher that doesn’t teach, just tests, and students who can’t learn, they just are what they are, in theory shouldn’t work, but in practice, it does. Partly because in every iteration, builder bot’s slaughterhouse keeps the best and discards the rest, and partly because teacher bot isn’t overseeing an old-timey, one-room schoolhouse with a dozen students, but an infinite warehouse with thousands of students. The test isn’t ten questions, but a million questions. And how many times does the test, build, test loop repeat? As many as necessary. At first students that survive are just lucky, but by combining enough lucky bots, and keeping only what works, and randomly messing around with new copies of that eventually a student bot emerges that isn’t lucky, that can perhaps barely tell bees from threes. As this bot is copied and changed,
slowly the average test score rises, and thus the grade needed to survive the next round gets higher and higher. Keep this up and eventually from the infinite warehouse (slaughterhouse) a student bot will emerge, who can tell a bee from a three in a photo it’s never seen before pretty well. But how the student bot does this, neither the teacher bot nor the builder bot, nor the human overseer, can understand. Nor the student bot itself. After keeping so many useful random changes,
the wiring in its head is incredibly complicated, and while an individual line of code may be understood, and clusters of code’s general purpose vaguely grasped, the whole is beyond, nonetheless, it works. But this is frustrating, especially as the student bot is very good at exactly only the kinds of questions it’s been taught to. It’s great with photos, but useless with videos or baffled if the photos are upside down, or things that are obviously not bees, it’s confident are. Since teacher bot can’t teach, all the human overseer can do is give it more questions, to make the test even longer, to include the kinds of questions the best bots get wrong. This is important to understand. It’s a reason why companies are
obsessed with collecting data. More data equals longer tests equals better bots. So when you get the “Are you human?” test on a website, you are not only proving that you are human,
(hopefully), but you are also helping to build the test to make bots that can read, or count, or tell lakes from mountains, or horses from humans. Seeing lots of questions about driving lately? Hmm…! What could that be building a test for? Now figuring out what’s in a photo, or on a sign, or filtering videos, requires humans to make correct enough tests. But there is another kind of test that makes itself. Tests ON the humans. For example, say entirely hypothetical NetMeTube wanted users to keep watching as long as possible? Well, how long a user stays on the site is easy to measure. So, teacher bot gives each student bot a bunch of NetMeTube users to oversee, the student bots watch what their user watches, looks at their files, and do their best to pick the videos
that keep the user on the site. The longer the average, the higher their test score. Build, test, repeat. A million cycles later, there’s a student bot who’s pretty good at keeping the users watching, at least compared to what a human could build. But when people ask:
“How does the NetMeTube algorithm select videos?” Once again, there isn’t a great answer other than pointing to the bot, and the user data it had access to, and most vitally, how the human overseers
direct teacher bot to score the test. That’s what the bot is trying to be good at to survive. But what the bot is thinking, or how it thinks it,
is not really knowable. All that’s knowable is this student bot
gets to be the algorithm, because it’s point one percent better than the previous bot at the test the humans designed. So everywhere on the internet, behind the scenes,
there are tests to increase user interaction, or set prices just right to maximize revenue, or pick the posts from all your friends you’ll like the most, or articles people will share the most, or whatever. If it’s testable, it’s teachable. Well, “teachable”, and a student bot will graduate from the warehouse
to be the algorithm of its domain. At least, for a little while. We’re used to the idea that the tools we use, even if we don’t understand them, someone does, but with our machines that learn we are increasingly in a position where we use tools, or are used by tools, that no one, not even their creators, understand. We can only hope to guide them with the tests we make, and we need to get comfortable with that, as our algorithmic bot buddies are all around,
and not going anywhere. OK. The bots are watching. You know what’s coming. This is where I need to ask you… To like… comment… …and subscribe. And bell me. And share on the TweetBook. The algorithm is watching. It won’t show people the video… unless you do this. Look what you’ve reduced me to, bots. What do you want? Do you want watch time? Is that what you want? Fine. (sigh…) Hey guys, did you know I also have podcasts you can listen to? Maybe even just in the background while you’re tidying up your all room for hours? Or whatever? There’s hours of audio entertainment for you,
and watch time for the bots overseeing your actions. Go ahead and – and take a click.
Entertain yourself. Help me. Help the bots.

100 thoughts to “How Machines Learn”

1. Tim Herrmann says:

Coment

2. Ferocious Twinkie says:

This scares me

3. Aryanna Hernandez says:

Yay

4. AMetalPenguin says:

COMMENT

5. Ozabyss says:

2017 na
2018 na
2019 yes

6. TheDeceptiveMoss says:

FEAR

7. Sir Kobalt says:

Hey! This almost doesn't look like Skynet

8. Ale - Alemán says:

Dude, I fucking love your bots animations, are just so great.

9. Craig Canale says:

After watching this video, this is the first time I'm actually liking, subscribing and turning notifications on

10. LilacPlushett says:

My recommendation bot is totally retarded and I love him

11. mewingkitty says:

"And share on the tweetbook" Yassssss lol

12. Uncle Creepy says:

So I have to watch more YouTube videos I like and stop falling asleep during shitty YouTube videos in order for the bots to actually suggest something I want to watch? Just so I can watch more YouTube videos?

13. Mika25797 HD says:

Wait why would a human filling in the are you human test help make the test for a robot why not just put the questions directly onto the test

14. Sadcat With ham says:

WOW Podcast!!

They're watching They're watching They're watching They're watching They're watching

15. murdoc4312 says:

Teacher bot can't teach, but teacher bot can test. sounds like what passes for people schools these days. this seem cool as long as the algorithms is helpful and does what the user. I tried Bing once, never again. I ask about Krystal burger in Orlando Fl., I got crystal caverns Arkansas And a dozen online new age crystal stores.

16. cgcoyote cow says:

Love you

17. PROBA Productions says:

What a horrible way of computing life …. no wonder it wants to kill us

18. decrisp1252 says:

They’re so CUTE!

19. TheYoRND says:

Voll dank bro

20. I deserve to die says:

Done

21. אור סגל ה2 says:

Not accurate

22. Vaz123 says:

You're wrong

I'm watching this video because those robots are adorable and I saw a link to plushies

23. mlouwagie says:

googleyoutube is giving us pages sites headlines in what ever we have typed anywhere…. to appease us…. to appease us …to appease us …into not demanding that they pay taxes or that we rise up againsted injustice….

24. Matthew Zacher says:

Comment.

25. Zalorin says:

A builder that that builds at random and a teacher that doesn't teach …. In theory shouldn't work
Evolution… takes a sip?

26. coding101 says:

I gave you watch time

27. CMP says:

7:20 I'm not sure if thats a ProZD reference, but I still find it funny.

28. Andy Brown says:

hi

29. andrei balatbat says:

how big is the plushie bots

30. Robin Lundqvist says:

we aren’t that far away from creating a super genius AI, are we?

31. ZALGO says:

Now I understand why I get random shit in my reccomended occasionally

32. An Turret says:

So this is how my AI works.

special …. word choice … key.!

34. Bread Biscuit says:

ay

35. ProtoNova says:

commented

36. Justin Huynh says:

Comment

37. Mocha says:

lol that's how science works. build a theory, test a Theory, come to a conclusion, rebuild a theory, test a theory, repeat

38. Hearts Of Iron 4 says:

Likes

39. paulw says:

What happens when the Bots leave college? Do they look for a job in YouTube.

40. Bruno Vásquez says:

If im geting this correctly, the reason youtube's copyright bots are failing is because companies have a bias when aproving the tests, letting false positives slide by and acepting bots that score a less than ideal as long as they aren't affected by them. If youtube keeps this bias in their tests, false and faulty copyright strikes will continue to plage the platform.

41. TheRainHarvester says:

Bots bip bop bee bop!

42. ice checksolvea says:

So because humans have pattern recognition, bots use natural selection?

43. Adrian Re says:

I feel like in the near future we might accidentally create consciousness

44. WildeBob says:

this video makes me sad because I just want to hug those tiny cute bots but I cant

45. Macaroon_Nuggets says:

Artificial selection.

46. Joan E Rotten says:

Sounds an awful lot like survival of the fittest A.I. style. Interesting

47. Nick Maerz says:

p

48. Minecraf Minecraft says:

The alagortim showed me this video

49. Lord death says:

Nice vid

50. Timmy Looeser says:

Ya

51. Lynn Fernandez says:

I really identify with the lil bot with the pencil in its eye. I feel you little dude, I also get confused about stuff

52. DANIEL SCHONS says:

0:03 well yes but actuli no

53. cloudcolumn cat says:

2:26 It's fun to understand this part of the video as people in public education.
In fact, it is sad. It's almost similar.. ;ㅡ;

54. That trap in the back of the van says:

ya well my recommendations are Russian babies playing with robomeck of woody

55. Barbod Jahed says:

1:51
"You look and you type what you think you see! Is it an E or is it a 3? That’s up to yeee…"

56. Brooks Carlson says:

This is such a great video- well organized, easy to understand on a difficult topic that I didn't even realize was so difficult. Oh Grey, is there anything you can't do? I mean besides a face reveal.

57. Ian Layman says:

Builds an algorithm that explains how it can tell a bee from a three.
STONKS

58. Joel Playz says:

Hi

59. shivam trivedi says:

😂🤣

60. Rob Setters says:

What's annoying is when YouTube overrides the algorithm to remove videos that would have been recommended because it doesn't like them, or push videos that wouldn't be recommended #mainstreammedia because of its own political ideology/crony capitalist agendas.

61. Death nova says:

lel oof get booted out of school/life bots

62. Ray X says:

This was incredibly clear and informative honestly!

63. RickySTT says:

Ahem, an algorithm did not bring me to this video. I’m watching it because a friend posted another of your videos to Facebook, and I decided to look at your channel, which started playing this video automatically.

64. Lillian Larsen says:

WOW!!!!!!!!!!!!! SUPER COOL!!!!!!!

65. Tattle Boad says:

can machines learn about their own algorithms?

66. 1stLt Rob Miller says:

This is how Skynet becomes a thing

67. Sachin Dangi says:

Felt almost poetic…Great job Grey

68. Daniel saldana says:

The bot are sooo cute

69. Rhein Bewachen says:

Teaching the algorithm to not demonetize YouTube videos that don’t need to be demonetized.

70. Noname #lol says:

Lol

71. Remi Lalaque says:

I am a bot and learned lot of things with this video ! By the way, you actually identify bees with its wings ???!?!??

72. Failed Hivemind says:

Thanks for making me praise slaughterhouses.

73. Chris Nat says:

Now I'll watch many of the same videos bc I don't wanna be an indirect murder of bots

74. Marco Anaya says:

I always wanted to go to the learning slaughterhouse… 4:30

75. Liam S says:

My teacher used this video and it was good

76. Splatoon Squid Squad says:

comment

77. Jason Fjordlund says:

Great AI info!

78. Gulsum Uygur says:

For you

79. kripperinostealsyourmemeserino says:

Pot casts

80. Snakinator says:

After seeing this I made sure to give those "are you human" tests a few incorrect answers because apparently I'm malicious

81. ThisIsCam says:

I advertise on Facebook, it can be a love/hate relationship with the bots.

For example, an ad set that has nondescript errors on some of the ads (thus not running them) will do great (on the ones that did run) high click-thru, good watch time, etc. But when I run the same ad set with the nondescript problems fixed it barely runs it.

82. Nicolas Rodriguez says:

It's 2019, but somehow, i spot the bee at 1:04

83. Maruthi Macha says:

Are you there CGP Grey?

84. BlindingLight 7 says:

That’s a lotta little bois

85. Well Wisdom says:

The scary thing is, that not a single "Tailor fit" ads are remotely accurate for me.

86. Pig_master 101 says:

How are you supposed to make a builder bot!?!

87. The Cereal Guy says:

Interesting

88. Connor McCormick says:

Are you still making vids?

89. Lillian Larsen says:

How do you build a bot.

90. Payne Persons says:

The bots want me to comment so I will do that. And your video was fun. Its comment worthy.

91. Aaron Oquendo says:

🔥

92. Brian Reher says:

This is a great intro to ML. If this is something that interests you, Kaggle is a good starting place.

93. Cyan Mino says:

i was gonna to comment "I NEEED MERCH OF THE ALGORITHMS!!!"
and suddenly a wild plushie appeared…
gosh, this duds know how to do their job…

94. Renegade Fusion says:

I just realized that I have my own little cute bot on YouTube! 🤖

95. Sean Lawrence says:

brilliant

96. Booted Kite says:

watching 24 hours of recommended videos to keep my bot alive

97. Myst stories says:

Selective breeding unless you use q deep learning

98. Ayyy LMAO says:

Hope our skynet will be benevolent

We still have 40 years to go

99. Arnold Schwarzenegger says:

AghaghFg